Abstract Trust is a multi-faceted phenomenon traditionally studied in human relations and more recently in human-machine interactions. In the context of AI-enabled systems, trust is about the belief of the user that in a given scenario the system is going to be helpful and safe. The system-side counterpart to trust is trustworthiness. When trust and trustworthiness are aligned with each other, there is calibrated trust. Trust, trustworthiness, and calibrated trust are all dynamic phenomena, evolving throughout the history and evolution of user beliefs, systems, and their interaction. In this paper, we review the basic concepts of trust, trustworthiness and calibrated trust and provide definitions for them. We discuss their various metrics used in the literature, and the causes that may affect their dynamics, particularly in the context of AI-enabled systems. We discuss the implications of the discussed concepts for various types of stakeholders and suggest some challenges for future research.

Software development relies on collaboration and alignment between a variety of roles, including software developers and user experience designers. The increasing focus on artificial intelligence in today's development projects has given rise to new challenges in this collaboration. We extend previous work on the process of designing human-AI systems by analysing collaborative practices between UX designers and AI developers through Mintzberg's theory on coordination mechanisms. We conducted 15 in-depth interviews with UX designers and AI developers currently working on AI projects. We contribute by identifying how coordination mechanisms impact the UX design process when developing AI systems , inter-team (a)symmetries in power relations, and a growing need for tools and cross-disciplinary knowledge to support these collaborative efforts. In particular, we outline the risks of coordinating AI development work through the standardisation of output and skills in separately organised UX and AI development teams. CCS Concepts • Software and its engineering → Collaboration in software development; • Human-centered computing → Collaborative and social computing.

Empathic medical conversation is central to patient-centered care within Narrative Medicine. However, difficulties, such as physicians’ limited empathic capabilities and lack of time, impede the practice. Research on real-time, on-site empathic medical exchanges has been limited in exploring technology to assist and enhance physicians’ capabilities. This paper proposed the Empathic Opportunity Perception and Distinction (EOPD) framework for building physician-AI collaboration based on Intelligence Augmentation (IA) for empathic conversations. The EOPD integrates two multi-modal machine learning (ML) models based on facial and verbal cues, presenting a physician-AI interaction framework and three distinctive visualization components: emotional reference, opportunity reminding and keyword collection, and situation understanding. To assess EOPD's effectiveness and gauge physicians’ and patients’ receptiveness, a prototype system named EMVIS ( EM otional VIS ualization ) was designed and developed. Results from the study demonstrated improvements in physicians’ empathy efforts and perceived empathy performance when using EMVIS, particularly for junior physicians. Physicians and patients held positive attitudes towards EMVIS, with patients expressing a high expectation that EMVIS would improve the physician-patient relationship. The research showed the efficacy of the multi-modal ML models in supporting complex affective empathy and EMVIS in facilitating and complementing empathy concerns. It highlighted the tailored support to junior and senior physicians and emphasized physician-AI collaboration to maintain user autonomy and mitigate potential biases. Future research should explore extensive system applications, tailor visual and interactive support for physicians, and implement adaptive and reflective ML models to improve the effectiveness and efficiency of empathy communications.

Abstract Carbon emissions from transport are increasing on a global scale with transport accounting for the largest portion. While there is greater awareness of climate change, few people are willing to change their travel habits. To gain a better understanding of this phenomenon, we were motivated to conduct a study to identify the obstacles encountered by travellers when opting for sustainable transport modes, assess their knowledge of carbon emissions from various transport modes and explore how this knowledge affects their decision-making process. The study employed mixed-method approaches, including the use of informational videos and gamified quizzes for intervention, pre- and post-intervention knowledge assessment tests, semi-structured interviews, observations, Self-Assessment Manikin for emotion measures and card sorting. The findings primarily derived from thematic analysis indicated that knowledge-based intervention can be a strong catalyst to encourage sustainable attitude and behaviour change. Insights so gained have important theoretical and practical implications for designing sustainable behaviour change in the future, especially the deployment of persuasive technology.

While chatbots are increasingly used for customer service, there is a knowledge gap concerning the impact of Conversational Breakdown in such chatbot interactions. In a 2 \({\times}\) 4 factorial design online experiment, we studied how Conversational Breakdown impacts user emotion and trust in a chatbot for customer service, given variations in task criticality and breakdown task order. Here, 257 participants were randomly assigned to complete high- or low-criticality tasks with a prototype chatbot for customer service, experiencing Conversational Breakdown for the first, second, third or none of their tasks. The task set was decided from a 63-participant pre-study. We found significant impact of Conversational Breakdown, including a marked order effect on overall trust, as well as a bounce-back effect on task-specific trust and emotion after subsequent successful task completion. We found no post-interaction effect of Task Criticality. Based on our findings, we discuss theoretical and practical implications and suggest future research.

Remote conferencing systems are increasingly used to supplement or even replace in-person teaching. However, prevailing conferencing systems restrict the teacher’s representation to a webcam live-stream, hamper the teacher’s use of body-language, and result in students’ decreased sense of co-presence and participation. While Virtual Reality (VR) systems may increase student engagement, the teacher may not have the time or expertise to conduct the lecture in VR. To address this issue and bridge the requirements between students and teachers, we have developed Tutor In-sight, a Mixed Reality (MR) avatar augmented into the student’s workspace based on four design requirements derived from the existing literature, namely: integrated virtual with physical space, improved teacher’s co-presence through avatar, direct attention with auto-generated body language, and usable workflow for teachers. Two user studies were conducted from the perspectives of students and teachers to determine the advantages of Tutor In-sight in comparison to two existing conferencing systems, Zoom (video-based) and Mozilla Hubs (VR-based). The participants of both studies favoured Tutor In-sight. Among others, this main finding indicates that Tutor In-sight satisfied the needs of both teachers and students. In addition, the participants’ feedback was used to empirically determine the four main teacher requirements and the four main student requirements in order to improve the future design of MR educational tools.

The digital divide is especially pertinent in Majority World Countries (MWCs), and this was exacerbated greatly by the pandemic. Tackling the digital divide underpins the work of Human–Computer Interaction for Development (HCI4D) and remains an important global endeavour. Our project aimed to understand how children and young people (CYP) in MWC coped during the pandemic and how technology played a role. Voices of CYP were complemented by those of their parents and professionals with whom CYP interacted regularly. Our empirical study involved 73 CYP and 76 adults from Brazil, Kenya, Pakistan, South Africa, and Turkey. Qualitative data from diaries, drawings and focus groups were analysed thematically. Four major themes were identified – ‘access’, ‘usage’, ‘risk’, and ‘future Among others, some intriguing findings were that CYP acutely felt the peer pressure on ICT ownership and tended to direct frustration at parents, who grappled with their untenable roles as gatekeepers to digital worlds. Implications for addressing the digital divide include long-term strategies to improve infrastructures and mobilise community-based collaborative efforts and enhance digital literacy.

Chatbots represent a promising approach to provide instructional content and facilitate active learning processes. However, there is a lack of knowledge as how to design chatbot interactions for active learning. In response to this knowledge gap, we conducted an experimental study (n = 164) comparing four modes for providing instructional content in chatbots, with varying demands for cognitive engagement. The four modes – passive, active, constructive, and interactive – were based on the ICAP framework of active learning. The learning content concerned identification of phishing emails and the four modes were distinguished by how the participants were invited to engage with the content during their chatbot interaction. The ICAP modes of higher cognitive engagement required participants to spend more time on the interaction and led to perceptions of higher subjective learning outcome. However, the effects of the different ICAP modes were not found to be significantly different in terms of user engagement, social presence, intention to use, or objective learning outcomes. The study represents an important first step towards understanding the design of chatbots for active learning.

Abstract Multimodal emotion detection has been one of the main lines of research in the field of Affective Computing (AC) in recent years. Multimodal detectors aggregate information coming from different channels or modalities to determine what emotion users are expressing with a higher degree of accuracy. However, despite the benefits offered by this kind of detectors, their presence in real implementations is still scarce for various reasons. In this paper, we propose a technology-agnostic framework, HERA, to facilitate the creation of multimodal emotion detectors, offering a tool characterized by its modularity and the interface-based programming approach adopted in its development. HERA (Heterogeneous Emotional Results Aggregator) offers an architecture to integrate different emotion detection services and aggregate their heterogeneous results to produce a final result using a common format. This proposal constitutes a step forward in the development of multimodal detectors, providing an architecture to manage different detectors and fuse the results produced by them in a sensible way. We assessed the validity of the proposal by testing the system with several developers with no previous knowledge about affective technology and emotion detection. The assessment was performed applying the Computer System Usability Questionnaire and the Twelve Cognitive Dimensions Questionnaire, used by The Visual Studio Usability group at Microsoft, obtaining positive results and important feedback for future versions of the system.

The ongoing transition within tech industry towards holistic service journey orchestration pushes user experience design (UXD) closer to service design (SD). At the same time, new digital channels for traditional services pushes SD closer to UXD. The increasing interplay between UXD and SD can cause confusion about their relationship, boundaries, and responsibilities. Little research has examined perceptions of these aspects in industry and academia. We surveyed 197 UXD and SD practitioners and academics about their understanding of the overlaps and boundaries between the two design practices. Our findings reveal several conflicting views around the scope of UXD and SD work, the subsumptive relation between UXD and SD, and the theoretical roots and methods each draws upon. We suggest constructive ways for the two design practices to complement each other. Our study shows that UX work is changing, and understanding this change is important to maintain the relevance of UX research.

While voice communication of emotion has been researched for decades, the accuracy of automatic voice emotion recognition (AVER) is yet to improve. In particular, the intergenerational communication has been under-researched, as indicated by the lack of an emotion corpus on child–parent conversations. In this paper, we presented our work of applying Support-Vector Machines (SVMs), established machine learning models, to analyze 20 pairs of child–parent dialogues on everyday life scenarios. Among many issues facing the emerging work of AVER, we explored two critical ones: the methodological issue of optimising its performance against computational costs, and the conceptual issue on the state of emotionally neutral. We used the minimalistic/extended acoustic feature set extracted with OpenSMILE and a small/large set of annotated utterances for building models, and analyzed the prevalence of the class neutral. Results indicated that the bigger the combined sets, the better the training outcomes. Regardless, the classification models yielded modest average recall when applied to the child–parent data, indicating their low generalizability. Implications for improving AVER and its potential uses are drawn.

Full Paper Chairs of the British Human-Computer Interaction Conference (BHCI) 2021 This special issue is a collection of nine papers, which are the expanded versions of the best papers presented at the 34th British Human–Computer Interaction Conference, 19–21 July 2021. The conference was held entirely online due to the pandemic. The theme of the conference ‘Post-pandemic HCI – Living Digitally’ aimed to raise hopes in the Human-Computer Interaction (HCI) community about an eventual end to the pandemic and to invite reflective as well as visionary thinking about how post-pandemic lives could or should be with (or without) technology. Two distinct scenarios may happen as the pandemic subsides. Some people discard the technologies that they have been obliged to adopt during the pandemic because they yearn to resume pre-pandemic routines and practices (e.g. in-person learning activities at schools). In contrast, some people become so reliant on the technological solutions they have utilized during the pandemic that they continue deploying them, despite being advised otherwise (e.g. working from home). Others may opt for variants of these two contrasting scenarios. Regardless of people’s preference on how dominant technologies should play in their everyday lives, the ever-increasing digitalization of essential services (e.g. health, transport, education) renders it inevitable that people’s well-being is shaped and regulated by digital technologies (Scoglio et al., 2019). In fact, the heavy usage of social media during the pandemic is a double-edged sword for psychological well-being. On the one hand, such platforms enable people to remain connected, sustaining their social and emotional well-being (e.g. Goldschmidt, 2020). On the other hand, the same platforms can exacerbate the vulnerability of those who are prey to manipulation by online predators (e.g. Williams et al., 2020). Indeed, one of the post-pandemic grand challenges is to identify effective means to improve people’s psychological well-being, which has been undermined by the negative experiences of the pandemic. Research on understanding the intricate relations between well-being and technology has visibly grown in the past decade (e.g. Cheng et al., 2019; Scoglio et al., 2019), which is strongly linked to the work on positive psychology (e.g. Seligman, 2012) and happiness studies (e.g. Veenhoven, 2008). Presumably, the research area of positive computing posited by Calvo and Peters (2014) has promoted this line of inquiry in the field of HCI. From the design perspective, the frameworks put forward by Desmet and colleagues (Desmet & Hassenzahl, 2012; Desmet and Pohlymeyer, 2013; Klapperich et al., 2019) have stimulated related research endeavours exploring the notion of eudaimonia and meaningfulness for HCI (e.g. Mekler and Hornbaek, 2016). Identifying robust theoretical and methodological frameworks for designing, developing and deploying digital technologies to enhance people’s (and other animal’s) well-being can arguably be the fourth wave of HCI (cf. the third wave HCI; Bødker, 2015). In short, the theme of the next HCI wave may be well-being-driven HCI. There exist alternative proposed themes, such as ‘Entanglement HCI’ (Frauenberger, 2019), which has inspired the soma-focused ‘more-than-human bodies’ approaches (Homewood et al., 2020). While we do not delve into these pioneering efforts on paving the ground for the next HCI wave, it is relevant to point out that their strong ethical views on HCI work resonates with the topic of well-being-driven digital technologies, which we take a closer look subsequently. Digital technologies, when properly designed, developed and deployed, can lead to pleasurable and meaningful everyday experiences, which in turn contribute to people’s psychological well-being or happiness (Lyubomirsky, 2007; Ryan and Deci, 2001). Psychological well-being can be broadly understood as the frequent experience of positive moments and a general feeling of being satisfied with life (Diener et al., 1999). As a multifarious concept, well-being should be understood from multiple perspectives, including psychological, sociological, philosophical, economic and technological (Amichai-Hamburger, 2009; Kahneman, 2011). Furthermore, research on well-being allows insights to be gained for policymaking, organizational management, gender equality and other societal impacts. Given the robust finding that happy people are healthier (e.g. Veenhoven, 2008) and more productive (e.g. Oswald et al., 2015), well-being can be considered as a highly desirable individual and collective goal, a yardstick for policy interventions, a central productivity factor for organizations and a key value proposition and market differentiator for products and services. Digital technologies, including those currently as widespread as smartphones, provide an infrastructure capable of enhancing well-being in many ways. Nonetheless, technology does not intrinsically have values (Amichai-Hamburger, 2009), but values are integrated into digital products and services through design (Verbeek, 2011; cf. value-sensitive design; Friedman et al., 2013). Moreover, depending on how users appropriate the original design goal, a digital product or service can have detrimental effects on well-being. Salient examples are the addictiveness of social media (e.g. Brailovskaia and Teichert, 2020) and excessive tracking of one’s health parameters with self-monitoring technologies (e.g. Storni, 2010), leading to debilitating anxiety and poor self-esteem. A key research challenge for the HCI community to tackle is how digital technologies can be designed to facilitate well-being. How do we fit the technical opportunities provided by digital technologies into people’s everyday lives? How can we make everyday life meaningful and enjoyable through interactive technologies? How can the burgeoning range of Artificial Intelligence (AI)-infused autonomous products and services be utilized to promote well-being? HCI researchers and practitioners often attempt to answer these questions with the established user-centred design (or human-centred design) (UCD/HCD) (ISO 9241-210:2019, n.d.). While UCD/HCD is a relevant starting point, it needs augmented as well as concretized to address the challenges for being not only human-centred but also well-being-driven design. These challenges have recently become more acute, thanks to high complexities and uncertainties of AI-infused systems (Yang et al., 2020). In the following, we discuss three main challenges. Developing models to understand the interplay between well-being and technology in context. To design for well-being requires foremost a deep understanding of what well-being is. While models of well-being in psychology and other social sciences (e.g. Kahneman, 2011) are inspiring, they are not concrete enough for designing interactive technologies for specific applications, especially AI-infused ones. It is imperative to develop models to summarize and systemize well-being-related knowledge, making it applicable to the design of well-being-driven technologies in diverse contexts. Identifying ethical, political and societal implications of well-being-driven HCI. Designing for well-being has deep ethical, political and societal implications. Current HCI and AI ethics work (e.g. Canca, 2020; Kisselburgh et al., 2020) need to be refined, given that well-being-driven design tends to realize certain values. The key ethical concern is whose values (Vallor, 2016). As different cultures, worldviews and people value different forms of well-being, it becomes a critical ethical choice what kinds of well-being to prioritize and drive with digital technologies. Furthermore, it is also important to ensure that the design is democratically legitimate and mindful of minorities, becoming a key political concern of well-being-driven HCI. While western societies tend to frame the pursuit of well-being as a private endeavour, it is strongly affected by collective systems and actions. Well-being-driven HCI should reconcile the pursuit of individual well-being with social responsibility towards collectives (Klapperich et al., 2019). Developing methods and tools for evaluating well-being-driven digital technologies. Understanding and positioning what well-being is and accepting it as a legitimate goal are important preconditions for well-being-driven digital technologies. On top of that, it is necessary to develop innovative and valid methods and tools to evaluate the user experience, user engagement and other specific well-being-oriented criteria of these technologies. For instance, how to measure validly and viably memory-based and moment-based experience (i.e. ‘experienced utility’; Kahneman, 2011), which is strongly linked to well-being (Soleimani and Law, 2017), is one of the challenges to tackle. The topics of the nine accepted papers for this special issue are related to the above discussions on well-being-driven technologies, albeit to different extents. Two of the papers have the word well-being in the title, with one addressing physical health (Goffe et al., 2022) and the other, mental health (Noronha and Campos, 2022). Indeed, health is a dominant domain for this batch of papers: two related topics are digital telecare (Taylor et al., 2022) and digital twins in healthcare (De Maeyer and Markopoulos, 2022). Apart from health, utilizing interactive technologies to improve education (Schaper et al., 2022) and transport (Goodman-Deane et al., 2022) contributes to well-being-oriented goals. Putting in a broader context, all these essential everyday services are only sustainable in a peace time when safety is assured, but sadly, this very basic existential condition cannot be taken for granted even in the 21st century (NB: the Ukraine war is still raging at the time of writing this editorial). The paper by Paracha et al. (2022) on peacekeeping is particularly timely and relevant. The remaining two papers (de Quincey and Mitchell, 2022; Wright and Kefalidou, 2022), while not directly addressing well-being, can have methodological implications for well-being research through their user experience-focused studies. In ‘Web augmentation for wellbeing’, Goffe et al. (2022) grounded their work in the concepts of digital civics and public health. Based on the empirical data gathered over a sustained period of 12 months on users’ food-ordering behaviours and experiences with a dedicated platform, the authors proposed modifications to the platform. The authors combined the participatory UCD approaches with the web augmentation method to address the goal of enabling users to make healthy food choices. While the impact on well-being is yet to demonstrate, the case study provides methodological insights. In ‘Harnessing virtual reality nature to promote well-being’, as the title indicates, Noronha and Campos (2022) aimed to demonstrate the utility of immersive experience enabled by virtual reality (VR) for improving people’s mental health through relaxation exercises. The nature scenes in VR proved effective in increasing people’s calmness while decreasing their arousal and energy level. This work strengthened the assumption that VR can be a well-being-enhancement tool. In ‘Supporting safe and independent living’, Taylor et al. (2022) identified the innovative digital telecare concepts for older adults by applying the UCD approaches and synthesizing user engagement findings. Apart from the identified themes for future work in the field-community-based support, the authors illustrated comprehensively the process of gathering users’ needs and translating them into actionable design concepts. In ‘Digital Twins in healthcare’, De Maeyer and Markopoulos (2022) reported how experts in digital transformation of healthcare envisaged the application of the emerging digital twins technology. In addition to practical insights on exploring the potential of this technology for personalized well-being, methodological insights on the use of Delphi study for eliciting expert views were obtained. Paracha et al. (2022) stated a lucid message: for the digital transformation of peacekeeping, it is recommendable to put humans before technological solutions. Using the value-sensitive design approach (Friedman et al., 2013), the authors argued for integrating human values into technological designs. Several core values were identified across three use contexts, including peacekeepers’ well-being. This work lent further evidence to the importance of human-centred technological interventions. In the same vein, Schaper et al. (2022) demonstrated how they successfully applied the UCD approaches to digitizing iteratively an analogue syllable-based reading intervention. Specifically, they conducted expert reviews and interviews with end-users (primary school pupils) to gather concepts for digital prototypes, which were usability evaluated. Goodman-Deane et al. (2022) aimed to address the ethical concern of inclusiveness for digital mobility services by paying particular attention to people with diverse backgrounds. As an integral part of the UCD process, the authors captured user requirements by conducting a relatively large-scale interview study with 1010 participants. De Quincey and Mitchell (2022) also applied the UCD approach with a different method and in a different context. Specifically, they applied a special type of card sorting—repeated-single-criterion sorting—to understand and analyse user-based music categorization. On a different domain, Wright and Kefalidou (2022) studied the phenomenon of synaesthesia in the context of mixed reality. They posited that multimodal design characteristics could inform a more synaesthetic design approach in MR, which may have implications for supporting people’s mental health (cf. Noronha and Campos’ work on VR-based relaxation). Overall, majority of the papers in this volume have contributed to the understanding of well-being-oriented research, especially in the domain of healthcare. Nevertheless, there exists a substantial challenge in bridging the gap between the provision of technological solutions and real-life adoption (Blandford, 2019), despite the systematic application of the UCD approaches. Furthermore, AI-infused systems may contest the prevailing concepts, methods and tools of UCD. This may instigate a paradigm shift that launches the next wave of HCI, which can be well-being-driven HCI!

Although multiple studies have shown the efficacy of Serious Games (SGs) in improving students' performance and learning experience, the wide use of SGs is yet to be realised. One of the challenges towards the adoption of SGs can be attributed to the complex and costly development process. In this paper, we present ARQS (Authentic Role-playing-game Quest System) tool, an educator-oriented authoring tool we developed for supporting the implementation of a serious role-playing-game (RPG). The authoring tool consists of features that simplify the creation of important RPG components e.g. avatars, virtual world. Furthermore, the authoring process employs block-based programming to configure aspects of RPG such as creating quests. The concept is adopted in order to provide a platform where educators without programming knowledge are able to use. A formative evaluation was conducted with ten educators from secondary and university levels, using a low-fidelity prototype. Outcomes from the evaluation showed promising results, attesting the potential of ARQS as an educational tool for the creation of interactive learning instructions in the form of real-world scenarios embedded within a role-playing-game.

Cued Recall Debriefing (CRD) is a form of retrospective think aloud approach. It involves re-immersing users to a level where emotional responses are comparable to those experienced during actual interaction with a system. To validate whether the robustness of CRD would vary with the time gap between the actual and recalled event and with the affective state preceding the recall, two empirical studies with altogether 100 participants were conducted. Specifically, participants’ emotions were measured in terms of galvanic skin response (GSR) , heart rate (HR) , and self-assessment manikin (SAM) rating when they were interacting with an email client seeded with usability problems. The same measures were taken when they viewed the videoed interactions. Two between-subject variables were ‘intervening time’ (from 0 minutes up to 24 hours) and ‘intervening affect’ (images with different valence and arousal). Advanced computational models were applied to optimise the shifting of GSR/HR waves generated at the actual interaction and recall phases, which were found to be significantly correlated. The shifting process is necessary for addressing the memory effect and is a methodological innovation. Overall, CRD proved to be a robust method that can be deployed to a broad range of HCI research and practice contexts.

The relationship between intellectual property rights (IPRs) and the development of creativity is always a controversial topic. However, it has seldom been explored from the user-centered design (UCD) perspective. This paper describes how the UCD approach has been employed to develop Design Patent Retrieval Application (acronym: DsPLAi), a mobile app aimed to integrate IPRs related information into early design processes to enhance designers’ IP practice and to facilitate the creative process. Interview studies were first conducted to identify end-users’ understanding of IPRs and related practices. Next, participatory design workshops with designers and IP processionals were organized to understand the interaction between the two parties and their needs, thereby deriving requirements for DsPLAi. A prototype of the app was developed and evaluated with ten industrial designers. The prototype received positive feedback in the usability evaluation. The empirical results showed that the provision of IPRs related information at an early stage could be helpful to the design process and that the designers were positive about the use of DsPLAi in their daily design routines.

Considering the potential physical limitations of older adults, the naturalness of touch-based gestures as an interaction method is questionable. Assuming touch-based gestures are natural, they should be highly learnable and amenable to enhancement through social interactions. To investigate whether social interactions can enhance the learnability of touch gestures for older adults with low digital literacy, we conducted a study with 42 technology- naïve participants aged 64 to 82. They were paired and encouraged to play two games on an interactive tabletop with the expectation to use the drag gesture to complete the games socially. We then compared these results with a previous study of technology-naïve older adults playing the same games individually. The results of the comparisons show that dyadic interactions had some benefits for the participants in helping them to become comfortable with the drag gesture by negotiation and imitation. Further qualitative analysis suggested that playing pairs generally helped learners to comfortably explore the digital environment using the newly acquired skill.

Consolidating usability problems (UPs) from problem lists from several users can be a cognitively demanding task for evaluators. It has been suggested that collaboration between evaluators can help this process. In an attempt to learn how evaluators make decisions in this process, the authors studied what justification evaluators give for extracting UPs and their consolidation when working both individually and collaboratively. An experiment with eight novice usability evaluators was carried out where they extracted UPs and consolidated them individually and then collaboratively. The data were analyzed by using conventional content analysis and by creating argumentation models according to the Toulmin model. The results showed that during UP, extraction novice usability evaluators could put forward warrants leading to clear claims when probed but seldom added qualifiers or rebuttals. Novice usability evaluators could identify predefined criteria for a UP when probed and this could be acknowledged as a backing to warrants. In the individual settings, novice evaluators had difficulty in presenting claims and warrants for their decisions on consolidation. Although further study is needed, the results of the study indicated that collaborating pairs had a tendency to argue slightly better than individuals. Through the experiment novice evaluators’ reasoning patterns during problem extraction and consolidation as well as during their assessment of severity and confidence could be identified.

Our main research question is how the choice of game type influences the success of digital educational games (DEGs), where success is defined as significant domain-specific knowledge gain (learning outcome) with positive player experience. We propose a methodological framework to address this question. The comparison of different game types is based on the previously developed Game Elements-Attributes Model (GEAM) and the Game Genre Map, which summarise game features and their relations. In addition, we present a research model considering the impact of player characteristics on learning outcome and player experience as well as their interrelation. Two empirical studies were conducted with 280 students. The application domain was computer programming. Study 1 compared three DEGs of the Mini-Game genre, differing in a single GEAM attribute—time pressure vs. puzzle solving and abstract vs. realistic settings. Study 2 compared DEGs of different genres, which vary in the implementation of several GEAM attributes. None of the player characteristics were found to be statistically significant factors. For both studies, significant differences were found in learning outcomes, for Study 2 also in some of the player experience dimensions. GEAM was demonstrated as a promising framework for games user research.

There are two approaches to measure people's experiences: memory-based and moment-based. Whereas the memory-based approaches are susceptible to the peak-end effect, the moment-based approaches appear to better reflect the experience of the present. In this paper, we propose that emotional assessment of think aloud verbalisations is a moment-based methodology for measuring UX. We conducted an empirical study with 46 participants in the domain of online shopping to evaluate their emotional experiences. Acoustic analysis of verbal data was used as the moment-based approach and self-report questionnaires as the retrospective or memory-based approach. Results of the study confirmed the previous finding that retrospective assessments did not reflect the actual experience. The results also suggested that retrospective evaluations of emotions were significantly correlated with the most frequently elicited emotion (i.e. modal emotion) during the interaction. In conclusion these results support the use of acoustic data analysis as an alternative approach to measuring UX.

The aim of the project Law in Children's Lives is to gamify the research activity of collecting data with a digital game to assess children's awareness of law in their everyday lives. Our main research goal is to address the theoretical and practical concerns in gamification through a user(child)-centred experiential approach. We grounded the design and evaluation of the game in the established User Experience (UX) theoretical frameworks -- Hassenzahl's hedonic-pragmatic model and McCarthy & Wright's four threads of experience. The game prototype consists of four microworlds with each comprising a set of scenarios where children are asked to select an action option and record their reasons by talking to the non-player character. The game was evaluated with 634 children aged 7-11 years. The levels of perceived fun, interestingness and ease of playing were generally high. The game could stimulate the children to think about the given scenarios and beyond them.

Frustration is used as a criterion for identifying usability problems (UPs) and for rating their severity in a few of the existing severity scales, but it is not operationalized. No research has systematically examined how frustration varies with the severity of UPs. We aimed to address these issues with a hybrid approach, using Self-Assessment Manikin, comments elicited with Cued-Recall Debrief, galvanic skin responses (GSR) and gaze data. Two empirical studies involving a search task with a website known to have UPs were conducted to substantiate findings and improve on the methodological framework, which could facilitate usability evaluation practice. Results showed no correlation between GSR peaks and severity ratings, but GSR peaks were correlated with frustration scores -- a metric we developed. The Peak-End rule was partially verified. The problematic evaluator effect was the limitation as it confounded the severity ratings of UPs. Future work is aimed to control this effect and to develop a multifaceted severity scale.

To provide teachers and students with more usable, desirable and useful online labs, the foremost step is to understand their existing usage patterns and experiences with such labs. This step was implemented with two online surveys, one targeting teachers and the other students. Altogether 915 respondents from 23 European countries were involved. Results indicate that the prevalence of online labs adoption was generally low and that the users had positive experiences despite some challenges such as difficult access (domain-specific labs) and language barrier (English user interface). Among others, a significant implication is to create a repository (portal) where resources can be used and shared with ease and enjoyability.

Two project management approaches, Agile and Lean, have increasingly been adopted in recent years for software development. Meanwhile, in the field of human–computer interaction (HCI), user experience (UX) has become central in research and practice. The new hybrids between the two fields—Agile UX and Lean UX—were born a few years ago. As Agile, Lean, and UX have different principles and practices, one can query whether the couplings are well justified and whether Agile or Lean is more compatible with UX work. We have conducted a conceptual analysis and tended to conclude that Lean instantiated as Kanban fits UX work better than Agile instantiated as Scrum. To explore further our claim, we performed a secondary data analysis of 10 semistructured interviews with practitioners working with Scrum and Kanban in different sectors (Study 1). This study enabled us to gain insights into the applications of the two processes in real-life cases, their strengths and weaknesses, and factors influencing the practicality of implementing them. Both processes seem not favorable for UX work in practice. Among others, one intriguing observation is loose adherence to the related guidelines and principles. A query derived from the analyses of the interviews is that “customer,” as compared with “user,” has more frequently been referred to by our interviewees, irrespective of the process they adopted. We have then been motivated to investigate this issue, using a web-based survey with another batch of practitioners (N = 73) in the software industry (Study 2). Results of the survey indicate that the practitioners in general had a reasonable understanding of the concepts “user” and “customer,” although a minority tended to treat them as synonyms. Limitations of the current studies and implications for future work are discussed.

Touch-based interactions, if intuitive and natural, should be highly learnable. However, the learnability of touch gestures as a computer interaction modality cannot be taken for granted and should be evaluated empirically. This is especially true when technology-naïve older adults are concerned, given the psychological and physical constraints associated with this age range. To investigate this issue, we conducted a study with 17 technology-naïve older adults, whose experience with technology was mostly limited to making calls with mobile phones. Specifically, the participants were presented with three simple digital games on an interactive tabletop surface and asked to play the games on their own with minimal instruction or help. The first two games required the use of a drag gesture whereas the third game required the use of a rotate gesture. The main research question was whether the older adults could learn the gestures effectively and efficiently. A set of variables such as task duration and gesture accuracy were measured. Results showed that the learnability of a drag gesture was relatively high and that the participants could transfer this skill across the games. In contrast, the learnability of a rotate gesture was low and most of the participants failed to demonstrate any improvement over the tasks of the game. These findings partially corroborate those of earlier work. Implications for future work are drawn, especially exploring the potential of social interactions between older adults for enhancing the learning effects.

Large classes at universities (>1600 students) create their own challenges for teaching and learning. Audience feedback is lacking and fine tuning of lectures, courses and exam preparation to address individual needs is very difficult to achieve. At RWTH Aachen University, a course concept and a knowledge map learning tool aimed to support individual students to prepare for exams in information science through theme-based exercises were developed and evaluated. The tool was grounded in the notion of self-regulated learning with the goal of enabling students to learn independently.

A mobile educational app combining the strengths of sound pedagogical frameworks Generic Learning Outcomes and of interactive technologies Augmented Reality (AR) and Quick Response (QR) code to deliver the historical information of the Howard family in the Reformation is being developed by an interdisciplinary team. Visiting the relatively small archaeological site -- Thetford Priory- with the new technologies may contribute to resolving an age-old puzzle through interpretations from multiple perspectives and to fostering the sustainability of such a site.

Massively Multi-player Online Role-playing Games (MMORPGs) have become a popular leisure activity. It has been suggested that the reason for their popularity is that they offer a new ‘third-place’ for people to socialise. However, designing for sociability in these games has been shown to be a challenge. In this article, we discuss the results of an online survey that was directed towards game design researchers and professionals. We then present a subsequent discussion on the results of the survey at a Special Interest Group (SIG) held at CHI 2011. Through analysis of the findings of the survey and the discussion, we propose six requirements that facilitate the design of MMORPGs: In-game Communication, Off-game Communication, Empathy, Grouping and Rewards, World Design, and Designed Relationships. We state that it is not necessary to add all the proposed requirements in the design of such games, but we also caution that a game that does not include any of the requirements presented here cannot belong to this genre. We discuss limitations of this work, and offer future research directions that result from this work.

Analysis is a key part of conducting usability evaluations, yet rarely systematically studied. Thus, we lack direction on how to do research on supporting practitioners' analysis and lose an opportunity for practitioners to learn from each other. We have surveyed 155 usability practitioners on the analysis in their latest usability evaluation. Analysis is typically flexible and light-weight. At the same time, practitioners see a need to strengthen reliability in evaluation. Redesign is closely integrated with analysis; more than half of the respondents provide visual redesign suggestions in their evaluation deliverables. Analysis support from academic research, including tools, forms and structured formats, does not seem to have direct impact on analysis practice. We provide six recommendations for future research to better support analysis.

Integration of e-learning into campus-based institutions is a challenging issue. A number of key issues and questions clearly need to be solved and answered before any attempt for implementing e-learning is taken. The purpose of this paper is hence to investigate how to implement e-learning in the higher education institutions (HEI) associated with the subject of electricity in Turkey, considering the student perspective. For this reason, a web-based survey has been conducted to measure the extent to which the representative students in the Turkish HEI are ready for e-learning and to investigate their perspectives on issues for implementing e-learning. Two techniques were used: an online questionnaire with close- and open-ended items and semi-structured online interviews. 704 responses were analysed. At the end of the questionnaire, respondents were invited to participate in a semi-structured interview. 63 accepted the invitation, but only 33 of them actually attended the interview. Based on the findings derived from the interviews, we develop a model to illustrate what principles are needed for implementing e-learning in the respective institutions. Overall, our findings show that e-learning is perceived as a solution for resolving some issues and it brings some innovation in the respective institutions in Turkey; and that it should be integrated into campus-based education and training via the blended learning approach.

This paper demonstrates the use of the eye tracking data to examine the effect of interacting with an adaptive educational game on learning. Empirical results indicate that there is no significant correlation between the length of fixation duration and the extent of knowledge gain and that players paid attention to the instructions prior to gameplay. A challenge of constructing a predictive model is undertaken.

Self-regulated learning (SRL) is a term that describes an individual's ability to learn how to learn. New pedagogical theories encourage teachers to motivate and support their students into achieving a high level of self-regulation in their learning. This paper reports on the preliminary findings of a survey that was conducted with educators in the UK and abroad, regarding their perception of SRL. The results of this survey provide an insight into the challenges involved in teaching students with different SRL levels, as well as the instruments used by teachers to encourage SRL among their students.

A major area of research is how a positive ‘user experience’ – or interaction experience – of the use of digital artefacts (e.g. web sites, virtual worlds and personal digital assistants) can be promoted (Hassenzahl and Tractinsky, 2006). This experience does not only include usability, but also other cognitive, socio-cognitive and affective aspects of users’ experience in their interaction with artefacts, such as users’ enjoyment, aesthetic experience, desire to repeat use, positive decision to use a digital artefact and enhanced mental models. Research in this area is timely because we are approaching the ‘loyalty decade’, where interaction experience will become the main success factor (Nielsen, 2008). Hence, the success of digital artefacts is to a large extent positively influenced by the extent to which they promote a high-quality experience in their users. Furthermore, user experience (UX) manifests as quality in design, in interaction and in value, with diverse measures from many methods and instruments. One of the challenges related to UX is how to select appropriate measures to address the particularities of an evaluation context. The necessity and utility of UX measures is apparent, because such measures enable professionals to benchmark competitive design artefacts and to select appropriate design options. However, both the construct validity and predictive power of some UX measures are of particular concern. Consequently, modelling users’ experience – as a basis for producing design guidance – is especially important.

Despite the growing interest in user experience (UX), it has been hard to gain a common agreement on the nature and scope of UX. In this paper, we report a survey that gathered the views on UX of 275 researchers and practitioners from academia and industry. Most respondents agree that UX is dynamic, context-dependent, and subjective. With respect to the more controversial issues, the authors propose to delineate UX as something individual (instead of social) that emerges from interacting with a product, system, service or an object. The draft ISO definition on UX seems to be in line with the survey findings, although the issues of experiencing anticipated use and the object of UX will require further explication. The outcome of this survey lays ground for understanding, scoping, and defining the concept of user experience.

Understanding the relation between usability measures seems crucial to deepen our conception of usability and to select the right measures for usability studies. We present a meta-analysis of correlations among usability measures calculated from the raw data of 73 studies. Correlations are generally low: effectiveness measures (e.g., errors) and efficiency measures (e.g., time) have a correlation of .247 ± .059 (Pearson's product-moment correlation with 95% confidence interval), efficiency and satisfaction (e.g., preference) one of .196 ± .064, and effectiveness and satisfaction one of .164 ± .062. Changes in task complexity do not influence these correlations, but use of more complex measures attenuates them. Standard questionnaires for measuring satisfaction appear more reliable than homegrown ones. Measures of users' perceptions of phenomena are generally not correlated with objective measures of the phenomena. Implications for how to measure usability are drawn and common models of usability are criticized.

The rise of social software poses the challenges to the design and evaluation of a pedagogically sound online learning environment (OLE). Our OLE addresses these challenges by the integration of three pedagogical concepts - cross-cultural collaboration, self-directed learning and social networking - with the aim to advance participants' competencies and by mixed-method approaches to evaluating the complex situations. A validation trial involving four European countries was conducted. Groups of students co-created a questionnaire, which was assessed to provide an indicator of task performance. Multi-source (surveys, blogs, emails, diaries, chats, videoconference, and interviews) and multi-perspective data (facilitators, students, researchers) were studied with social network analysis, content analysis and conversation analysis. Several a posteriori research questions are addressed.

No systematic empirical study on investigating the effects of performing task variants on user cognitive strategy and behaviour in usability tests and on learnability of the system being tested has been documented in the literature. The current use-inspired basic research work aims to identify the underlying cognitive mechanisms and the practical implications of this specific endeavour. The focus of our work was to assess user rationality and system learnability. The software application tested was a multilingual learning resource repository. Eleven German and eleven Slovenian participants were involved in two user tests (UTs). Usability problems (UPs) identified in two quasi-isomorphic tasks were categorized with respect to a scheme of associated skills. Actions of the two tasks of each of the 22 users were segmented and coded according to a scheme of cognitive activities. Results showed that generally the users adopted different strategies for working out the given task and its variant, and that the system could be proved learnable. User Rational Action Model and implications for future research on user tests are inferred.

Research on heuristic evaluation in recent years has focused on improving its effectiveness and efficiency with respect to user testing. The aim of this paper is to refine a research agenda for comparing and contrasting evaluation methods. To reach this goal, a framework is presented to evaluate the effectiveness of different types of support for structured usability problem reporting. This paper reports on an empirical study of this framework that compares two sets of heuristics, Nielsen’s heuristics and the cognitive principles of Gerhardt-Powals, and two media of reporting a usability problem, i.e. either using a web tool or paper. The study found that there were no significant differences between any of the four groups in effectiveness, efficiency and inter-evaluator reliability. A more significant contribution of this research is that the framework used for the experiments proved successful and should be reusable by other researchers because of its thorough structure.

Abstract In this article, we assess the usability in an Internet‐based system for e‐learning in a cross‐cultural environment. The context of the evaluation and testing was a training program launched with the intention of introducing and promoting a new way of learning about and understanding the emerging technologies in regions with a low educational level and a high unemployment rate. The aim of the study was to assess the usability of the e‐learning system with different methods and approaches to get a good assessment of its learnability and applicability in various circumstances.

Social software is used widely in organizational knowledge management and professional learning. The PROLEARN network of excellence appreciates the trend of lowering the barriers between knowledge and learning management strategies for organizations and individuals. But, companies should not underestimate the needs for systematic support based on sound theories and technologies. We illustrate the requirements by examples and research issues for collaborative adaptive learning platforms for workplace learning in organizations

In this article, a case study on evaluating the downstream utility of user tests performed on a digital library with reference to the developer effect is reported. Downstream utility is defined as the effectiveness with which the resolution to a usability problem (UP) is implemented, and developer effect is defined as developers' bias toward fixing UPs with particular characteristics. To measure the effectiveness of the user tests, the actual impacts of fixing or not fixing the UPs identified were analyzed. To address a theoretical void in studying the persuasive power of usability evaluation results, Information Integration Theory was employed. Six research questions that predict the persuasiveness of different qualities of usability problems to induce fixes and the effectiveness of such fixes were investigated. Multiperspective data have been collected from usability specialists, the development team of the digital library, and its old as well as new users. Implications for reporting UPs and future research are inferred.

MAUSE-COST294 is a community of interest with its members sharing a common goal of bringing more science to bear on usability evaluation methods. This goal is being realized through systematic collaborative efforts of a network of usability experts and learners. Scientific activities are being coordinated under the respective frameworks of four Working Groups (WGs) with each of them having specific objectives, inputs and anticipated outputs. Further, MAUSE partners' research projects on usability will be integrated into the WGs. Specifically, several partners are actively involved in researching the usability of mobile user interfaces.

User effect in terms of influencing the validity and reliability of results derived from standard usability tests has been studied with different approaches during the last decade, but inconsistent findings were obtained. User effect is further complicated by other confounding variables. With the use of various computational models, we analyze the extent of user effect in a relatively complex arrangement of international usability tests in which four different European countries were involved. We explore five aspects of user effect, including optimality of sample size, evaluator effect, effect of heterogeneous subgroups, performance of task variants, and efficiency of problem discovery. Some implications for future research are drawn.

Two experiments were conducted to investigate the relationship between planning and debugging and the effect of program authorship on debugging strategies. Three groups of participants with different programming experiences were recruited. In the first experiment, the participants were asked to develop and debug their self-generated program whereas in the second experiment, they were asked to debug an otherwritten program where some logical errors were planted. Situated cognition approach, being an emergent cognitive paradigm, furnishes an alternative framework to understand the problems of interest. Deweyan notion of inquiry and Gibsonian theory of affordance are of particular relevance. The results show that planning is ineffective for debugging, irrespective of the programming expertise level and program authorship. Besides, situated debugging is demonstrated to be the preferred strategy which is not significantly related to the program authorship. A model of planning for program debugging and a theory of two-faceted transparency are postulated for explicating the observations.