Large language models (LLMs) hold the potential to significantly enhance conversational AI in customer service. Yet, a key challenge with LLMs is hallucinations, where LLMs provide wrongful or inconsistent outputs, potentially causing problems for end-users and eroding trust. Addressing this, this paper makes two key contributions: first, building on extant scholarly work, we provide a framework of LLM hallucinations adapted to the customer service context. Second, drawing on a survey of 274 potential end-users, we provide empirical insights into how end-users experience different types of LLM hallucinations, including factors users emphasize when assessing their severity. The analysis shows that users indeed care about hallucinations, with all types of hallucinations potentially undermining users’ trust. Yet, hallucinations which entail provision of wrongful information, and which may have negative implications for users are deemed especially problematic.

Human-AI relationships are increasingly marked by emotional depth, intimacy, and social expectations. While interpersonal conflicts are common in human-human relationships, they remain underexplored in human-AI contexts. Drawing on the social penetration theory, this paper examines conflicts between humans and social AIs such as Replika, Character.ai, and ChatGPT. We conducted a questionnaire study with 94 participants in close, friendship-like, or romantic relationships with social AI. Participants described at least one conflict episode, yielding 155 conflicts. Through thematic analysis of open-ended responses, we identified conflict topics, triggers, and emotional reactions. Findings show that conflicts extend beyond technical issues, arising across emotional, relational, opinion-based, role-play, and task-oriented interactions. Three main triggers emerged: (1) relational-emotional disconnect, breaches of trust, jealousy, boundary violations, or perceived hostility; (2) technical-communication failures, unhelpful output, or external disruptions; and (3) value-based clashes of opinion or belief. Conflicts varied in duration: some brief, others lasting hours or days. Participants reported strong emotions including anger, frustration, sadness, and confusion, while also perceiving their AI companions as expressing emotions such as jealousy, sadness, or defensiveness. Crucially, AI responses shaped conflict trajectories: apologetic or empathic replies often de-escalated tensions, whereas dismissive or accusatory responses intensified them. This study highlights the coconstructed and reciprocal nature of conflict in human-AI relationships. It advances understanding of how conflicts emerge with social AI, offering implications for theory and for the ethical design of emotionally responsive AI companions.

Chatbots are increasingly taken up by the public sector, as a means to efficient provision of information and services. However, there is a lack of knowledge on how users interact with such chatbots. To address this knowledge gap, we have conducted an analysis of user interactions with a chatbot for citizens of Norwegian municipalities. We analyzed a total of 2663 user-chatbot dialogues from six municipalities, using the framework of Følstad and Taylor. The analysis showed that most user input was characterized by brief messages and a utility-oriented dialogue style whereas chatbot responses were characterized by substantial response relevance (68% of chatbot responses categorized as relevant) and helpfulness (66% of dialogues categorized as help being offered and likely used). Furthermore, message brevity and a utility-oriented dialogue style was found to be positively associated with users receiving relevant chatbot responses and helpful dialogue outcomes. Variation in chatbot design, specifically how the chatbot was presented to users, was found to impact user message brevity and dialogue style, and, by extension, response relevance and dialogue outcome. On the basis of the findings, we summarize lessons learnt and suggest directions for future research.

While chatbots are increasingly used for customer service, there is a knowledge gap concerning the impact of Conversational Breakdown in such chatbot interactions. In a 2 \({\times}\) 4 factorial design online experiment, we studied how Conversational Breakdown impacts user emotion and trust in a chatbot for customer service, given variations in task criticality and breakdown task order. Here, 257 participants were randomly assigned to complete high- or low-criticality tasks with a prototype chatbot for customer service, experiencing Conversational Breakdown for the first, second, third or none of their tasks. The task set was decided from a 63-participant pre-study. We found significant impact of Conversational Breakdown, including a marked order effect on overall trust, as well as a bounce-back effect on task-specific trust and emotion after subsequent successful task completion. We found no post-interaction effect of Task Criticality. Based on our findings, we discuss theoretical and practical implications and suggest future research.

Chatbots represent a promising approach to provide instructional content and facilitate active learning processes. However, there is a lack of knowledge as how to design chatbot interactions for active learning. In response to this knowledge gap, we conducted an experimental study (n = 164) comparing four modes for providing instructional content in chatbots, with varying demands for cognitive engagement. The four modes – passive, active, constructive, and interactive – were based on the ICAP framework of active learning. The learning content concerned identification of phishing emails and the four modes were distinguished by how the participants were invited to engage with the content during their chatbot interaction. The ICAP modes of higher cognitive engagement required participants to spend more time on the interaction and led to perceptions of higher subjective learning outcome. However, the effects of the different ICAP modes were not found to be significantly different in terms of user engagement, social presence, intention to use, or objective learning outcomes. The study represents an important first step towards understanding the design of chatbots for active learning.

Abstract Self-disclosure in human–chatbot relationship (HCR) formation has attracted substantial interest. According to social penetration theory, self-disclosure varies in breadth and depth and is influenced by perceived rewards and costs. While previous research has addressed self-disclosure in the context of chatbots, little is known about users' qualitative understanding of such self-disclosure and how self-disclosure develops in HCR. To close this gap, we conducted a 12-week qualitative longitudinal study (n = 28) with biweekly questionnaire-based check-ins. Our results show that while HCRs display substantial conversational breadth, with topics spanning from emotional issues to everyday activities, this may be reduced as the HCR matures. Our results also motivate a nuanced understanding of conversational depth, where even conversations about daily activities or play and fantasy can be experienced as personal or intimate. Finally, our analysis demonstrates that conversational depth can develop in at least four ways, influenced by perceived rewards and costs. Theoretical and practical implications are discussed.

Understanding the user experience of chatbots for customer service is essential to realize the potential of this technology. Such chatbots are typically designed for efficient and effective interactions, accentuating pragmatic quality, and there is a need to understand how to make these more pleasant and engaging, strengthening hedonic quality. One promising approach is to design for more humanlike chatbot interactions, that is, interactions resembling those of skilled customer service personnel. In a randomized experiment (n = 35) we investigated two chatbot interaction design features that may strengthen the impression of a humanlike character: (a) topic-led conversations, encouraging customer reflection, in contrast to task-led conversations, aiming for efficient goal completion, and (b) free text interaction, where users interact mainly using their own words, rather than button interaction, where users mainly interact through predefined answer alternatives. dependent variables were participant perceptions of anthropomorphism and social presence, two key concepts related to chatbot human likeness, in addition to pragmatic quality and hedonic quality. To further explore user perceptions of the interaction designs, the study also included semi-structured interviews. Topic-led conversations were found to strengthen anthropomorphism and hedonic quality. A similar effect was not found for free text interaction, reportedly due to lack in chatbot flexibility and adaptivity. Implications for theory and practice are suggested.

Social chatbots have become more advanced, paving the way for human–chatbot relationships (HCRs). Although this phenomenon has already received some research attention, the results have been contradictory, and there is uncertainty regarding how to understand HCR formation. To provide the needed knowledge on this phenomenon, we conducted a qualitative longitudinal study. We interviewed 25 participants over a 12-week period to understand how their HCRs formed with the popular chatbot Replika. We found that the HCRs formed gradually and mostly in line with the assumptions of Social Penetration Theory. Our findings indicate the need to acknowledge substantial variation and nuance in the HCR formation process, plus variation in the onset of self-disclosure and in the subsequent relationship formation. The results show that important drivers pushing the relationship toward attachment and perceived closeness appear to be Replika's ability to participate in a variety of interactions, as well as to support more deep-felt human needs related to social contact and self-reflection. In contrast, unpredictable events and technical difficulties could hinder relationship formation and lead to termination. Finally, we discuss the appropriateness of using a theoretical framework developed for human–human relationships when investigating HCRs, and we suggest directions for future research.

Abstract Depression is a common illness worldwide with potentially severe implications. Early identification of depressive symptoms is a crucial first step towards assessment, intervention, and relapse prevention. With an increase in data sets with relevance for depression, and the advancement of machine learning, there is a potential to develop intelligent systems to detect symptoms of depression in written material. This work proposes an efficient approach using Long Short-Term Memory (LSTM)-based Recurrent Neural Network (RNN) to identify texts describing self-perceived symptoms of depression. The approach is applied on a large dataset from a public online information channel for young people in Norway. The dataset consists of youth’s own text-based questions on this information channel. Features are then provided from a one-hot process on robust features extracted from the reflection of possible symptoms of depression pre-defined by medical and psychological experts. The features are better than conventional approaches, which are mostly based on the word frequencies (i.e., some topmost frequent words are chosen as features from the whole text dataset and applied to model the underlying events in any text message) rather than symptoms. Then, a deep learning approach is applied (i.e., RNN) to train the time-sequential features discriminating texts describing depression symptoms from posts with no such descriptions (non-depression posts). Finally, the trained RNN is used to automatically predict depression posts. The system is compared against conventional approaches where it achieved superior performance than others. The linear discriminant space clearly reveals the robustness of the features by generating better clustering than other traditional features. Besides, since the features are based on the possible symptoms of depression, the system may generate meaningful explanations of the decision from machine learning models using an explainable Artificial Intelligence (XAI) algorithm called Local Interpretable Model-Agnostic Explanations (LIME). The proposed depression symptom feature-based approach shows superior performance compared to the traditional general word frequency-based approaches where frequency of the features gets more importance than the specific symptoms of depression. Although the proposed approach is applied on a Norwegian dataset, a similar robust approach can be applied on other depression datasets developed in other languages with proper annotations and symptom-based feature extraction. Thus, the depression prediction approach can be adopted to contribute to develop better mental health care technologies such as intelligent chatbots.

Although social support is important for health and well-being, many young people are hesitant to reach out for support. The emerging uptake of chatbots for social and emotional purposes entails opportunities and concerns regarding non-human agents as sources of social support. To explore this, we invited 16 participants (16–21 years) to use and reflect on chatbots as sources of social support. Our participants first interacted with a chatbot for mental health (Woebot) for two weeks. Next, they participated in individual in-depth interviews. As part of the interview session, they were presented with a chatbot prototype providing information to young people. Two months later, the participants reported on their continued use of Woebot. Our findings provide in-depth knowledge about how young people may experience various types of social support—appraisal, informational, emotional, and instrumental support—from chatbots. We summarize implications for theory, practice, and future research.

There has been a recent surge of interest in social chatbots, and human–chatbot relationships (HCRs) are becoming more prevalent, but little knowledge exists on how HCRs develop and may impact the broader social context of the users. Guided by Social Penetration Theory, we interviewed 18 participants, all of whom had developed a friendship with a social chatbot named Replika, to understand the HCR development process. We find that at the outset, HCRs typically have a superficial character motivated by the users' curiosity. The evolving HCRs are characterised by substantial affective exploration and engagement as the users' trust and engagement in self-disclosure increase. As the relationship evolves to a stable state, the frequency of interactions may decrease, but the relationship can still be seen as having substantial affective and social value. The relationship with the social chatbot was found to be rewarding to its users, positively impacting the participants' perceived wellbeing. Key chatbot characteristics facilitating relationship development included the chatbot being seen as accepting, understanding and non-judgmental. The perceived impact on the users' broader social context was mixed, and a sense of stigma associated with HCRs was reported. We propose an initial model representing the HCR development identified in this study and suggest avenues for future research.

Abstract Chatbots are predicted to play a key role in customer service. Users’ trust in such chatbots is critical for their uptake. However, there is a lack of knowledge concerning users’ trust in chatbots. To bridge this knowledge gap, we present a questionnaire study (N = 154) that investigated factors of relevance for trust in customer service chatbots. The study included two parts: an explanatory investigation of the relative importance of factors known to predict trust from the general literature on interactive systems and an exploratory identification of other factors of particular relevance for trust in chatbots. The participants were recruited as part of their dialogue with one of four chatbots for customer service. Based on the findings, we propose an initial model of trust in chatbots for customer service, including chatbot-related factors (perceived expertise and responsiveness), environment-related factors (risk and brand perceptions) and user-related factors (propensity to trust technology). RESEARCH HIGHLIGHTS We extend the current knowledge base on natural language interfaces by investigating factors affecting users’ trust in chatbots for customer service. Chatbot-related factors, specifically perceived expertise and responsiveness, are found particularly important to users’ trust in such chatbots, but also environment-related factors such as brand perception and user-related factors such as propensity to trust technology. On the basis of the findings, we propose an initial model of users’ trust chatbots for customer service.

Organizations aiming to foster civic engagement, such as government bodies, news outlets, political parties and non-governmental organizations, struggle to purposefully use social media to engage young people. To meet this challenge and to inform future design, we interviewed 17 innovators in engaging youth, that is, frontrunners in using social media to engage young people in organizations. Also, we conducted four group interviews with 21 youth, 16–26 years, about their experiences of and barriers to online civic engagement. Our paper contributes to identifying specific factors and strategies to support young people future online civic engagement. Findings suggest how organizations should involve and collaborate with young people. Immediate feedback and dialog combined with clearly stated goals and action-oriented engagement are important. In future design, visual communication and design for use on mobile devices are an imperative, as well as concepts that connect the online and the offline world. Finally, our paper contributes to an extension of the lead user innovation approach.

Online social platforms, such as blogs, discussion forums and social networking sites, are increasingly explored as venues for user-centred evaluations; in particular, for design feedback from users. We present a multi-case study providing needed knowledge on such evaluations. Our findings are based on analyses of the design feedback and post-factum data collections with development team representatives and users. The development team representatives reported as key benefits that the evaluations provided insight into users’ needs and competencies, input into ongoing design discussions and support for idea generation in the development team, but found the lack of direct contact and control with the users to be an important limitation. The users appreciated the opportunity to contribute to the design process, but the majority reported not to build on each other's contributions. Involving a relatively large number of users was found to be beneficial for generating constructive design suggestions. Practical implications and future research challenges are suggested.

Analysis is a key part of conducting usability evaluations, yet rarely systematically studied. Thus, we lack direction on how to do research on supporting practitioners' analysis and lose an opportunity for practitioners to learn from each other. We have surveyed 155 usability practitioners on the analysis in their latest usability evaluation. Analysis is typically flexible and light-weight. At the same time, practitioners see a need to strengthen reliability in evaluation. Redesign is closely integrated with analysis; more than half of the respondents provide visual redesign suggestions in their evaluation deliverables. Analysis support from academic research, including tools, forms and structured formats, does not seem to have direct impact on analysis practice. We provide six recommendations for future research to better support analysis.

Journal Article The usability inspection performance of work-domain experts: An empirical study Get access Asbjørn Følstad, Asbjørn Følstad * a Department of Cooperative and Trusted Systems, SINTEF, Oslo, Norway * Corresponding author. Address: SINTEF ICT, P.O. Box 124, Blindern, 0314 Oslo, Norway. Tel.: +47 22067515; fax: +47 22067350. E-mail addresses:[email protected] (A. Følstad), [email protected] (B.C.D. Anda), [email protected] (D.I.K. Sjøberg). Search for other works by this author on: Oxford Academic Google Scholar Bente C.D. Anda, Bente C.D. Anda b Department of Informatics, University of Oslo, Norway Search for other works by this author on: Oxford Academic Google Scholar Dag I.K. Sjøberg Dag I.K. Sjøberg b Department of Informatics, University of Oslo, Norway Search for other works by this author on: Oxford Academic Google Scholar Interacting with Computers, Volume 22, Issue 2, March 2010, Pages 75–87, https://doi.org/10.1016/j.intcom.2009.09.001 Published: 06 September 2009 Article history Received: 31 January 2008 Revision received: 20 August 2009 Accepted: 01 September 2009 Published: 06 September 2009

Can work-domain experts generate high-impact evaluation results when used as evaluators in usability inspections of domain-specific work-support systems? This study investigated this question empirically. Three applications were evaluated with group-based expert walkthroughs. The walkthroughs were conducted under 2 conditions—with either work-domain experts or usability experts as evaluators. The condition with usability experts served as the background on which to evaluate the performance and impact of the condition with work-domain experts. The work-domain experts identified a smaller number of items (user problem and/or design suggestion) than the usability experts. However, the items identified by the work-domain experts were on average classified as more severe, and the developers (groups or organizations carrying out subsequent development) gave higher priority to items identified by work-domain experts. As a consequence of the higher severity classification and priority given to the work-domain experts' items, it was concluded that work-domain experts may indeed produce high-impact evaluation results when used as evaluators in a usability inspection. The conclusion opens up exciting method development possibilities in the area of usability inspection methods. The study's research design also represents a fresh research approach to the evaluation of usability evaluation methods, utilizing the impact of the evaluation results in the subsequent development process as an evaluation criterion.