Large language models (LLMs) hold the potential to significantly enhance conversational AI in customer service. Yet, a key challenge with LLMs is hallucinations, where LLMs provide wrongful or inconsistent outputs, potentially causing problems for end-users and eroding trust. Addressing this, this paper makes two key contributions: first, building on extant scholarly work, we provide a framework of LLM hallucinations adapted to the customer service context. Second, drawing on a survey of 274 potential end-users, we provide empirical insights into how end-users experience different types of LLM hallucinations, including factors users emphasize when assessing their severity. The analysis shows that users indeed care about hallucinations, with all types of hallucinations potentially undermining users’ trust. Yet, hallucinations which entail provision of wrongful information, and which may have negative implications for users are deemed especially problematic.

Chatbots designed to achieve specific goals are increasingly used in service roles. Human-like characteristics in chatbots are seen as beneficial to interaction. Given the purpose-driven nature of goal-oriented chatbots, whether humanlikess of various characteristics of these chatbots has an effect on their perception and use is unclear. This study explores the role of humanlikeness in goal-oriented chatbots through three experiments focused on agent behavior, interaction modality, and communication medium in a travel-assistant context. The findings suggest that while human-like behavioral attributes enhance perceptions of anthropomorphism, animacy, and likeability, they do not significantly affect reliability, perceived intelligence, or safety. The text medium was also perceived as more trustworthy and anthropomorphic than the voice medium. Overall, successful task completion seems to be the most crucial factor in shaping user perceptions, suggesting that humanlikeness may be less critical in goal-oriented chatbot interactions.

While chatbots are increasingly used for customer service, there is a knowledge gap concerning the impact of Conversational Breakdown in such chatbot interactions. In a 2 \({\times}\) 4 factorial design online experiment, we studied how Conversational Breakdown impacts user emotion and trust in a chatbot for customer service, given variations in task criticality and breakdown task order. Here, 257 participants were randomly assigned to complete high- or low-criticality tasks with a prototype chatbot for customer service, experiencing Conversational Breakdown for the first, second, third or none of their tasks. The task set was decided from a 63-participant pre-study. We found significant impact of Conversational Breakdown, including a marked order effect on overall trust, as well as a bounce-back effect on task-specific trust and emotion after subsequent successful task completion. We found no post-interaction effect of Task Criticality. Based on our findings, we discuss theoretical and practical implications and suggest future research.

The digitization of mental health care holds promises of affordable and ubiquitously available treatment, e.g., with conversational agents (chatbots). While technology can guide people to care for themselves, we examined how people can care for another being as a way to care for themselves. We created a self-compassion chatbot (Vincent) and compared between caregiving and care-receiving conditions. Care-giving Vincent asked participants to partake in self-compassion exercises. Care-receiving Vincent shared its foibles, e.g., embarrassingly arriving late at an IP address, and sought out advice. While self-compassion increased for both conditions, only those with care-receiving Vincent significantly improved. In tandem, we offer qualitative data on how participants interacted with Vincent. Our exploratory research shows that when a person cares for a chatbot, the person's self-compassion can be enhanced. We further reflect on design implications for strengthening mental health with chatbots.