Abstract Modern e-commerce websites often provide users with a variety of components, such as faceted filters and conversational recommender systems, that act as product advisors to help them find relevant products. However, these components are often treated separately and presented as independent components, leading to increased cognitive load and disruption in the search process. Also, the reasoning behind the resulting product recommendations is often not transparent. To address these limitations, we propose a novel approach that relies on a knowledge graph structure to seamlessly integrate faceted filtering and conversational advisors based on graphical user interfaces (GUI). Concretely, the knowledge graph is used to suggest filter values and products based on the user’s answers in the advisor, and, conversely, to determine follow-up questions based on the user’s selected filter values. The user interface also visualizes and explains the underlying relationships between answers given to the advisor and relevant product features in the filter component in order to increase the transparency of the search process. We conducted two user studies with a total of 448 participants to compare a system that integrates the different components according to our approach with a baseline system in which the mechanisms operate separately. Sequence analysis of the logged interaction data provided insights into participants’ behavior as they interacted with both systems. The results indicate that displaying recommended products and related explanations directly in the filter component increases acceptance and trust in the system. Also, the combination of a conversational advisor with values displayed in a filter interface, along with explanations of the underlying relationships, significantly contributes to the knowledge and understanding of those product features that are important in terms of the current search goal.

We propose the concept of meta-intents which represent high-level user preferences related to the interaction and decision-making in conversational recommender systems (CRS) and present a questionnaire instrument for measuring meta-intents. We conducted a two-stage user study, an exploratory study with 212 participants on Prolific, and a confirmatory study with 394 participants on Prolific. We obtained a reliable and stable meta-intents questionnaire with 22 question items, corresponding to seven latent factors (concepts). These seven factors cover important interaction preferences and are closely related to users’ decision-making process. For example, the factor dialog-initiative reflects whether users prefer to follow the system’s guidance or ask their own questions in a CRS. We conducted statistical analyses of meta-intents in two domains (smartphones and hotels), and a general chatbot scenario. We also investigated the influence of additional factors (demography, decision-making style) on meta-intents through Structural Equation Modeling (SEM). Our results provide preliminary evidence that the proposed meta-intents are domain and demography (gender, age) independent. They can be linked to the general decision-making style and can thus be instrumental in translating general decision-making factors into more concrete design guidance for CRS and their potential personalization. Meta-intents also provide a basis for future analyses of interaction behavior in CRS and the development of a cognitively founded theoretical framework.

Recommender Systems have seen substantial progress in terms of algorithmic sophistication recently. Yet, the systems mostly act as black boxes and are limited in their capacity to explain why an item is recommended. In many cases recommendations methods are employed in scenarios where users not only rate items, but also convey their opinion on various relevant aspects, for instance by the means of textual reviews. Such user-generated content can serve as a useful source for deriving explanatory information to increase system intelligibility and, thereby, the user's understanding.

User studies are increasingly considered important in research on recommender systems. Although participants typically cannot consume any of the recommended items, they are often asked to assess the quality of recommendations and of other aspects related to user experience by means of questionnaires. Not being able to listen to recommended songs or to watch suggested movies, might however limit the validity of the obtained results. Consequently, we have investigated the effect of consuming suggested items. In two user studies conducted in different domains, we showed that consumption may lead to differences in the assessment of recommendations and in questionnaire answers. Apparently, adequately measuring user experience is in some cases not possible without allowing users to consume items. On the other hand, participants sometimes seem to approximate the actual value of recommendations reasonably well depending on domain and provided information.

In user studies of recommender systems, participants typically cannot consume the recommended items. Still, they are asked to assess recommendation quality and other aspects related to user experience by means of questionnaires. Without having listened to recommended songs or watched suggested movies, however, this might be an error-prone task, possibly limiting validity of results obtained in these studies. In this paper, we investigate the effect of actually consuming the recommended items. We present two user studies conducted in different domains showing that in some cases, differences in the assessment of recommendations and in questionnaire results occur. Apparently, it is not always possible to adequately measure user experience without allowing users to consume items. On the other hand, depending on domain and provided information, participants sometimes seem to approximate the actual value of recommendations reasonably well.