Knowledge graphs have been used to improve robot dialogues by providing more sophisticated world knowledge. We now propose a new role for knowledge graphs in GenAl-based HRI that aims to reduce dialogue errors by better conversational grounding. This approach uses both domain knowledge graphs and dialogue history graphs, constructing shared knowledge via entity linking. We present first steps towards these aims, and also address sustainability by supporting the use of smaller models.

The paper describes methods for anticipating follow-up questions in exploratory information search.There are two main cases: information stored in knowledge graphs, and information in unstructured texts such as Wikipedia.In the first case, follow-up questions are anticipated by extracting subgraphs relevant to user queries, passing the subgraphs to an LLM to generate responses.In the second case, entities and their relationships are extracted from the texts and added to short-term knowledge graphs relevant to initial queries.Follow-up questions are then anticipated by extracting subgraphs relevant to subsequent queries and passing the subgraphs to the LLM, as in the first case.The short-term graphs in dialogue memory are often sufficient to answer follow-up questions.If they are not, the described steps are repeated as required.

The paper discusses two current approaches to conversational AI, using large language models and knowledge graphs, and compares types of errors that occur in human-robot interactions based on these approaches. It provides example dialogues and describes solutions to several error types including false implications, ontological errors, theory of mind errors, and handling of speech recognition errors. The paper addresses issues of particular concern for earning user trust.

The paper describes an approach that combines work from three fields with previously separate research commu-nities: social robotics, conversational AI, and graph databases. The aim is to develop a generic framework in which a variety of social robots can provide high-quality information to users by accessing semantically-rich knowledge graphs about multiple different domains. An example implementation uses a Furhat robot with Rasa open source conversational AI and knowledge graphs in Neo4j graph databases.

The paper addresses aspects of cooperative and uncooperative behaviour in natural language dialogue between humans and social robots. The principles of cooperation in human-human dialogues are taken as the basis for cooperative behaviour in human-robot dialogues. Several approaches are described that can improve human-robot cooperation. These include more flexible recognition of user intents, more flexible searches using knowledge graphs, generating more cooperative responses using semantic metadata, and generating more human-friendly responses using Wikipedia. These approaches are demonstrated in a series of videos.

The paper discusses ethical issues in dialogue engagement to build a long-term trusting relationship between a human and a robot agent over a period of time. Engagement refers not only to frequent interactions but also to recognition of the partner and their non-verbal communication which indicates level of interest, activity, and affective state. In applications related to real-world care-taking and instructional tasks, a robot agent’s ability to recognize human partners and react appropriately can prove crucial for smooth interaction. Recognizing the partner’s face and remembering their individual interactions can contribute positively to the acceptance of social robot applications by enabling adaptation to the user’s needs and building long-term relations. For this purpose, we describe work on short-term and long-term memories in the robot’s dialogue manager, and the use of facial landmark recognition to identify the user.

Peer reviewed

The demo shows ERICA, a highly realistic female android robot, and WikiTalk, an application that helps robots to talk about thousands of topics using information from Wikipedia. The combination of ERICA and WikiTalk results in more natural and engaging human-robot conversations.

The demo shows a practical application of an open-source research toolkit developed by University of Cambridge. The toolkit (PyDial) supports research on deep reinforcement learning for multi-domain dialogues. The application (CityTalk) is a spoken dialogue system for robots that give information to tourists about local hotels and restaurants. We had a very positive experience using the toolkit, but in a few areas we decided to do things our own way.

This paper concerns interaction with social robots and focuses on the evaluation of a robot application that allows users to access interesting information from Wikipedia. The evaluation method compares the users' expectations with their experience with the robot, and takes into account their self-declared previous experience with robots. The results show that most participants had an overall positive experience, even though the averages indicate a slight negative tendency related to expectations of the robot's behavior and being understood by the robot. Interestingly, the most experienced users seem to be the most critical.

At SIGDIAL-2013 our talking robot demonstrated Wikipedia-based spoken information access in English.Our new demo shows a robot speaking different languages, getting content from different language Wikipedias, and switching languages to meet the linguistic capabilities of different dialogue partners.

The demo shows WikiTalk, a Wikipedia-based open-domain information access dialogue system implemented on a talking humanoid robot. The robot behaviour integrates speech, nodding, gesturing and face-tracking to support interaction management and the presentation of information to the partner.