AI agents are increasingly interacting with humans in their daily lives. This has highlighted several problems in current application of human-agent interactions (HAI) such as non-use and rejection and concern over impact on human behaviour. This workshop will explore cognitive, social, and psychological aspects of HAI, including how human behaviour and mental models influence and are influenced during these interactions. We aim to contribute with potential classical and novel theories that could be applied to ground HAI, understand what is needed and how to develop efficient, ethical, social, and engaging HAI.

Health-promoting digital agents, taking on the role of an assistant, coach or companion, are expected to have knowledge about a person’s medical and health aspects, yet they typically lack knowledge about the person’s activities. These activities may vary daily or weekly and are contextually situated, posing challenges for the human-agent interaction. This pilot study aimed to explore the experiences and behaviors of older adults when interacting with an initially unknowledgeable digital agent that queries them about an activity that they are simultaneously engaged in. Five older adults participated in a scenario involving preparing coffee followed by having coffee with a guest. While performing these activities, participants educated the smartwatch-embedded agent, named Virtual Occupational Therapist (VOT), about their activity performance by answering a set of activity-ontology based questions posed by the VOT. Participants’ interactions with the VOT were observed, followed by a semi-structured interview focusing on their experience with the VOT. Collected data were analyzed using an activity-theoretical framework. Results revealed participants exhibited agency and autonomy, deciding whether to adapt to the VOT’s actions in three phases: adjustment to the VOT, partial adjustment, and the exercise of agency by putting the VOT to sleep after the social conditions and activity changed. Results imply that the VOT should incorporate the ability to distinguish when humans collaborate as expected by the VOT and when they choose not to comply and instead act according to their own agenda. Future research focuses on how collaboration evolves and how the VOT needs to adapt in the process.

Abstract In this paper we propose a method to augment a Reinforcement Learning agent with legibility. This method is inspired by the literature in Explainable Planning and allows to regularize the agent’s policy after training, and without requiring to modify its learning algorithm. This is achieved by evaluating how the agent’s optimal policy may produce observations that would make an observer model to infer a wrong policy. In our formulation, the decision boundary introduced by legibility impacts the states in which the agent’s policy returns an action that is non-legible because having high likelihood also in other policies. In these cases, a trade-off between such action, and legible/sub-optimal action is made. We tested our method in a grid-world environment highlighting how legibility impacts the agent’s optimal policy, and gathered both quantitative and qualitative results. In addition, we discuss how the proposed regularization generalizes over methods functioning with goal-driven policies, because applicable to general policies of which goal-driven policies are a special case.