Embodiment of users within robotic systems has been explored in human-robot interaction, most often in telepresence and teleoperation. In these applications, synchronized visuomotor feedback can evoke a sense of body ownership and agency, contributing to the experience of embodiment. We extend this work by employing embreathment, the representation of the user's own breath in real time, as a means for enhancing user embodiment experience in robots. In a within-subjects experiment, participants controlled a robotic arm, while its movements were either synchronized or non-synchronized with their own breath. Synchrony was shown to significantly increase body ownership, and was preferred by most participants. We propose the representation of physiological signals as a novel interoceptive pathway for human–robot interaction, and discuss implications for telepresence, prosthetics, collaboration with robots, and shared autonomy.

Robot appearance crucially shapes Human-Robot Interaction (HRI) but is typically described via broad categories like anthropomorphic, zoomorphic, or technical. More precise approaches focus almost exclusively on anthropomorphic features, which fail to classify robots across all types, limiting the ability to draw meaningful connections between robot design and its effect on interaction. In response, we present METAMORPH, a comprehensive framework for classifying robot morphology. Using a metamodeling approach, METAMORPH was synthesized from 222 robots in the IEEE Robots Guide, offering a structured method for comparing visual features. This model allows researchers to assess the visual distances between robot models and explore optimal design traits tailored to different tasks and contexts.

This paper introduces a novel benchmark that has been designed as a test bed for evaluating whether artificial agents are able to understand how to perform everyday activities, with a focus on the cooking domain. Understanding how to cook recipes is a highly challenging endeavour due to the underspecified and grounded nature of recipe texts, combined with the fact that recipe execution is a knowledge-intensive and precise activity. The benchmark comprises a corpus of recipes, a procedural semantic representation language of cooking actions, qualitative and quantitative kitchen simulators, and a standardised evaluation procedure. Concretely, the benchmark task consists in mapping a recipe formulated in natural language to a set of cooking actions that is precise enough to be executed in the simulated kitchen and yields the desired dish. To overcome the challenges inherent to recipe execution, this mapping process needs to incorporate reasoning over the recipe text, the state of the simulated kitchen environment, common-sense knowledge, knowledge of the cooking domain, and the action space of a virtual or robotic chef. This benchmark thereby addresses the growing interest in human-centric systems that combine natural language processing and situated reasoning to perform everyday activities.

We present the Meta-Ontology for Introspection (MOI): Inspired by fundamental processes of the human mind, cognitive architectures (CAs) explore ever more methods to leverage metacognition. Still, an ontological model to trace metacognitive experiences for learning or as input for metacognitive control routines has yet to be developed. Based on a review of existing standards, we formally identify the relevant scope in the form of Competency Questions (CQs) and extend SOMA, a well-established formal ontology initially designed to interpret episodic memories of a robotic CA. The resulting MOI can model a CA’s software and capabilities of single components, trace information processing and inter-component communication, label self-lived mental events, and capture causal relationships. We evaluate MOI via the CQs and exemplarily demonstrate its reasoning capabilities.

While a growing number of households contain home assistants, they mainly remain voice-only devices where the virtual agent is not represented visually. The visual representation of the agent is limited to the device's housing and abstract light animations that signify the assistant's state to its users. However, the audio channel is limited in conveying information beyond semantic content. Embodied virtual assistants can enhance interaction with conversational interfaces by adding a visual layer to further convey personality and human characteristics. In this work, we conducted an online survey (N=78) to explore people's preferences for visualizing humanoid assistants. Our findings suggest that participants prefer an agent who appears mature, healthy, competent, and attractive. Furthermore, demographic similarities between the users and agents are wished for the agent to look more relatable. We discuss these findings and their implications for the design of virtual humanoid home assistants.

Abstract Human beings and other biological agents appear driven by curiosity to explore the affordances of their environments. Such exploration is its own reward – children have fun when playing – but it probably also serves the practical purpose of learning theories with which to predict outcomes of actions. Cognitive robots however have yet to match the performance of human beings at learning and reusing manipulation skills. In this paper, we implement a method that emulates the curiosity drive and uses it as a heuristic to guide (simulated) exploration of a particular task – pouring liquids. The result of this exploration is a collection of symbolic rules linking qualitative descriptions of object arrangements and the pouring action with qualitative descriptions of likely outcomes. The manner in which qualitative descriptions of object arrangements and actions are converted to numerical descriptions for the purpose of simulation parametrization is via probability distributions, which themselves are adjusted in the process of simulated exploration. This allows the grounding of the symbolic descriptions to attempt to adapt itself to the task. The resulting symbolic rules form a theory that, together with the probability distributions that ground it in numerical parametrizations, is intended to be used to predict qualitative outcomes or select manners of pouring towards achieving a goal.

When playing sports in virtual reality foot interaction is crucial for many disciplines. We investigated how the visibility of the foot influences penalty shooting in soccer. In a between-group experiment, we asked 28 players to hit eight targets with a virtual ball. We measured the performance, task load, presence, ball control, and body ownership of inexperienced to advanced soccer players. In one condition, the players saw a visual representation of their tracked foot which improved the accuracy of the shots significantly. Players with invisible foot needed 58% more attempts. Further, with foot visibility the self-reported body ownership was higher.

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave many pieces of information necessary for performing the task unspecified. Humans can solve such problems fast as we reduce the search space by recourse to prior knowledge such as a connected collection of plans that describe how certain goals can be achieved at various levels of abstraction. Rather than enumerating fine-grained physical contexts SOMA sets out to include socially constructed knowledge about the functions of actions to achieve a variety of goals or the roles objects can play in a given situation. As the human cognition system is capable of generalizing experiences into abstract knowledge pieces applicable to novel situations, we argue that both physical and social context need be modeled to tackle these challenges in a general manner. The central contribution of this work, therefore, lies in a comprehensive model connecting physical and social entities, that enables flexibility of executions by the robotic agents via symbolic reasoning with the model. This is, by and large, facilitated by the link between the physical and social context in SOMA where relationships are established between occurrences and generalizations of them, which has been demonstrated in several use cases in the domain of everyday activites that validate SOMA.

The use of voice assistants (VAs) is spreading widely. Most common VAs consist of a single, usually female voice that responds to the user’s inquiry. We designed a VA system appearing as a group of agents, each with a different voice and a specialized task domain. We conducted a quantitative user study comparing our multi-agent approach with a conventional single-agent assistant in a smart home scenario as virtual reality (VR) simulation. The results show significantly higher user experience ratings for the multi-agent concept. Based on our findings, we discuss the potentials and challenges of designing multi-party VA systems.

We propose an approach to facilitate adjustable grip for object interaction in virtual reality. It enables the user to handle objects with loose and firm grip using conventional controllers. Pivotal design properties were identified and evaluated in a qualitative pilot study. Two revised interaction designs with variable grip were compared to the status quo of invariable grip in a quantitative study. The users performed placing actions with all interaction modes. Performance, clutching, task load, and usability were measured. While the handling time increased slightly using variable grip, the usability score was significantly higher. No substantial differences were measured in positioning accuracy. The results lead to the conclusion that variable grip can be useful and improve realism depending on tasks, goals, and user preference.

Digital home assistants have an increasing influence on our everyday lives. The media now reports how children adapt the consequential, imperious language style when talking to real people. As a response to this behavior, we considered a digital assistant rebuking impolite language. We then investigated how adult users react when being rebuked by the AI. In a between-group study (N = 20), the participants were being rejected by our fictional speech assistant "Eliza" when they made impolite requests. As a result, we observed more polite behavior. Most test subjects accepted the AI's demand and said "please" significantly more often. However, many participants retrospectively denied Eliza the entitlement to politeness and criticized her attitude or refusal of service.

Enabling multiple-purpose robots to follow textual instructions is an important challenge on the path to automating skill acquisition. In order to contribute to this goal, we work with physical exercise instructions as an everyday activity domain where textual descriptions are usually focused on body movements. Body movements are a common element across a broad range of activities that are of interest for robotic automation. Developing a text-to-animation system, as a first step towards understanding language for machines, is an important task. The process requires natural language understanding (NLU) including non-declarative sentences and the extraction of semantic information from complex syntactic structures with a large number of potential interpretations. Despite a comparatively high density of semantic references to body movements, exercise instructions still contain a large amount of underspecified information. Detecting and bridging or filling such underspecified elements is extremely challenging when relying on methods from NLU alone. Humans, however, can often add such implicit information with ease, due to its embodied nature. We present a process that contains a combination of a semantic parser and a Bayesian network. It explicates the information that is contained in textual movement instructions so that an animation execution of the motion-sequences performed by a virtual humanoid character can be rendered. Human computation is then employed to determine best candidates and to further inform the models in order to increase performance adequacy.

In multiagent-based simulation systems the agent programming paradigm is adopted for simulation. This simulation approach offers the promise to facilitate the design and development of complex simulations, both regarding the distinct simulation actors and the simulation environment itself. We introduce the simulation middleware PlaSMA which extends the JADE agent framework with a simulation control that ensures synchronization and provides a world model based on a formal ontological description of the respective application domain. We illustrate the benefits of an ontology grounding for simulation design and discuss further gains to be expected from recent advances in ontology engineering, namely the adaption of foundational ontologies and modelling-patterns.

Herein we describe the QuickWoZ system, a Wizard-of-Oz (WoZ) tool that allows for the remote control of the behavior of animated characters in a 3D environment. The complete scene, character, behaviors and sounds can be defined in simple XML documents, which are parsed at runtime, so that setting up an experiment can be done without programming expertise. Quick selection lists and buttons enable the wizard to easily control the agents' behavior and allow for fast reactions to the subjects' input.