Abstract This paper proposes an approach for training visuo-haptic object recognition models for robots using synthetic datasets generated by 3D virtual simulations. In robotics, where visual object recognition has witnessed considerable progress due to an abundance of image datasets, the scarcity of diverse haptic samples has resulted in a noticeable gap in research on machine learning incorporating the haptic sense. Our proposed methodology addresses this challenge by utilizing 3D virtual simulations to create realistic synthetic datasets, offering a scalable and cost-effective solution to integrate haptic and visual cues for object recognition seamlessly. Acknowledging the importance of multimodal perception, particularly in robotic applications, our research not only closes the existing gap but envisions a future where intelligent agents possess a holistic understanding of their environment derived from both visual and haptic senses. Our experiments show that synthetic datasets can be used for training object recognition in haptic and visual modes by incorporating noise, performing some preprocessing, data augmentation, or domain adaptation. This work contributes to the advancement of multimodal machine learning toward a more nuanced and comprehensive robotic perception.

Abstract Empathy plays a fundamental role in building relationships. To foster close relationships and lasting engagements between humans and robots, empathy models can provide direct clues into how it can be done. In this study, we focus on capturing in a quantitative way indicators of early empathy realization between a human and a robot using a process that encompasses affective attachment, trust, expectations and reflecting on the other’s perspective within a set of collaborative strategies. We hypothesize that an active collaboration strategy is conducive to a more meaningful and purposeful engagement of realizing empathy between a human and a robot compared to a passive one. With a deliberate design, the interaction with the robot was presented as a maze game where a human and a robot must collaborate in order to reach the goal using two strategies: one maintaining control individually taking turns (passive strategy) and the other one where both must agree on their next move based on reflection and argumentation. Quantitative and qualitative analysis of the pilot study confirmed that a general sense of closeness with the robot was perceived when applying the active strategy. Regarding the specific indicators of empathy realization: (1) affective attachment , affective emulation was equally present throughout the experiment in both conditions, and thus, no conclusion could be reached; (2) trust , quantitative analysis partially supported the hypothesis that an active collaborative strategy will promote teamwork attitudes, where the human is open to the robot’s suggestions and to act as a teammate; and (3) regulating expectation , quantitative analysis confirmed that a collaborative strategy promoted a discovery process that regulates the subject’s expectation toward the robot. Overall, we can conclude that an active collaborative strategy impacts favorably the process of realizing empathy compared to a passive one. The results are compelling to move the design of this experiment forward into more comprehensive studies, ultimately leading to a path where we can clearly study engagements that reduce abandonment and disillusionment with the process of realizing empathy as the core design for active collaborative strategies.

Current trends envisage an evolution of collaboration, engagement, and relationship between humans and devices, intelligent agents and robots in our everyday life. Some of the key elements under study are affective states, motivation, trust, care, and empathy. This paper introduces an empathy test-bed that serves as a case study for an existing empathy model. The model describes the steps that need to occur in the process to provoke meaning in empathy, as well as the variables and elements that contextualise those steps. Based on this approach we have developed a fun collaborative scenario where a user and a social robot work together to solve a maze. A set of exploratory trials are carried out to gather insights on how users perceive the proposed test-bed around attachment and trust, which are basic elements for the realisation of empathy.

In this paper we explore the field of human robot interaction (HRI) by adding a new non-physical player in the interaction rather than only humans and robots. It is a proven fact that physical robots enhance the immersion perception compared to only virtual agents. Thus, we created a shared environment with a Pleo Robot and a Virtual Pleo Robot, which we called Vleo, connected through a server to explore this new paradigm in order to see if the engagement during the interaction is improved in intensity and duration. A straightforward set of interactions between Pleo and Vleo have been designed to create narratives and therefore, tested with a group of 8-12 year old children. Results from the test suggest that virtual social robots are a good way to enhance interaction with physical social robots.