Maintaining motivation and sustained engagement in pediatric neurorehabilitation remains a significant challenge, particularly for children with neuromotor impairments. Traditional therapy methods often lack personalized adaptability, which can limit adherence and effectiveness. The proposed framework addresses this gap by integrating multimodal sensor data, including EEG, posture, and gaze, to continuously monitor emotional, cognitive, and behavioral engagement during therapy sessions. This real-time assessment enables dynamic adaptation of game-based exercises with the aim of optimizing motivation, reducing disengagement, and promoting functional recovery. This concept was implemented in a clinical setting with children diagnosed with coordination disorders and neuropsychomotor delays. Preliminary results with four participants indicate that by tracking engagement levels and supporting session personalization, it is possible to stimulate the child’s motivation across multiple sessions. These findings suggest that incorporating adaptive, engagement-driven frameworks can provide a useful tool to improve rehabilitation efficacy, offering a way toward more personalized and responsive therapeutic strategies in pediatric neurorehabilitation.

People's successful coexistence with robots strictly depends on people's acceptance of robots' presence in their daily activities. This is particularly relevant when the robot's actions may interfere with or intrude on people's activities, creating discomfort and possible rejection. We believe that people's acceptance of a robot may vary depending on the activities they are involved in. In this study, we investigate the impact of a robot's actions on people's engagement in an activity while the robot has the task of monitoring them. We observed the behaviours of 18 older adults with respect to the robot while they were carrying out tasks that require different cognitive workloads (e.g., working at the PC, talking on the phone). We used subjective and objective metrics, such as social cues, to evaluate people's engagement in the robot and their disengagement in their own tasks. We observed that people were distracted by the robot's behaviours based on the cognitive loads required by their activity. Our results show that variation in people's engagement in the robot and the task is affected by their perception of the usefulness of and trust in the robot, and by individuals' personality traits and acceptance of the robot. People with higher trust in the robot, and a higher degree of conscientiousness and emotional stability, tend to continue with their task, paying less attention to the robot. We observed, in contrast, that a robot perceived as a social entity caught more easily their attention when people have a higher extroverted personality. Our findings also showed that variations in the affective and emotional demeanour of the participants are a predictor of their distraction to an external observer.

Providing a robot with the capabilities of understanding and effectively adapting its behaviour based on human mental states is a critical challenge in Human–Robot Interaction, since it can significantly improve the quality of interaction between humans and robots. In this work, we investigate whether considering human mental states in the decision-making process of a robot improves the transparency of its behaviours and mitigates potential human’s false beliefs about the environment during collaborative scenarios. We used Bayesian inference within a Hierarchical Reinforcement Learning algorithm to include human desires and beliefs into the decision-making processes of the robot, and to monitor the robot’s decisions. This approach, which we refer to as Hierarchical Bayesian Theory of Mind, represents an upgraded version of the initial Bayesian Theory of Mind, a probabilistic model capable of reasoning about a rational agent’s actions. The model enabled us to track the mental states of a human observer, even when the observer held false beliefs, thereby benefiting the collaboration in a multi-goal task and the interaction with the robot. In addition to a qualitative evaluation, we conducted a between-subjects study (110 participants) to evaluate the robot’s perceived Theory of Mind and its effects on transparency and false beliefs in different settings. Results indicate that a robot which considers human desires and beliefs increases its transparency and reduces misunderstandings. These findings show the importance of endowing Theory of Mind capabilities in robots and demonstrate how these skills can enhance their behaviours, particularly in human–robot collaboration, paving the way for more effective robotic applications.

Deception is a complex phenomenon that is deeply intertwined with human social interactions. Deception between humans often manifests through verbal, vocal, and visible behaviors. While it often has unethical implications, it can also serve valuable social functions, such as maintaining relationships, protecting emotions, or managing difficult situations. For this reason, it is also being investigated in robotic applications. In this work, we propose a framework for implementing specific robot behaviors through non-strategic cues of deception (known as leakage), during deceptive communication with humans. We dwell on the ethical dimensions of robotic deception by acknowledging the implication of possible physical or psychological harm in sensitive contexts, such as healthcare and assistive robotics. We propose to mitigate possible drops in trust towards robots using more transparent and human-like deceptive behaviors, by equipping robots with seemingly unintentional behaviors that betray deception. To this extent, we propose a low-risk educational scenario where a robot interacts with students in a problem-solving game to test deception leaking's effects on students' perceptions of the robot, perceived human-likeness, intentionality, engagement, and trust in the robot.

Global inequalities in access to essential resources such as education, healthcare, and technology continue to widen social and economic disparities, especially in underserved and underrepresented communities. The growing integration of foundation models and other machine learning systems in robots offers promising and personalized solutions that can adapt to various individuals, situations, and environments, potentially addressing some of these gaps. By learning from interactions and evolving with local conditions, these systems can provide individualized support, such as assisting older adults with daily tasks, aiding children with special needs in learning environments, or empowering people with disabilities to live more independently. Building trust and fostering collaboration between humans and robots will help ensure that these systems meet the unique needs of all individuals, especially within long-term human-robot interaction (HRI). With this year's theme of “Overcoming Inequalities with Adaptation”, in line with the overall theme of the conference “Robots for a Sustainable World”, the fifth edition of the ”Lifelong Learning and Personalization in Long-Term Human-Robot Interaction (LEAP-HRI)”l workshop aims to bring together insights across diverse disciplines, exploring how continually evolving robots can effectively operate in diverse environments, promoting greater equity, inclusivity, and empowerment for individuals and communities. The workshop aims to facilitate collaborations across diverse scientific perspectives through a keynote presentation, panel discussions, and in-depth discussions on the contributed talks, attempting to shape a more sustainable and equitable future through adaptive advancements in long-term HRI.

Assistive robots can be valuable conversational partners for cooperative tasks, such as storytelling, fostering creativity and social bonding. Through the use of foundational models, such as LLMs, robots can more effectively and naturally generate story narrations that are enjoyed by humans. In such a scenario, however, it is fundamental to consider the users’ feedback and reactions to adapt the story and the interaction in a way that actively sustains their interest. In this work, we propose an LLM-assisted storytelling generation method that employs different robot’s communication modalities to stimulate the user’s behavioral, affective, and cognitive engagement during the interaction and affect the narration of the story. Moreover, we investigated the introduction of an adaptive interaction policy to choose the most suitable actions based on the user’s observed engagement. We conducted a user study with 36 participants to assess our proposed approach, and demonstrated that it manages to effectively assist participants in an engaging way, with the robot being perceived as friendly and trustworthy. Moreover, the policy adaptation results in a perception of the robot with a higher arousal while a more interactive approach led to a better perceived social intelligence.

Patients falling from their beds is still one of the major complications of the hospital’s treatments. To address this issue, this research investigates the use of social robots for the identification of potential bed-related falls in hospitals. Using a robot camera and human pose estimation techniques, the patient’s position in the bed is extracted, and a threshold-based algorithm is used to identify any anomalies that could indicate a fall. Due to the absence of publicly available datasets, a synthetic dataset was created using a simulation environment to develop and tune the detection algorithm. A user study was conducted to validate the proposed approach and evaluate people’s perception of the robot. The system achieved an accuracy of 90.9% in the controlled setting using real data. Participants rated the robot as significantly more trustworthy and behaviorally aware when it detects a possible fall, suggesting that timely and meaningful assistance improved the perceived social competence of the robot. These findings highlight the feasibility of deploying social robots as monitoring systems in sensitive clinical settings, offering a cost-effective and socially acceptable solution.

For social robots to collaborate effectively, they must infer and correct false human beliefs, especially when misconceptions directly impact task outcomes or pose safety risks to humans. In this work, we investigated whether a robot’s ability to detect and rectify users’ false beliefs improves trust and perceived social competence. In an in-person between-subject study, 98 participants collaborated with a robot to solve a task. Participants interacted with either a robot that actively corrected their false beliefs by using Theory of Mind or one that complied with their incorrect instructions. Contrary to expectations, trust, mental state attribution, and perceived warmth or competence did not differ between groups. The results also showed that human reluctance to trust the robot’s input persisted, suggesting that belief correction alone cannot overcome relational barriers. In addition, the study showed that participants who trusted the robot’s corrections perceived it as more socially attuned.

Social robots have gained significant importance in Human-Robot Interaction, particularly in domains requiring personalized support, such as education, healthcare, and entertainment. The rise of e-sports has created a demand for effective coaching systems that can provide tailored guidance to players, paving the way for the integration of social robots as e-coaches. This pilot study explores the role of Furhat, a social robot, as an e-coach in a football video game. Thanks to computer vision techniques, Furhat analyzes the human player’s performance in real-time and provides adaptive feedback tailored to individual gameplay styles. The study investigates the effectiveness of the robot in providing both technical guidance and emotional support. Forty participants, divided into casual and hardcore gamers, engaged with Furhat in a controlled experimental setting. Results revealed that casual gamers sought general guidance and linguistic clarity, while hardcore gamers prioritized context-relevant, well-timed feedback. Although robot gender had a minimal overall impact, a statistically significant interaction was observed between robot gender and gamer type on adaptability perception (p = 0.046). Performance data showed that 85% of participants either maintained or improved their gameplay, with a majority reporting positive comfort and engagement levels during robotic interaction.

Food recommender systems have become valuable tools across various domains, including health-oriented applications that provide personalised dietary advice. Recent studies have shown the potential of integrating recommender systems with assistive robots to promote healthy eating habits, especially among older adults. While transformers and Large Language Models showed advanced reasoning capability for effective recommendation systems, they might have limited knowledge and understanding of the users’ personal preference and requirements. This lack of information can negatively affect their effectiveness and user’s satisfaction. We present a novel transformer-assisted, multi-interface recommendation system for generating food recommendations based on user profiles using a custom dataset including dietary and nutritional information. We conducted a user study with 40 participants for evaluating whether a robot is able to persuade users’ in accepting its food recommendation. Our study found that participants responded positively to the interactions with the robot, showing high satisfaction and trust in the recommendations.

Human-robot interaction in cooperative and assistive scenarios requires robotic systems to assess the task state and coherently choose their next move.Moreover, it is also fundamental to correctly recognize how the user's stress and emotional response are changing to offer support appropriately.The robot should be able to adapt to different user reactions, considering the situational context, and displaying empathetic behaviors aiming to support and encourage the users.In this work, we aim to assess the impact of empathetic supporting behaviors on the perception of the robot and the users' performance during a collaborative task, as opposed to assistive strategies focusing only on the task's performance.With this objective in mind, we propose a robotic architecture to assist a user in playing a memory game in real-time using a Furhat robot.We conducted a user study where 60 participants played with the robot to evaluate the effects of the two types of Theory of Mind on the assistive task and their perception of the robot.To this extent, the participants interacted with a robot endowed with either Cognitive or Affective Theory of Mind to respectively allow the robot to understand intentions and beliefs, and to show empathetic behaviors to improve the collaboration.The two conditions resulted in achieving the same results in terms of task performance, but the participants rated the emotionally engaged robot higher in perceived social intelligence.

In this work, we investigate whether the transparency of a robot’s behaviour is improved when human preferences on the actions the robot performs are taken into account during the learning process . For this purpose, a shielding mechanism called Preference Shielding is proposed and included in a reinforcement learning algorithm to account for human preferences. We also use the shielding to decide when to provide explanations of the robot’s actions. We carried out a within-subjects study involving 26 participants to evaluate the robot’s transparency. Results indicate that considering human preferences during learning improves legibility compared with providing only explanations. In addition, combining human preferences and explanations further amplifies transparency. Results also confirm that increased transparency leads to an increase in people’s perception of the robot’s safety, comfort, and reliability. These findings show the importance of transparency during learning and suggest a paradigm for robotic applications when a robot has to learn a task in the presence of or in collaboration with a human.

Background. People commonly use deception to gain advantages for themselves and their significant ones, such as with children, for educational purposes, or for protecting someone else feelings. As robots increasingly are being used in various human-centered environments, experts in robotics and social sciences are trying to adapt similar deceptive techniques to social robots, such as in assistive and service applications. However, robots’ ability to engage in deceptive behaviors presents both potential benefits and significant ethical challenges. In this work, we present a systematic review to synthesize current research on the implementation of deceptive robotic behaviors during Human–Robot Interactions (HRI), and its effects on people. Methods. Adopting a comprehensive and flexible methodological approach, we systematically searched Scopus and Web of Science without restricting the publication date. The review focused on studies that explicitly examined the effects of robotic deception on human participants, covering a broad spectrum of methodologies, populations, and outcomes. Results. A total of 16 studies met the inclusion criteria, showing that robotic deception in HRI leads to diverse emotional, cognitive, and behavioral responses. The findings indicate that robotic deception can have diverse impacts, ranging from eroding trust to enhancing engagement and performance under certain conditions. Conclusions. Our systematic review highlights the importance of careful design and management in robotic systems to harness the benefits of deception while mitigating its negative impacts on trust. We advise that future research should explore conditions under which deception may be beneficial and develop strategies to effectively manage its use in HRI.

Abstract Socially assistive robots represent a promising tool in assistive contexts for improving people’s quality of life and well-being through social, emotional, cognitive, and physical support. However, the effectiveness of interactions heavily relies on the robots’ ability to adapt to the needs of the assisted individuals and to offer support proactively, before it is explicitly requested. Previous work has primarily focused on defining the actions the robot should perform, rather than considering when to act and how confident it should be in a given situation. To address this gap, this paper introduces a new data-driven framework that involves a learning pipeline, consisting of two phases, with the ultimate goal of training an algorithm based on Influence Diagrams. The proposed assistance scenario involves a sequential memory game, where the robot autonomously learns what assistance to provide when to intervene, and with what confidence to take control. The results from a user study showed that the proactive behaviour of the robot had a positive impact on the users’ game performance. Users obtained higher scores, made fewer mistakes, and requested less assistance from the robot. The study also highlighted the robot’s ability to provide assistance tailored to users’ specific needs and anticipate their requests.

Interaction ruptures in human-robot interaction (HRI) refer to scenarios when seamless interactions are disrupted. Such ruptures can be directly observed by the robot at times, e.g., not responding to a human utterance. However, often the ruptures could be more passive and subtle and require an analysis of the human’s behavior. In this work, we focus on detecting such ruptures by analyzing multimodal information in a face-to-face interaction setting. More specifically, this paper describes the PRISCA team’s participation in the ERR@HRI Challenge 2024, which was recently proposed to benchmark multimodal learning approaches to interaction rupture detection in HRI. Central to our approach is a feature-fusion strategy for multimodal representation learning, where we train a neural network with separate recurrent layers that act as temporal encoders to learn modality-specific representations. Our approach was ranked 3rd in the ERR@HRI challenge. We present detailed experimentation on the released dataset from the challenge and a thorough analysis of the results. We further discuss the limitations of current approaches and implications for future works. Code will be made available at https://github.com/pradippramanick/prisca-errhri/.

The explainability of a robot’s actions is crucial to its acceptance in social spaces. Explaining why a robot fails to complete a given task is particularly important for non-expert users to be aware of the robot’s capabilities and limitations. So far, research on explaining robot failures has only considered generating textual explanations, even though several studies have shown the benefits of multimodal ones. However, a simple combination of multiple modalities may lead to semantic incoherence between the information across different modalities - a problem that is not well-studied. An incoherent multimodal explanation can be difficult to understand, and it may even become inconsistent with what the robot and the human observe and how they perform reasoning with the observations. Such inconsistencies may lead to wrong conclusions about the robot’s capabilities. In this paper, we introduce an approach to generate coherent multimodal explanations by checking the logical coherence of explanations from different modalities, followed by refinements as required. We propose a classification approach for coherence assessment, where we evaluate if an explanation logically follows another. Our experiments suggest that fine-tuning a neural network that was pre-trained to recognize textual entailment, performs well for coherence assessment of multimodal explanations. Code & data: https://pradippramanick.github.io/coherent-explain/.

The evaluation of groups’ dynamics and engagement levels in Human-Robot Interaction (HRI) is crucial for enabling a robot to adapt its behavior and achieve sustained interaction. To address these tasks, various methods have been explored, but these aspects are often treated in isolation. In this work, we present a unified machine-learning approach that addresses the recognition of social group dynamics in a multiparty setting, using that information as a possible index of engagement from an egocentric perspective. We introduce an interaction grouping classifier that considers the robot as a potential group member. Our approach utilizes neural networks trained on a combination of an existing dataset and a newly created one, specifically labeled for this study. The proposed method leverages egocentric data to detect social groups and employs this information as an index of engagement, as it requires agents to be part of the same interacting group, including the robot. Experimental results demonstrate high accuracy in both engagement recognition and group detection. Additionally, tests on F-formations reveal complexities in scenarios involving the robot, underscoring the challenges in these configurations.

This study explores the complex dynamics between humans and robots, focusing on how emotional states influence proxemics. We conducted a user study using a standard mobile robot to investigate whether emotions elicited from a loudspeaker, affect human perception of robot proximity. Based on previous research on Human-Human Interaction (HHI), we analysed participants’ responses to robots displaying different behaviours. Participants observed the robot’s approach while experiencing positive or negative emotions. Our findings suggest that emotional states induced by external stimuli can affect participants’ perception of robot proximity. In detail, the results indicate that while comfortable stopping distances were unaffected by participants’ emotional state, individuals who experienced positive emotions judged the same proxemics distance used while performing an avoidance behaviour to be more acceptable compared to the case of negative emotions. This study describes the extent to which our emotions can alter the perception of robot behaviours, ultimately affecting our acceptance of these novel social agents.

Persuasion is defined as the process of changing people’s attitudes and behaviours and social assistive robots are used to favour such changes through a variety of mechanisms. While human-robot deception is considered to present philosophical and psychological issues, it still may have positive consequences. In particular, prosocial deception can be beneficial and have positive influences, such as increasing trust. This work presents an exploration of robot deception and its effects on people’s changes in behaviour and trust in a social assistive robot. In particular, we explore the effects of different types of deception states (superficial, external, and hidden states) on people’s compliance with a social robot during an assistive game scenario. We collected the responses of 63 participants to evaluate their perception of trust in the robot, and their perception of the robot’s deceiving behaviours. Our results showed that the deceiving behaviours of the robot affected people’s trust and that superficial state deception has higher negative effects on people’s perception of the robot and trust in it compared to the other two deceptive states.

The development of social robots with advanced conversational capabilities aiming to engage humans in natural interactions has recently surged. This paper investigates the dynamic aspect of Human-Robot Interaction (HRI), focusing on the regulation of interpersonal distance based on human emotion. Through a user study with a humanoid robot, we explore how participants perceive and respond to rule-based versus randomly generated robot behaviours in adjusting inter-personal space during an unconstrained conversation. Results suggest that participants perceive the robot using rule-based behaviours as more socially competent and adaptable to human behaviour and emotions compared to the random ones. These findings highlight the importance of considering subtle non-verbal cues and adapting robot behaviour based on human emotions to improve the quality of HRI, and consequently facilitate the successful integration of human natural nuances in robots.

Abstract Growing evidence shows the potential benefits of robot-assisted therapy for children with Autism Spectrum Disorder (ASD). However, when developing new robotics technologies, it must be considered that this condition often causes increased anxiety in unfamiliar settings. Indeed, children with ASD have difficulties accepting changes like introducing multiple new technological devices in their routines, therefore, embedded solutions should be preferred. Also, in this context, robots should be small as children find the bigger ones scary. This leads to limited computing resources onboard as small batteries power them. This article presents a study on gesture recognition using video recorded only by the camera embedded in a NAO robot, while it was leading a clinical procedure. The video is 2D and low quality because of the limits of the NAO-embedded computing resources. The recognition is made more challenging by robot movements, which alter the vision by moving the camera and sometimes by obstructing it with the robot’s arms for short periods. Despite these challenging real-world conditions, in our experiments, we have tuned and improved state-of-the-art algorithms to yield an accuracy higher than $$90\%$$ 90 % in the gesture classification, with the best accuracy being $$94\%$$ 94 % . This level of accuracy is suitable for evaluating the children’s performance and providing information for the diagnosis and continuous assessment of the therapy. We have also considered the performance improvement of using a low-power GPU-AI accelerator embedded system, which could be included in future robots, to enable gesture analysis during the therapy, which could be adapted to the child’s performance. Graphical abstract

The lack of transparency in robotic learning processes poses a significant challenge to effective human-robot collaboration. This is particularly relevant in non-industrial settings because it prevents humans from adequately comprehending a robot’s intentions, progress, and decision-making rationale, which is essential for seamless interaction. To address this issue, this work presents a study where users observe a robot endowed with three distinct emotional/behavioural mechanisms for conveying transparent information about its learning process. The proposed mechanisms use inner speech, emotions, and a combination of the two communication styles (hybrid). To assess and evaluate the transparency of these behavioural models, a between-subject study was conducted with 108 participants. Results indicate that the people’s perception of the robot’s warmth dimension increased when it utilized a hybrid model to explain its learning state. Additionally, increased transparency was observed when the robot used inner speech during the learning process.

This work studies the impact of a robot’s appearance on how people judge robots’ trustworthiness in a public space scenario. An online experimental study was conducted to investigate the effect of the robot’s appearance on the perception of its role and on participants’ willingness to comply with the robot’s request to share sensitive information. The context of the interaction and the robot’s role was presented to the participants using a pre-recorded video filmed from a first-person perspective, encountering and interacting with a Pepper robot at a foreign University. We recruited 54 participants of different ages and nationalities. Each participant was tested with one of the three conditions in which the robot played the same role (which was not explicitly conveyed to the participants) but with a different appearance. Qualitative and quantitative measures were used to collect participants’ responses to evaluate their trust perception in showing their ID documents to the robot and letting the robot take a picture of them. Results showed that the context of interaction played a big part in helping the participants infer the robot’s role and the judgment of sensitivity of the information. Our findings provide insights and a better understanding of which are the factors affecting the perception of trustworthiness of robot for a privacy-sensitive human-robot interaction (HRI).

In the past few years, human-robot deception has been receiving growing attention in several fields (e.g., human-robot interaction, laws, philosophy, and psychology). While deception both in human-human and human-robot interactions may have positive consequences, it still presents philosophical and psychological controversy. In particular, verbal deceptions (i.e., in the form of lies or misleading information) may be judged as intentional behaviour at times. While intentionality has been recognised as fundamental in the development of trust, it is not yet fully clear which mechanisms can be designed to foster trust and the potential issues connected to deception. To this extent, in this study, we investigate whether the ability of mentalizing may be one of such mechanisms. We conducted a user study during a public fair, where participants played an assistive game with a robot endowed with Theory of Mind (ToM). We collected the responses from 37 participants to evaluate their perception of trust in the robot. During the game, the robot may occasionally have deceptive behaviours suggesting the wrong move to the human players. Our results showed that a deceptive robot was less trusted compared to a non-deceiving one. We also found that people’s perception of the robot was positively affected by the frequency of exposure to deception (i.e., wrong suggestions).

The ability to impact the attitudes and behaviours of others is a key aspect of human-human interaction. The same capability is a desideratum in human-robot interaction when it can have an impact on healthy behaviours. The robot’s interaction style plays a significant role in achieving effective communication, leading to better outcomes, improved user experience, and overall enhanced robot performance. Nonetheless, little is known about how different robots’ communication styles impact users’ performance and decision-making. In this article, we build upon previous work, in which a robot was endowed with two personality behavioural patterns: one more antagonist and other-comparative and the other one more agreeable and self-comparative. We conducted a user study where N = 66 participants played a game with a robot displaying the two multimodal communication styles. Our results indicated that i) participants’ decision-making was not influenced by the designed robot’s communication styles, ii) participants who interacted with the agreeable robot performed better in the game, and iii) the more participants are knowledgeable about robots, the lower they performed in the game.

People's acceptance and trust in robots are a direct consequence of people's ability to infer and predict the robot's behavior. However, there is no clear consensus on how the legibility of a robot's behavior and explanations should be assessed. In this work, the construct of the Theory of Mind (i.e., the ability to attribute mental states to others) is taken into account and a computerized version of the theory of mind picture sequencing task is presented. Our tool, called the human–robot interaction (HRI) video sequencing task (HRIVST), evaluates the legibility of a robot's behavior toward humans by asking them to order short videos to form a logical sequence of the robot's actions. To validate the proposed metrics, we recruited a sample of 86 healthy subjects. Results showed that the HRIVST has good psychometric properties and is a valuable tool for assessing the legibility of robot behaviors. We also evaluated the effects of symbolic explanations, the presence of a person during the interaction, and the humanoid appearance. Results showed that the interaction condition had no effect on the legibility of the robot's behavior. In contrast, the combination of humanoid robots and explanations seems to result in a better performance of the task.

Abstract We present and discuss a runtime architecture that integrates sensorial data and classifiers with a logic-based decision-making system in the context of an e-Health system for the rehabilitation of children with neuromotor disorders. In this application, children perform a rehabilitation task in the form of games. The main aim of the system is to derive a set of parameters the child’s current level of cognitive and behavioral performance (e.g., engagement, attention, task accuracy) from the available sensors and classifiers (e.g., eye trackers, motion sensors, emotion recognition techniques) and take decisions accordingly. These decisions are typically aimed at improving the child’s performance by triggering appropriate re-engagement stimuli when their attention is low, by changing the game or making it more difficult when the child is losing interest in the task as it is too easy. Alongside state-of-the-art techniques for emotion recognition and head pose estimation, we use a runtime variant of a probabilistic and epistemic logic programming dialect of the Event Calculus, known as the Epistemic Probabilistic Event Calculus. In particular, the probabilistic component of this symbolic framework allows for a natural interface with the machine learning techniques. We overview the architecture and its components, and show some of its characteristics through a discussion of a running example and experiments.

Abstract Socially assistive robotics (SAR) is getting a lot of attention for its potential in assisting elderly users. However, for robotic assistive applications to be effective, they need to satisfy the particular needs of each user and be well perceived. For this purpose, a personalization based on user’s characteristics such as personality and cognitive profile, and their dynamic changes is a crucial factor. Moreover, most of the existing solutions rely on the availability of specific technological infrastructures, generally requiring high economic investment, and that cannot be easily placed in different environments. Personalization and adaptation of assistive robotics applications to different user’s characteristics and needs, and even to different technological environments, are still not fully addressed in real environments. In the present work, the results of the UPA4SAR project are presented. The project aimed at providing a social robotic system to deliver assistive tasks for home care of patients with mild cognitive impairment in a personalized and adaptive way. We introduce the general architecture of the system and the developed robotic behaviors. Personalization and dynamic adaptation of assistive tasks are realized using a service-oriented approach by taking into account both user’s characteristics and environmental dynamic conditions. Field experimentation of the project was carried out with 7 patients, using the robotic system autonomously running in their homes for a total of 118 days. Results showed a reliable functioning of the proposed robotic system, a generally positive reaction, and a good acceptability rate from patients.

This work investigated how robots with different service style in food industries are perceived by customers. We analysed responses from 53 participants who evaluated the video interaction of a robot waiter while taking orders and that delivered a customer service experience based on different styles of a service robot interaction (service-focused vs. social-focused). Results indicated that social-focused robots are perceived to have more natural and pleasant behaviours compared with service-focused robots, and therefore better accepted by customers. Social-focused robots were also perceived as having a more persuasive effect on the users' choices. In contrast, service-focused robots were perceived as hasty, and participants did not feel in charge of the food choices to order.

The aim of this workshop is to foster the exchange of insights on past and ongoing research towards effective and long-lasting collaborations between humans and robots. This workshop will provide a forum for representatives from academia and industry communities to analyse the different aspects of HRI that impact on its success. We particularly focus on AI techniques required to implement autonomous and proactive interactions, on the factors that enhance, undermine, or recover humans' acceptance and trust in robots, and on the potential ethical and legal concerns related to the deployment of such robots in human-centred environments. Website: https://sites.google.com/view/trains-hri-2022.

People and robots may need to cross each other in narrow spaces when they are sharing environments. It is expected that autonomous robots will behave in these contexts safely but also show social behaviors. Thereby, developing an acceptable behavior for autonomous robots in the area mentioned above is a foreseeable problem for the Human-Robot Interaction (HRI) field. Our current work focuses on integrating legible non-verbal behaviors into the robot's social navigation to make nearby humans aware of its intended trajectory. Results from a within-subjects study involving 33 participants show that deictic gestures as navigational cues for humanoid robots result in fewer navigation conflicts than the use of a simulated gaze. Additionally, an increase in the perceived anthropomorphism is found when the robot uses the deictic gesture as a cue. These findings show the importance of social behaviors for people avoidance and suggest a paradigm of such behaviors in future humanoid robotic applications.

When navigating in a shared environment, the extent to which robots are able to effectively use signals for coordinating with human behaviors can ameliorate dissatisfaction and increase acceptance. In this paper, we present an online video study to investigate whether familiar acoustic signals can improve the legibility of a robot’s navigation behavior. We collected the responses of 120 participants to evaluate their perceptions of a robot that communicates with one of the three used non-verbal navigational cues (an acoustic signal, an acoustic in pair with a visual signal, and a dissimilar frequency acoustic signal). Our results showed a significant legibility improvement when the robot used both light and acoustic signals to communicate its intentions compared to using only the same acoustic sound. Additionally, our findings highlighted that people also perceived differently the robot’s intentions when they were expressed through two frequencies of the mere sound. The results of this work suggest a paradigm that can help the development of mobile service robots in public spaces.

Expressiveness is a critical feature for the communication between humans and robots, and it helps humans to better understand and accept a robot. Emotions can be expressed through a variety of modalities: kinesthetic (via facial expression), body posture and gestures, auditory, thus the acoustic features of speech, and semantic, thus the content of what is said. One of the most effective modalities to communicate emotions is through facial expressions. Social robots often show facial expressions with coded animations. However, the robot must be able to express appropriate emotional responses according to the interaction with people. In this work, we consider verbal interactions between humans and robots and propose a system composed of two modules for the generation of facial emotions by recognising the arousal and valence values of a written sentence. The first module, based on Bidirectional Encoder Representations from Transformers, is deployed for emotion recognition in a sentence. The second, an Auxiliary Classifier Generative Adversarial Network, is proposed for the generation of facial movements for expressing the recognised emotion in terms of valence and arousal.

Educational robotics is a field aiming at investigating the use of robots in schools to support teaching and learning activities. While several robotic solutions exist in support of the STEM teaching activities, in this work, we present "Classmate Robot" as a new social robot to be used in the classrooms as a support to the learning experience through interaction. Classmate Robot has been designed and developed to improve the effectiveness of the activities by providing a framework where the robot’s behaviors can be personalized, and learning applications can be easily integrated on top of the robot interaction capabilities. This approach aims to increase the engagement of learners. We introduce the ROS-based architecture developed that is divided into three main layers plus an application layer. As a social robot, it combines several multimodal social cues to interact and communicate with students and teachers. Moreover, the robot is endowed with a set of behaviors designed to be compliant with its role of "classmate" in the interaction with the students.

The success of human-robot interaction is strongly affected by the people’s ability to infer others’ intentions and behaviours, and the level of people’s trust that others will abide by their same principles and social conventions to achieve a common goal. The ability of understanding and reasoning about other agents’ mental states is known as Theory of Mind (ToM). ToM and trust, therefore, are key factors in the positive outcome of human-robot interaction. We believe that a robot endowed with a ToM is able to gain people’s trust, even when this may occasionally make errors.In this work, we present a user study in the field in which participants (N=123) interacted with a robot that may or may not have a ToM, and may or may not exhibit erroneous behaviour. Our findings indicate that a robot with ToM is perceived as more reliable, and they trusted it more than a robot without a ToM even when the robot made errors. Finally, ToM results to be a key driver for tuning people’s trust in the robot even when the initial condition of the interaction changed (i.e., loss and regain of trust in a longer relationship).

As people are getting more used to interact with social robots, their expectations of these robots also increase. However, robots are not always able to meet such expectations due to the limitations of the hardware and the software, or it might be possible that robots are simply unable to correctly elaborate the information about the agents, environment and context, and as consequence they produce erroneous behaviours. For example, a robot might get incongruous responses of people from the observations of multimodal systems. In such case, a technique used by humans is verbal irony, or sarcasm which it is a form of verbal irony, to recover from the situation. To this extent, we present a two parts study where we aimed to endow a robot with sarcasm. Results showed that social interacting behaviours, such as paying attention during a conversation, being transparent on the process of thinking and elaborating a response, allow people’s to perceive a robot with higher anthropomorphism and animacy. Moreover, robot failure recovery mechanisms are easier recognised by people when they use verbal incongruence.

During the last years, many solutions have been proposed to achieve a natural Human-Robot Interaction (HRI) and Communication paving the way to new paradigms of under-standing and adaptation based on mutual affective perception. Especially in human-robot social interaction, it is helpful not only that people can understand the robot’s behavioral state, but also robots possess the ability to detect, interpret and adaptively react to human affective responses. Typical approaches are able to assess humans’ affective responses from the observation of overt behavior. However, there are cases in which the overt observable behaviors could not match with the internal states (e.g., people with diseases compromising normal emotional responses). In such cases, having an objective measure of the users’ state from ‘inside’ is of paramount importance. This work presents an affect detection model able to provide a measure of the human affective state, with particular focus on the stress state, from the analysis of EEG users’ activity during the interaction with a social humanoid robot endowed with diverse affective elicitation behaviors. We argue that monitoring the stress state of a human during HRI is necessary to adapt the robot behavior in a way to avoid possible counterproductive effects of its use.

Guaranteeing people’s engagement during an interaction is very important to elicit positive and effective emotions in public service scenarios. A robot should be able to detect its interlocutor’s level and mode of engagement to accordingly modulate its behaviours. However, there is not a generally accepted model to annotate and classify engagement during an interaction. In this work, we consider engagement as a multidimensional construct with three relevant dimensions: affective, cognitive and behavioural. To be automatically evaluated by a robot, such a complex construct requires the selection of the proper interaction features among a large set of possibilities. Moreover, manually collecting and annotating large datasets of real interactions are extremely time-consuming and costly. In this study, we collected the recordings of human-robot interactions in a bartending scenario, and we compared different feature selection and regression models to find the features that characterise a user’s engagement in the interaction, and the model that can efficiently detect them. Results showed that the characterisation of each dimension separately in terms of features and regression obtains better results with respect to a model that directly combines the three dimensions.

When modeling the social behavior of a robot, the simulation of a specific personality or different interaction style may affect the perception of the interaction itself and the acceptability of the robot. Different interaction styles may be simulated through the use of verbal and non-verbal features that may not be easily recognized by the user as intended by the designer. For this reason, this study aimed to evaluate how three different robot interaction styles (i.e., Friendly, Neutral, and Authoritarian) were perceived by humans in the context of a robot carrying out cognitive tests. The Self-Assessment Manikin (SAM) was proposed to measure the perceived Valence, Arousal, and Dominance. We expected that a Neutral behavior is characterized by low Arousal, a Friendly by high Valence, and an Authoritarian by high Dominance. Moreover, the perception of a Socially Assistive Robot’s behavior is closely linked to trust, which is a key component to the success of any care-provider/user relationship. Hence, a Trust Perception Scale was used to explore the effect of the interaction style on trust. The results confirmed our hypothesis and showed a significant difference between each value with the others. Furthermore, we expected to obtain a higher value of trust with the Authoritarian since the performance of the users who interacted with the Authoritarian was better than the others. However, this hypothesis was not confirmed by the results.

The high request for autonomous human–robot interaction (HRI), combined with the potential of machine learning (ML) techniques, allow us to deploy ML mechanisms in robot control. However, the use of ML can make robots’ behavior unclear to the observer during the learning phase. Recently, transparency in HRI has been investigated to make such interactions more comprehensible. In this work, we propose a model to improve the transparency during reinforcement learning (RL) tasks for HRI scenarios: the model supports transparency by having the robot show nonverbal emotional-behavioral cues. Our model considered human feedback as the reward of the RL algorithm and it presents emotional-behavioral responses based on the progress of the robot learning. The model is managed only by the temporal-difference error. We tested the architecture in a teaching scenario with the iCub humanoid robot. The results highlight that when the robot expresses its emotional-behavioral response, the human teacher is able to understand its learning process better. Furthermore, people prefer to interact with an expressive robot as compared to a mechanical one. Movement-based signals proved to be more effective in revealing the internal state of the robot than facial expressions. In particular, gaze movements were effective in showing the robot's next intentions. In contrast, communicating uncertainty through robot movements sometimes led to action misinterpretation, highlighting the importance of balancing transparency and the legibility of the robot goal. We also found a reliable temporal window in which to register teachers’ feedback that can be used by the robot as a reward.

The COVID-19 pandemic will have a profound and long-lasting impact on the entire scientific endeavor. Scientists already are adapting research programs to adapt to changes in what is prioritized—and what is possible; educators are changing the way that the next generation of researchers are trained, and flagship conferences in many fields are being cancelled, postponed, and fundamentally transformed. These broad-reaching changes are particularly impactful to human-oriented domains such as human-robot interaction (HRI). Because in-person human-subject experiments can take a year or more to conduct, the research we will see published in the field in the immediate future may appear to be “business as usual,” with accounts of laboratory studies with large numbers of in-person participants. The research currently being performed, however, is of course a different story entirely. Studies that were under way when the current crisis began will be truncated, resulting either in work that cannot be published or in work whose true impact is difficult to accurately assess. Yet HRI research performed in the coming years will be changed in fundamentally different ways; the inability to perform—or expect future performance of—in-person human subjects research, especially research involving tactile or multiparty interaction, will change both the dominant methodological techniques employed by HRI researchers and the very research questions that the field chooses to—and is able to—address. These challenges demand that HRI researchers identify precisely how the field can maintain research quality and impact while the ability to conduct human-subject studies is severely impaired for an undetermined amount of time. A natural inclination may be simply to wait the crisis out in the hope of a speedy return to normalcy; however, in this article, we argue that the community can also take this opportunity to reevaluate and refocus how research in this field is conducted and how students are mentored in ways that will yield benefits for years to come after the current crisis has ended.

In this paper we advocate the use of Inductive Logic Programming as a device for explaining black-box models, e.g. Support Vector Machines (SVMs), when they are used to learn user preferences. We present a case study where we use the ILP system ILASP to explain the output of SVM classifiers trained on preference datasets. Explanations are produced in terms of weak constraints, which can be easily understood by humans. We use ILASP both as a global and a local approximator for SVMs, score its fidelity, and discuss how its output can prove useful e.g. for interactive learning tasks and for identifying unwanted biases when the original dataset is not available. Finally, we highlight directions for further work and discuss relevant application areas.

As the field of service robotics has been rapidly growing, it is expected for such robots to be endowed with the appropriate capabilities to interact with humans in a socially acceptable way. This is particularly relevant in the case of customer relationships where a positive and affective interaction has an impact on the users' experience. In this paper, we address the question of whether a specific behavioral style of a barman-robot, acted through para-verbal and non-verbal behaviors, can affect users' engagement and the creation of positive emotions. To that end, we endowed a barman-robot taking drink orders from human customers, with an empathic behavioral style. This aims at triggering to alignment process by mimicking the conversation partner's behavior. This behavioral style is compared to an entertaining style, aiming at creating a positive relationship with the users, and a neutral style for control. Results suggest that when participants experienced more positive emotions, the robot was perceived as safer, so suggesting that interactions that stimulate positive and open relations with the robot may have a positive impact on the affective dimension of engagement. Indeed, when the empathic robot modulates its behavior according to the user's one, this interaction seems to be more effective than when interacting with a neutral robot in improving engagement and positive emotions in public-service contexts.

Abstract Emotions represent a key aspect of human life and behavior. In recent years, automatic recognition of emotions has become an important component in the fields of affective computing and human-machine interaction. Among many physiological and kinematic signals that could be used to recognize emotions, acquiring facial expression images is one of the most natural and inexpensive approaches. The creation of a generalized, inter-subject, model for emotion recognition from facial expression is still a challenge, due to anatomical, cultural and environmental differences. On the other hand, using traditional machine learning approaches to create a subject-customized, personal, model would require a large dataset of labelled samples. For these reasons, in this work, we propose the use of transfer learning to produce subject-specific models for extracting the emotional content of facial images in the valence/arousal dimensions. Transfer learning allows us to reuse the knowledge assimilated from a large multi-subject dataset by a deep-convolutional neural network and employ the feature extraction capability in the single subject scenario. In this way, it is possible to reduce the amount of labelled data necessary to train a personalized model, with respect to relying just on subjective data. Our results suggest that generalized transferred knowledge, in conjunction with a small amount of personal data, is sufficient to obtain high recognition performances and improvement with respect to both a generalized model and personal models. For both valence and arousal dimensions, quite good performances were obtained (RMSE = 0.09 and RMSE = 0.1 for valence and arousal, respectively). Overall results suggested that both the transferred knowledge and the personal data helped in achieving this improvement, even though they alternated in providing the main contribution. Moreover, in this task, we observed that the benefits of transferring knowledge are so remarkable that no specific active or passive sampling techniques are needed for selecting images to be labelled.

In this work, we introduce a set of robot's behavior aimed at being used for monitoring and interaction with elderly people affected by Alzheimer disease. Robot's behaviors for a low cost robotic device rely on the use of microservices running on a local server. A microservice is an independent, self-contained, self-scope, and self-responsibility component of the robotic system proposed for decoupling the implemented functions needed to obtain the proper robot behaviors. The developed robotic behaviors include navigation, interaction, and monitoring capabilities. The requests and the signals of the patients are handled and managed relying on event-based communications between the system components. The use of design patterns like this one increases the overall reliability of a service composition. The system is currently operating in a private house with an elderly couple.

Companion robots used in the field of elderly assistive care can be of great value in monitoring their everyday activities and well-being. However, in order to be accepted by the user, their behavior, while monitoring them, should not provide discomfort: robots must take into account the activity the user is performing and not be a distraction for them. In this paper, we propose a Reinforcement Learning approach to adaptively decide a monitoring distance and an approaching direction starting from an estimation of the current activity obtained by the use of a wearable device. Our goal is to improve user activity recognition performance without making the robot's presence uncomfortable for the monitored person. Results show that the proposed approach is promising for real scenario deployment, succeeding in accomplishing the task in more than 80%of episodes run.

Social robots are effective in influencing and motivating human behavior. To gain a deeper understanding of how the robot emotional non-verbal behaviors might shape the human perception of the interaction while providing recommendations, we conducted a between-subjects experimental study using a humanoid robot in a movie recommendation scenario. This experiment aims at evaluating whether an incoherent use of emotional behavior, with respect to the presented contents, may produce a sort of humorous effect that positively affect the user perception of the recommendation. We evaluated, using an off-the-shelf solution, engagement and emotions shown by the users. Our results showed that a robot incoherent behavior does not distract the user, but it increases his/her engagement producing a positive emotional response. Such a difference is significant in the case of female subjects and depends on the considered emotions.

Socially assistive robotics technology is expected to play a crucial role in supporting home patients with neurological disorders. Nevertheless, the adoption of such technology in real home environments is still far to be reached since it presents several challenges mainly related to its acceptance in the everyday life. In this domain, a high degree of personalization is required usually obtained by a customization of a robotic system with respect to specific assistive tasks. On the contrary, socially assistive robots are usually general-purpose platforms requiring a considerable effort for customization. In this work, to limit static and costly customization of robotic systems, a service-oriented approach is adopted to represent and manage assistive tasks to be performed by a social robotic system, allowing to decouple a given functionality from its concrete implementation that can be provided by different devices and with different execution modalities. The service-oriented approach for robotics applications, known as Robot-as-a-Service, is becoming attractive for decoupling the robot hardware from the functionalities it provides.

Socially Assistive Robotics is becoming a promising technology for home care support to patients affected by neurological disorders, such as dementia. Several requirements needs to be addressed when such technology has to be adopted in realistic scenarios both from an economic and an acceptance point of view. In this work we present the approach adopted in designing and implementing a prototype of an assistive robotic system composed of low cost and general purpose devices that can be easily deployed in the patient homes, to support them in performing cognitive and physical stimulation activities that are known to have a beneficial effect on the progression of the disease. In order to improve the acceptability level of an invasive technology as a robotic system, activities are represented in terms of workflow of services, where the delivery mode of each service is personalized for an individual patient classified considering the cognitive status, and the personality profile. The system is composed of different layers, with a middleware able to automatically select the services that are more suitable to the patient profile, and to schedule them according to the daily routine of the patient.

Several studies focused on single human activity recognition, while the classification of group activities is still under-investigated. In this paper, we present an approach for classifying the activity performed by a group of people during daily life tasks at work. We address the problem in a hierarchical way by first examining individual person actions, reconstructed from data coming from wearable and ambient sensors. We then observe if common temporal/spatial dynamics exist at the level of group activity. We deployed a Multimodal Deep Learning Network, where the term multimodal is not intended to separately elaborate the considered different input modalities, but refers to the possibility of extracting activity-related features for each group member, and then merge them through shared levels. We evaluated the proposed approach in a laboratory environment, where the employees are monitored during their normal activities. The experimental results demonstrate the effectiveness of the proposed model with respect to an SVM benchmark.

Socially Assistive Robots used for elderly care are required to determine the location of a person and to approach him/her in order to provide assistance. Human tracking systems are applied to detect and track people that are already in the proximity of the robot, while its limited field of view makes the user easily lost. Moreover, navigation algorithms typically need the availability of reliable sensors on the robot and the possibility of marking possible user locations. On the contrary, in this work, we investigate the opportunity to use a reactive control mechanism for detecting and approaching people. Our approach is tested on two commercial mobile robots that present a different sensors configuration and by using off-the-shelf algorithms for people localization and tracking. Results show the feasibility of the approach with respect to the considered domain that does not require precise positioning, but hopes for a real application of such low-cost robot into the wild. Features of the considered robots and their impact on performance are also discussed.

Social Assistive Robots are mainly designed for tasks requiring the interaction with the user. However, they could also be involved in other non-interactive tasks which execution may potentially distract the user from his/her current activity. In this direction, we aim at evaluating the disengagement levels caused by a robot's movements in the human's surroundings. In particular, we consider a robotic system approaching an elder person to monitor his/her behavior, while he/she is occupied in carrying out a specific daily activity. We conducted a classic video analysis of human behaviors in order to identify relevant non-verbal disengagement signals such as the human gaze and pose variation when he/she is approached by the robot. Results obtained from ambient cameras are compared with the ones from the robot camera showing a moderate correlation between them. Additionally, the role of other contextual factors, such as the activity posture, approaching distance, and cognitive load are discussed in the direction of an automatic evaluation of the user's distraction to be used by the robot to plan its motion.

Robotic psychological assessment is a novel field of research that explores social robots as psychometric tools for providing quick and reliable screening exams. In this study, we involved elderly participants to compare the prototype of a robotic cognitive test with a traditional paper-and-pencil psychometric tool. Moreover, we explored the influence of personality factors and technology acceptance on the testing. Results demonstrate the validity of the robotic assessment conducted under professional supervision. Additionally, results show the positive influence of Openness to experience on the interaction with robot's interfaces, and that some factors influencing technology acceptance, such as Anxiety, Trust, and Intention to use, correlate with the performance in the psychometric tests. Technical feasibility and user acceptance of the robotic platform are also discussed.

Smart home systems have become increasingly widespread in the last few years. State-of-the-art smart home architectures concentrate on modeling the user physical behavior and on discovering possible behavioral pattern, while they provide very little personalization of the services based on the individual cognitive characteristics. In this direction, the analysis of the user activity on social networks offers a reliable and efficient way to obtain psychological traits of a human being in a manner that can be easily integrated with smart systems. In this paper, we outline a robot based architecture that blends ubiquitous computing and personality analysis to provide a custom-tailored system. An entertainment recommender scenario with a social robot is then analyzed as a case study.

A robotic system that is designed to coexist with humans has to adapt its behavioral and social interaction parameters not only with respect to the task it is supposed to accomplish, but also with respect to the human being it is interacting with by profiling her habits, preferences, and personality. This is particularly relevant in the domain of assistive robotics where the behavioral adaptability has been shown to enhance the users' acceptability of a robot. In this work, we propose a neuro-fuzzy-Bayesian system able to adapt the robot proxemics behavior with respect to the human users' personality and the action she is currently performing. The user's personality is evaluated according to the Big-Five factors model and the activity recognition is obtained by classifying data from a wearable device through the use of a Bayesian Network classifier. As shown by a statistical study, the proposed framework is capable of computing the most appropriate robot proxemics behavior in order to improve human feeling in interacting with artificial agents, such as robots.

The growing availability of services accessible through the network makes it possible to build complex applications resulting from their composition that are usually characterized also by non-functional properties, known as Quality of Service (QoS).To exploit the full potential of service technology, automatic QoS-based composition of services is crucial.In this work a framework for automatic service composition is presented that relies on planning and service negotiation techniques for addressing both functional and non-functional requirements.The proposed approach allows for dynamic service composition and QoS attributes, and it can be applied when services are provided in the contest of a competitive market of service providers without knowledge disclosure.

One of the major issue in designing group recommendation techniques relates to the difficulty of the evaluation process.Up-today, no freely available dataset exists that contains information about groups, like, for example, the group's choices or social aspects that may characterize the group's members.The objective of the paper is to analyze the possibility to make an evaluation of ranking-based groups recommendation techniques by using offline testing.Typically, the evaluation of group recommendations is computed, as in the classical single user case, by comparing the predicted group's ratings with respect to the single users' ratings.Since the information contained in the datasets are mainly such user's ratings, here, ratings are used to define different ranking metrics.Results suggest that such an attempt is hardly feasible.Performance seems not to be affected by the choice of ranking technique, except for some particular cases.This could be due to the averaging effect of the evaluation with respect to the single users' ratings, so a deeper analysis or specific dataset are necessary.

In this work, we propose a deep learning approach for the detection of the activities of daily living (ADL) in a home environment starting from the skeleton data of an RGB-D camera. In this context, the combination of ad hoc features extraction/selection algorithms with supervised classification approaches has reached an excellent classification performance in the literature. Since the recurrent neural networks (RNNs) can learn temporal dependencies from instances with a periodic pattern, we propose two deep learning architectures based on Long Short-Term Memory (LSTM) networks. The first (MT-LSTM) combines three LSTMs deployed to learn different time-scale dependencies from pre-processed skeleton data. The second (CNN-LSTM) exploits the use of a Convolutional Neural Network (CNN) to automatically extract features by the correlation of the limbs in a skeleton 3D-grid representation. These models are tested on the CAD-60 dataset. Results show that the CNN-LSTM model outperforms the state-of-the-art performance with 95.4% of precision and 94.4% of recall.

Recent studies on recommender systems raise attention to the importance of context, intended both as the external environment and even as the user's internal state, such as, for example, the mood in which the users are going to perform the recommended activities. This is a key factor also in the group recommendation domain, where the context is characterized by the presence of other people with whom the activities must be performed. In this case, social influences and relationships between users come into play, and the individual satisfaction that each user will obtain could change in relation to those social characteristics. In this work, we start an experimental analysis on how ties' strength and possible conflicts in a relationship can influence the opinion shift, with the aim to derive a model that can be used to adapt individual utilities to the "Group Context" before aggregating them into the group's ones.

Summary Recommendation systems based on collaborative filtering methods can be exploited in the context of providing personalized artworks tours within a museum. However, to be effectively used, we have several problems to be addressed: user preferences are not expressed as rating and recommendation systems must provide for new users efficient and simple preferences elicitation processes that do not require much effort and time. In this work, we present and evaluate 2 state‐of‐the‐art approaches that share the aim not to rely on individual item ratings. The first method uses a clustering algorithm to categorize items and provide recommendations, while the second one is inspired by the matrix factorization approach to select a couples of item groups that users have to evaluate to obtain preference profiles. We evaluate the 2 approaches with both an off‐line simulation and a user study with the aim to find the optimal configuration as well as to evaluate the effectiveness of the 2 proposed methods. Results show that the elicitation processes permit to obtain preference profiles in a time substantially less than the baseline method, while the differences in terms of prediction accuracy are minimal.

Recommendations to a group of users can be provided by the aggregation of individual users’ recommendations using social choice functions. Standard aggregation techniques do not consider the possibility of evaluating social interactions, roles, and influences among group’s members, as well as their personalities, which are, indeed, crucial factors in the group’s decision-making process. Instead of defining a specific social choice function to take into account such features, the proposed solution relies on the definition of a utility function, for each agent, that takes into account other group members’ preferences. Such function models the level of a user’s altruistic behavior starting from his/her agreeableness personality trait. Once such utility values are evaluated, the goal is to recommend items that maximize the social welfare. Performance is evaluated with a pilot user study and compared with respect to Least Misery. Results showed that while for small groups LM performs slightly better, in the other cases the two methods are comparable.

In many real life scenarios, which span from domestic interactions to industrial manufacturing processes, the objects to be manipulated are non-rigid and deformable, hence, both the location of the object and its deformation have to be tracked. Different methodologies have been applied in literature, using different sensors and techniques for addressing this problem. The main contribution of this paper is to propose a Weightless Neural Network approach for non-rigid deformable object tracking. The proposed approach allows deploying an on-line training on the shape features of the object, to adapt in real-time to changes, and to partially cope with occlusions. Moreover, the use of parallel classifiers trained on the same set of images allows tracking the movements of the objects. In this work, we evaluate the filtering/segmentation performance that is a fundamental step for the correct operation of our approach, in the scenario of pizza making.

People detection and tracking are essential capabilities in human-robot interaction (HRI). Typically, a tracker performance is evaluated by measuring objective data, such as the tracking error. However, in HRI applications, human- tracking performance does not have to be evaluated by considering it as a passive sensing behavior, but as an active sensing process, where both the robot and the human are involved within-the-loop. In this context, we foresee that the robotic non-verbal feedback, such as the head movement, plays an important role in improving the system tracking performance, as well as in reducing the human effort in the interactive tracking process. In order to verify this assumption, we evaluate a tracker performance in a joint task between a human and a robot, modeled as a game, and in three different settings. We adopt common HRI performance measures, such as the robot attention demand or the human effort, to evaluate the HRI human tracking performance scaling up with respect to the used robot feedback channels.

The aim of the proposed investigation is to provide the users with the capability of creating robot teams “on- the-fly” using grouping strategies expressed through speech. Our working hypothesis is that people are inclined to assemble objects into macro-entities, or groups, according to perceptual principles. We observed the real linguistic utterances used by individuals in a testing environment, showing that the type of robots and their mutual arrangements can affect both the choice of elements to form a team, and the way such choice is made. Moreover, we provide an initial insight for the capabilities needed by a robot for reasoning about its membership in a team.

We propose a framework where the human-robot interaction is modeled as a multimodal dialogue which is regulated by an attentional system that guides the system towards the execution of structured tasks. We introduce a simple case study to illustrate the system at work in different conditions considering top-down regulations and dialogue flows in synergic and conflicting situations.

In this work, we present a reliable and continuous gesture recognition method that supports a natural and flexible interaction between the human and the robot. The aim is to provide a system that can be trained online with few samples and can cope with intra user variability during the gesture execution. The proposed approach relies on the generation of an ad-hoc Hidden Markov Model (HMM) for each gesture exploiting a direct estimation of the parameters. Each model represents the best prototype candidate from the associated gesture training set. The generated models are then employed within a continuous recognition process that provides the probability of each gesture at each step. The proposed method is evaluated in two case studies: a hand-performed letters recognizer and a natural gesture recognizer. Finally, we show the overall system at work in a simple human-robot interaction scenario.

We propose a framework where the human-robot interaction is modeled as a multimodal dialogue which is regulated by an attentional system that guides the robot towards the execution of structured tasks. Specifically, we propose an approach where the dialogue between the human and the robot is represented as a Partially Obervable Markov Decision Process (POMDP), while the associated dialogue policy is enhanced by top-down attentional mechanisms that provide contextual and task-related contents. We introduce simple case studies that illustrate the system at work in different conditions considering top-down regulations and dialogue flows in synergistic and conflicting situations.

Task recognition and future human activity prediction are of importance for a safe and profitable human-robot cooperation. In real scenarios, the robot has to extract this information merging the knowledge of the task with contextual information from the sensors, minimizing possible misunderstandings. In this paper, we focus on tasks that can be represented as a sequence of manipulated objects and performed actions. The task is modelled with a Dynamic Bayesian Network (DBN), which takes as input manipulated objects and performed actions. Objects and actions are separately classified starting from RGB-D raw data. The DBN is responsible for estimating the current task, predicting the most probable future pairs of action-object and correcting possible misclassification. The effectiveness of the proposed approach is validated on a case of study, consisting of three typical tasks of a kitchen scenario.

Evaluating human machine interaction in the case of multimodal systems is often a difficult task involving the monitoring of multiple sources, data fusion and results interpretation. While subtasks are highly dependent on the specific goal of the application and on the available interaction modalities, it is possible to formalize this workflow into a standard process and to consider a generic measure to estimate the ease of use of a specific application. In this work, we present CoWME, a modular software architecture describing multimodal human machine interaction evaluation, from data collection to final evaluation, in a formal way, in terms of cognitive workload. Communication protocols between modules are described in XML while data fusion is delegated to a configurable rule engine. An interface module is introduced between the monitoring modules and the rule engine to collect and summarize data streams for cognitive workload evaluation. We present a deployment example showing how this architecture is deployed by monitoring an interactive session with an Android application taking into account stressed speech detection, mydriasis and touch analysis.

This paper presents a POMDP-based dialogue system for multimodal human-robot interaction (HRI). Our aim is to exploit a dialogical paradigm to allow a natural and robust interaction between the human and the robot. The proposed dialogue system should improve the robustness and the flexibility of the overall interactive system, including multimodal fusion, interpretation, and decision-making. The dialogue is represented as a Partially Observable Markov Decision Process (POMDPs) to cast the inherent communication ambiguity and noise into the dialogue model. POMDPs have been used in spoken dialogue systems, mainly for tourist information services, but their application to multimodal human-robot interaction is novel. This paper presents the proposed model for dialogue representation and the methodology used to compute a dialogue strategy. The whole architecture has been integrated on a mobile robot platform and has bee n tested in a human-robot interaction scenario to assess the overall performances with respect to baseline controllers.

Human safety and effective human-robot communication are main concerns in HRI applications. In order to achieve such goals, a system should be very robust, allowing little chance for misunderstanding the user's commands. Moreover, the system should permit natural interaction reducing the time and the effort needed to achieve tasks. The main purpose of this work is to develop a general framework for flexible and multimodal human-robot communication. The proposed architecture should be easy to modify and expand, adding or modifying input channels and changing the multimodal fusion strategies. In this paper, we introduce our general approach and provide a case study with two modalities (gesture and speech).

We present a robotic control system endowed with attentional mechanisms suitable for balancing the trade off between safe human-robot interaction and effective task execution. These mechanisms allow the robot to increase or decrease the degree of attention toward relevant activities modulating the frequency of the monitoring rate and the speed associated to the robot movements. In this framework, we consider pick-and-place and give-and-receive attentional behaviors.

The possible modulatory influence of motivations and emotions is of great interest in designing robotic adaptive systems. In this paper, an attempt is made to connect the concept of periodic behaviour activations to emotional modulation, in order to link the variability of behaviours to the circumstances in which they are activated. The impact of emotion is studied, described as timed controlled structures, on simple but conflicting reactive behaviours. Through this approach it is shown that the introduction of such asynchronies in the robot control system may lead to an adaptation in the emergent behaviour without having an explicit action selection mechanism. The emergent behaviours of a simple robot designed with both a parallel and a hierarchical architecture are evaluated and compared.

This work aimed to propose an adaptive web site in the field of cultural heritage that can dynamically suggest links, based on not intrusive profiling methodologies integrated with topographical information. A fundamental issue, typical in web sites that refer to real sites, is to help the user to orient himself geographically. Our system can support the user in its exploration of physical/virtual space suggesting new physical locations structured as a thematic itinerary through the excavations.

The main goal of our current research is the design of a robotic architecture that has the capability of adapting the robot's behavior to the rate of change of a dynamic environment. We present a model which takes free inspiration from some features of biological clocks. In particular, we associate the concept of Innate Releasing Mechanisms (IRM) to the concept of periodic behavior activation in order to link the variability of the behavior to the circumstances in which it is activated. We propose an architecture in which the frequency of access to the sensory system is modified in accordance to the environmental changes. To this purpose we use the Schema Theory paradigm. Some first experimental results are reported and discussed.

We present a novel technique to combine services, provided by mobile as well as fixed devices, on-the-fly and in a context-sensitive way within smart environments containing groups of people. The key intuition at the base of our approach is that, in those environments, similar functions are provided simultaneously by many agents on board of different devices; the issue becomes then of combining and coordinating their services in order to exploit their aggregated power or improve delivery. At the core of our work is the idea of implicit organizations, that is, teams of agents that continuously renegotiate how to provide services while the environment evolves. A concept demonstrator shows how implicit organizations can support a smart virtual meeting room. We foresee the application of our approach in assisting users and groups of users in active environments, such as an active museum. Here, our approach can enhance knowledge transfer and communication for groups visiting together cultural exhibits.