Robots are starting to be used in tasks involving human–robot interactions. For them to be efficient in these tasks, they must be seen as suitable interaction partners. One method to achieve this is to enable them to use proper verbal and non-verbal communication. Selecting non-verbal behaviours that appropriately complement the robot’s verbal messages is complex and requires roboticists to understand how each dimension of communication, as well as their combination, affects how the user perceives the message. In this work, we evaluated the effect of selecting appropriate non-verbal expressions given the robot’s speech on how users perceive the robot and its expressiveness. To do this, we conducted a within-subjects experiment where participants played cards with two robots — one that used a co-speech gesture prediction module for selecting its non-verbal expressions and another that used random expressions. The results showed that using the gestures predicted by our system improves the experience of participants during interactions. Specifically, participants perceived the robot using the co-speech gesture prediction module as having a higher level of agency, and as having a more coherent expressiveness. • We conducted a within-subjects that sought to evaluate how an appropriate selection of co-speech gestures has over how users perceive a social robot and its expressiveness. • Participants observed a researcher having a staged interaction with two robots, and then played a card game with them. • The addition of the co-speech gesture prediction module led to a statistically significant improvement in how users perceived the agency of a social robot. • The use of the gesture prediction module led to participants perceiving a higher coherence between the robot’s verbal and non-verbal communication when compared with a robot that uses random, neutral non-verbal gestures. • The gestures of the robot that used the co-speech gesture prediction module were perceived as being more expressive, although the differences were only marginally significant. These differences become significant among users 40 years old or younger.

The applications of social robots have continued to rise in number over the years: there are now mainly used in nursing homes and hospitals, performing different tasks like assistance, physical and cognitive rehabilitation, and supervision. However, robots’ ability to learn from humans remains a problem to be solved. This paper presents the development of an Active Object Learning architecture in which the robot can identify unknown objects and autonomously initiate the learning process, supported by user interactions. We study the feasibility of generating a real-time dataset, including techniques for reducing image repetition and online classification model training. The work considers the analysis of machine learning metrics such as F1-Score, accuracy, confidence, and hit rate as the robot learns new objects. The accuracy of the classifier remains constant as the number of classes increases, indicating that the regularisation methods reduce overfitting and decrease the generalisation gap. The results prove that the robot can perform the classifications correctly and differentiate between known and unknown objects. The paper also includes a case study to demonstrate the feasibility of the proposed system integrated into a real social robot.

Social robots are increasingly becoming part of our society. In this sense, such robots’ interaction capabilities depend largely on their ability to detect and respond to stimuli in their environment as humans would. This paper describes a bio-inspired exogenous attention-based perception architecture for social robots. Exogenous attention regulates those involuntary attentional processes driven by the salience of perceived stimuli. The architecture incorporates bio-inspired concepts, such as inhibition to return, focus of attention selection and detecting salient stimuli, that have been validated on a real social robot. We have integrated several multimodal mechanisms for recognising salient visual, auditory, and tactile stimuli. Then, the attention architecture’s performance was evaluated by comparing it to a ground truth gathered from interactions of 50 users, obtaining comparable responses in real-time.

Affective feedback from social robots is a useful technique for communicating to people whether they are interacting “well” with the robot or not. However, some users, such as people with physical or cognitive difficulties, may not be able to interact in all the desired ways. In these cases, affective feedback from the robot could be excessively negative—an “unhappy” robot, leading to an unrewarding experience for the user. This article presents a motivation-based architecture for an autonomous multimodal social robot, that incorporates an affective feedback mechanism which generates an affective state by combining the internal needs of the robot and the social interaction quality. The balance between these two factors can dynamically change, allowing the robot to adapt its affective feedback to the user's interaction style and capabilities. We have implemented this architecture in a simulation and in a MiRo social robot, and report experiments examining the behavior of the system in interactions with different experimental user profiles. The results show that the adaptive mechanism allows the robot to change its affective feedback to give more positive encouragement to users than in non-adaptive cases.

Socially assistive robots in healthcare have reported positive results in recent years, for example, in reducing the impact of mild cognitive impairment in older adults. The lack of a qualified workforce and the increase in the older adult population in developed countries have encouraged designers to develop socially assistive robots that operate autonomously by bringing in cognitive and decision-making methods to facilitate the caregivers’ tasks, select the most appropriate activities, and personalize the interaction. This paper presents the development of a cognitive human-inspired decision-making system for autonomous social assistive robots managing the personalized selection of exercises in cognitive stimulation and providing affective support to their users. The decision-making system receives inputs from the robot’s perceptions, user information stored in the robot’s memory, events in an agenda, and information from a bio-inspired module. These inputs generate autonomous decisions that drive the robot’s behavior depending on each situation. We show the system’s capacity, integrated into our Mini social robot, to adapt the interaction, select tailored exercises based on the user’s features, and execute exercises previously programmed by a caregiver to alleviate cognitive deterioration and accompany older people. Besides, the system generates a natural robot behavior based on biologically inspired methods to personalize activities, engage the user, and increase the number of robot services.

Artificial intelligence and robotics are advancing at an incredible pace; however, there is a risk associated with the data privacy and personal information of users interacting with these systems and platforms. In this context, the federated learning approach emerged to enable large-scale, distributed learning without the need to transmit or store any information necessary to train the learning models. In a previous paper, we presented a system capable of detecting, locating, and classifying what kind of contact occurs between humans and one of our robots using innovative contact microphone technology. In this work we go further, improving the previously presented touch system with a multi-user, multi-robot, distributed, and scalable learning approach that is able to learn in a collaborative and incremental way while respecting the privacy of the user’s information. The system has been successfully evaluated in a real environment with 28 different users divided in 7 different groups. To assess the performance of our system with this federated learning approach, we compared it to the same distributed learning system without federated learning. That is, the control group for this comparison is a central node directly receiving all the training examples obtained by each robot locally. We found that in this context the inclusion of federated learning improves the results concerning traditional distributed learning.

Adapting to dynamic environments is essential for artificial agents, especially those aiming to communicate with people interactively. In this context, a social robot that adapts its behaviour to different users and proactively suggests their favourite activities may produce a more successful interaction. In this work, we describe how the autonomous decision-making system embedded in our social robot Mini can produce a personalised interactive communication experience by considering the preferences of the user the robot interacts with. We compared the performance of Top Label as Class and Ranking by Pairwise Comparison, two promising algorithms in the area, to find the one that best predicts the user preferences. Although both algorithms provide robust results in preference prediction, we decided to integrate Ranking by Pairwise Comparison since it provides better estimations. The method proposed in this contribution allows the autonomous decision-making system of the robot to work on different modes, balancing activity exploration with the selection of the favourite entertaining activities. The operation of the preference learning system is shown in three real case studies where the decision-making system works differently depending on the user the robot is facing. Then, we conducted a human-robot interaction experiment to investigate whether the robot users perceive the personalised selection of activities more appropriate than selecting the activities at random. The results show how the study participants found the personalised activity selection more appropriate, improving their likeability towards the robot and how intelligent they perceive the system. query Please check the edit made in the article title.

The external aspects of a robot affect how people behave and perceive it while interacting. In this paper, we study the importance of the mouth displayed by a social robot and explore how different designs of an artificial LED-based mouths alter the participants' judgments of a robot's attributes and their attention to the robot's message. We evaluated participants' judgments of a speaking robot under four conditions: 1) without a mouth; 2) with a static smile; 3) with a vibrating, wave-shaped mouth; and 4) with a moving, human-like mouth. A total of 79 participants evaluated their perceptions of an on-video robot showing one of the four conditions. The results show that the presence of a mouth, as well as its design, alters the perception of the robot. In particular, the presence of a mouth makes the robot to be perceived more lifelike and less sad. The human-like mouth was the one participants liked the most and, along with the smile, they were the friendliest ones. On the contrary, participants rated the mouthless robot and the one with the wave-like mouth as the most dangerous ones.

This LBR describes a novel method for user recognition in HRI, based on analyzing the peculiarities of users voices, and specially focused at being used in a robotic system. The method is inspired by acoustic fingerprinting techniques, and is made of two phases: a)enrollment in the system: the features of the user's voice are stored in files called voiceprints, b)searching phase: the features extracted in real time are compared with the voiceprints using a pattern matching method to obtain the most likely user (match).

This paper presents a system in which a robot uses Active Learning (AL) to improve its learning capabilities for pose recognition. We propose a sub-type of Feature Queries, Rank Queries (RQ), in which the user states the relevance of a characteristic of the learning space. In the case of pose learning, these queries refer to the relevance of a single limb for a certain pose. We test the use of RQ with 24 users to learn 3 pointing poses and compare the learning accuracy against a passive learning approach. Our results show that RQ can increase the robot's learning accuracy.

An intuitive and robust user recognition system is at the key of a natural interaction between a social robot and its users. The gender of a new user can be guessed without explicitly asking it of her, which can then be used to personalize the interaction flow. In this LBR, a novel algorithm is used to estimate the gender of a person based on its morphological shape. More specifically, the vertical outline of the breast of the user is used to estimate his or her gender, based on similar shapes seen during training.

This paper develops a wireless communication system that connects robots with many remote control devices used by many different users. The most important issue of this system is safety. To get high safety level a quick and efficient communication system is required. The emergency system and its communication system must work in parallel and independently of the main control of the robot. The robot must react to an emergency signal, but as a previous step, it must make sure that the security system is enabled and it so must also have some knowledge of how many remote control devices are related and if any of them has lost the wireless connection. Besides all the research and design stage to develop the communication system, the system has been implemented and tested. To build it, a microcontroller Arduino Fio and a radio frequency module Xbee has been used. Finally, the system has been tested in order to characterize the communication system, settling, connection time and the battery life.

In this study, an autonomous social robot is living in a laboratory where it can interact with several items (people included). Its goal is to learn by itself the proper behaviors in order to maintain its well-being at as high a quality as possible. Several experiments have been conducted to test the performance of the system. The Object Q-Learning algorithm has been implemented in the robot as the learning algorithm. This algorithm is a variation of the traditional Q-Learning because it considers a reduced state space and collateral effects. The comparison of the performance of both algorithms is shown in the first part of the experiments. Moreover, two mechanisms intended to reduce the learning session durations have been included: Well-Balanced Exploration and Amplified Reward. Their advantages are justified in the results obtained in the second part of the experiments. Finally, the behaviors learned by our robot are analyzed. The resulting behaviors have not been preprogrammed. In fact, they have been learned by real interaction in the real world and are related to the motivations of the robot. These are natural behaviors in the sense that they can be easily understood by humans observing the robot.

In many occasions people need to go to certain places without having any prior knowledge about the environment. This situation may occur when the place is visited for the first time, or even when there is not any available map to situate us. In those cases, the signs of the environment are essential for achieving the goal. The same situation may happen for an autonomous robot. This kind of robots must be capable of solving this problem in a natural way. In order to do that, they must use the resources present in their environment. This paper presents a RFID-based signage system, which has been developed to guide and give important information to an autonomous robot. The system has been implemented in a real indoor environment and it has been successfully proved in the autonomous and social robot Maggie. At the end of the paper some experimental results, carried out inside our university building, are presented.

Research on dialog systems is a very active area in social robotics. During the last two decades, these systems have evolved from those based only on speech recognition and synthesis to the current and modern systems, which include new components and multimodality. By multimodal dialogue we mean the interchange of information among several interlocutors, not just using their voice as the mean of transmission but also all the available channels such as gestures, facial expressions, touch, sounds, etc. These channels add information to the message to be transmitted in every dialogue turn. The dialogue manager (IDiM) is one of the components of the robotic dialog system (RDS) and is in charge of managing the dialogue flow during the conversational turns. In order to do that, it is necessary to coherently treat the inputs and outputs of information that flow by different communication channels: audio, vision, radio frequency, touch, etc. In our approach, this multichannel input of information is temporarily fused into communicative acts (CAs). Each CA groups the information that flows through the different input channels into the same pack, transmitting a unique message or global idea. Therefore, this temporary fusion of information allows the IDiM to abstract from the channels used during the interaction, focusing only on the message, not on the way it is transmitted. This article presents the whole RDS and the description of how the multimodal fusion of information is made as CAs. Finally, several scenarios where the multimodal dialogue is used are presented.

User simultaneous detection and tracking is an issue at the core of human–robot interaction (HRI). Several methods exist and give good results; many use image processing techniques on images provided by the camera. The increasing presence in mobile robots of range-imaging cameras (such as structured light devices as Microsoft Kinects) allows us to develop image processing on depth maps. In this article, a fast and lightweight algorithm is presented for the detection and tracking of 3D clusters thanks to classic 2D techniques such as edge detection and connected components applied to the depth maps. The recognition of clusters is made using their 2D shape. An algorithm for the compression of depth maps has been specifically developed, allowing the distribution of the whole processing among several computers. The algorithm is then applied to a mobile robot for chasing an object selected by the user. The algorithm is coupled with laser-based tracking to make up for the narrow field of view of the range-imaging camera. The workload created by the method is light enough to enable its use even with processors with limited capabilities. Extensive experimental results are given for verifying the usefulness of the proposed method.

In this paper we are presenting the C++ Object Oriented Mechanics Library (OOML). The OOML is a WYGIWYM (what you get is what you mean) tool that allows the fast development of 3D objects for fabrication on a 3D Printer. Designing with a WYGIWYM paradigm help students to communicate and reason geometrically, by applying geometry in real-world settings, and by solving problems through the integrated study of number systems, geometry, algebra, data analysis, probability, and trigonometry. These designed objects can be converted to STL files. The STL file can be used in 3D printers for fast prototyping or with traditional mechanization processes. We have designed and evaluated with students the OOML. Results show that the OOML plus the 3D Printer boosts their creativity. As a non searched result, we have observed that OOML also helps students to understand better the Object Oriented Programming Paradigm.

In this paper we present an Open Source Community Oriented Project Based Learning (CO-PBL) educational model for engineering studies. We explain the pedagogical basis of our proposal and present a training course in which the model has been implemented. The chosen project has consisted of the design, building, and programming of a printable mobile robot (PrintBot). To print the robot's parts we used open source 3D printers, creating a community of students (Plastic Valley) interacting with other designers spread all over the world. As it is shown in this paper, this initiative has proven to be highly motivating to the students and it enabled an explosion of creativity.

Advances in voice recognition have made possible applications in robotics controlled by voice only. However, user input through gestures and robot output gestures both create a more vivid interaction experience. In this article, we present an aloud reading application offering all these interaction methods for the HRI-research robot Maggie. It gives us a testbed for user studies investigating the effect of these additional interaction methods.

In this paper, a new approach to the generation and the role of artificial emotions in the decision-making process of autonomous agents (physical and virtual) is presented. The proposed decision-making system is biologically inspired and it is based on drives, motivations, and emotions. The agent has certain needs or drives that must be within a certain range, and motivations are understood as what moves the agent to satisfy a drive. Considering that the well-being of the agent is a function of its drives, the goal of the agent is to optimize it. Currently, the implemented artificial emotions are happiness, sadness, and fear. The novelties of our approach are, on one hand, that the generation method and the role of each of the artificial emotions are not defined as a whole, as most authors do. Each artificial emotion is treated separately. On the other hand, in the proposed system it is not mandatory to predefine either the situations that must release any artificial emotion or the actions that must be executed in each case. Both the emotional releaser and the actions can be learned by the agent, as happens on some occasions in nature, based on its own experience. In order to test the decision-making process, it has been implemented on virtual agents (software entities) living in a simple virtual environment. The results presented in this paper correspond to the implementation of the decision-making system on an agent whose main goal is to learn from scratch how to behave in order to maximize its well-being by satisfying its drives or needs. The learning process, as shown by the experiments, produces very natural results. The usefulness of the artificial emotions in the decision-making system is proven by making the same experiments with and without artificial emotions, and then comparing the performance of the agent.

An objective of natural Human-Robot Interaction (HRI) is to enable humans to communicate with robots in the same manner humans do between themselves. This includes the use of natural gestures to support and expand the information that is exchanged in the spoken language. To achieve that, robots need robust gesture recognition systems to detect the non-verbal information that is sent to them by the human gestures. Traditional gesture recognition systems highly depend on the light conditions and often require a training process before they can be used. We have integrated a low-cost commercial RGB-D (Red Green Blue - Depth) sensor in a social robot to allow it to recognise dynamic gestures by tracking a skeleton model of the subject and coding the temporal signature of the gestures in a FSM (Finite State Machine). The vision system is independent of low light conditions and does not require a training process.

Human–robot interaction (HRI) 1 1HRI can be defined as the study of humans, robots, and the ways in which they influence each other. * Portions of this work were previously presented at the 26th Robotics Symposia which was held on-line in March 2021. is one of the main fields in the study and research of robotics. Within this field, dialogue systems and interaction by voice play an important role. When speaking about human–robot natural dialogue we assume that the robot has the capability to accurately recognize what the human wants to transmit verbally and even its semantic meaning, but this is not always achieved. In this article we describe the steps and requirements that we went through in order to endow the personal social robot Maggie, developed at the University Carlos III of Madrid, with the capability of understanding the natural language spoken by any human. We have analyzed the different possibilities offered by current software/hardware alternatives by testing them in real environments. We have obtained accurate data related to the speech recognition capabilities in different environments, using the most modern audio acquisition systems and analyzing not so typical parameters such as user age, gender, intonation, volume, and language. Finally, we propose a new model to classify recognition results as accepted or rejected, based on a second automatic speech recognition (ASR) opinion. This new approach takes into account the precalculated success rate in noise intervals for each recognition framework, decreasing the rate of false positives and false negatives.

When a reinforcement learning agent executes actions that can cause frequent damage to itself, it can learn, by using Q-learning, that these actions must not be executed again. However, there are other actions that do not cause damage frequently but only once in a while, for example, risky actions such as parachuting. These actions may imply punishment to the agent and, depending on its personality, it would be better to avoid them. Nevertheless, using the standard Q-learning algorithm, the agent is not able to learn to avoid them, because the result of these actions can be positive on average. In this article, an additional mechanism of Q-learning, inspired by the emotion of fear, is introduced in order to deal with those risky actions by considering the worst results. Moreover, there is a daring factor for adjusting the consideration of the risk. This mechanism is implemented on an autonomous agent living in a virtual environment. The results present the performance of the agent with different daring degrees.

Due to the anticipated future, extensive use of robots, human beings will probably share common spaces with them. The relationships between robots and humans will be conducted at close distances. Predicting people's future positions helps robots understand human behavior and react safely and naturally. In this paper, we propose a method for predicting people's positions in crossing behaviors, i.e. different trajectories people follow when they are crossing each other. We conducted a field experiment to gather various crossing behaviors of pedestrians in a shopping mall environment and analyzed them by focusing on “hot areas” spaces where people modify their trajectories for crossing. We clustered typical crossing behaviors in hot areas and modeled them using Hidden Markov Models for predictions. Our algorithm more accurately predicts the future positions of pedestrians by considering moving direction and speed.

In this paper a sequence manager system for Robot Teaching is presented. This system allows the user to edit, execute and debug the sequence by means of speech with a multimodal social robot. The ongoing goal of the paper is to make human-robot interaction easier for the non-expert users. To achieve this we are designing a game for children where they play to teach the robot a sequence of actions and conditions by means of human-robot interaction. A Sequence Function Chart (SFC) representation for sequence implementation is proposed. This representation is transparent for the non-expert user, that just uses natural language to interact with the robot. We think that this work will be notable for the development of social robots in our society and to bring these robots closer to the general public.

This paper presents a framework for multimodal human-robot interaction. The proposed framework is being implemented in a personal robot called Maggie, developed at RoboticsLab of the University Carlos III of Madrid for social interaction research. The control architecture of this personal robot is a hybrid control architecture called AD (Automatic-Deliberative) that incorporates an Emotion Control System (ECS) Maggie’s main goal is to interact establish a peer-to-peer relationship with humans. To achieve this goal, a set of human-robot interaction skills are developed based on the proposed framework. The human-robot interaction skills imply tactile, visual, remote voice and sound modes. The multi-modal fusion and synchronization are also presented in this paper.

Life on this planet is full of astonishing examples of cooperation. Individual species depend upon one another for sustenance, often forming surprising alliances to achieve a common goal: continuance of the species. The majority of living things also display amazing altruism in order to protect and provide the best care for their offspring, incomparable to any form of sacrifice shown by human beings. Studying the cooperation patterns between living things and their intelligent behaviors has been source of inspiration for many new algorithms, theories and systems. This paper addresses the concept of cooperation and why it is important. It highlights the available biologically-inspired models and algorithms, and their potential applications. The paper also presents a general typology of cooperation patterns, which can help to understand how systems could work cooperatively in an intelligent manner.

The building of remote laboratories for laboratory experiments in mobile robots requires expertise in a number of different disciplines, such as Internet programming, telematic and mechatronic systems, etc. Remote laboratories offer students access to complementary experiments, not available at their own university, as support to lectures. An intuitive user interface is required for inexperienced people to control the robot remotely. This paper describes a design pattern-based architecture to build remote laboratories for mobile robotics. The proposed remote laboratory is currently used to provide remote experiments on indoor mobile robotics, addressing different approaches to solve the main problems of mobile robotics, such as sensing, motion control, localization, world modeling, planning, etc. These experiments are being used in several mobile robotics and autonomous systems courses, at the undergraduate and graduate levels.

This paper presents practical, high-level topological navigation tasks, making use of our general purpose landmark learning and detection system, which includes the possibility of reading text or icons inside detected landmarks. Room identification from inside, without any initialization, is achieved through its landmark signature. Room search along a corridor is done by reading the content of room nameplates placed around for human use; this allow the robot to take high-level decisions, and results in a higher integration degree of mobile robotics in real life.

In this paper a system for generation of topological maps is presented. This system is considered as one of the deliberative skills of the mobile robots architecture named AD. AD is a two level architecture: deliberative and automatic. Those skills which require high computational time as consequence of high level reasoning are found in the deliberative level, while the automatic level skills interact with robot sensors and actuators. The topological map generated with this deliberative skill is the map belonging to the EDN navigation system, and is named Navigation Chart. In the Navigation Chart, the information obtained from the chart is stored as nodes and as edges. Nodes correspond to the sensorial events and edges correspond to the sensorimotor skills.

A human-mobile manipulator cooperation module is designed to support a target task consisting of the transportation of a rigid object between a mobile manipulator and a master human worker. Our approach introduces an intention recognition capability in the robot, based on the search for spectral patterns in the force signal measured at the arm gripper. The mobile manipulator takes advantage of this capability by generating its own motion plans in order to collaborate in the execution of the task. This has been designated as active cooperation.

This paper describes a vision-based landmark recognition system for use with mobile robot navigation tasks. A search algorithm based on genetic techniques for pattern recognition in digital images is presented. The developed system allows the topologic localization of a mobile robot using natural and artificial landmarks. Text strings inside landmarks can be read and interpreted, if present. The resulting system was tested onboard a B21 mobile robot and proved useful. The presented experimental results show the effectiveness of the proposed algorithm.

To navigate in complex environments, an autonomous mobile robot needs to reach a compromise between the need for reacting to unexpected events and the need for having efficient and optimized trajectories. Sensor based path planning can be used to achieve this goal. We present a new sensor based method which consist of building up local Voronoi diagrams using measurements from a scanning laser. The space is divided into regular cells, and the Euclidean distance is calculated between each cell and the objects. The cells which are equidistant to several objects belong to the Voronoi diagram. Experimental results obtained by running this algorithm are also presented. These results show that our method can be used in sensor based path planning.

This paper presents a fuzzy perception planner that takes into account the time cost, the suitability of every landmark detection and the different uncertainties the robot encounters along its path for mobile robot localization. The sensor used is a camera with a motorized zoom on a pan and tilt platform and the artificial landmarks are circles detected through normalized gray scale correlation. An extended Kalman filter is used to correct the position and orientation of the vehicle. The resulting self-localization module has been integrated successfully in a more complicated navigation system.

We present an implementation of a reinforcement learning algorithm through the use of a special neural network topology, the AHC (adaptive heuristic critic). The AHC is used as a fusion supervisor of primitive behaviors in order to execute more complex robot behaviors, for example go to goal, surveillance or follow a path. The fusion supervisor is part of an architecture for the execution of mobile robot tasks which are composed of several primitive behaviors which act in a simultaneous or concurrent fashion. The architecture allows for learning to take place at the execution level, it incorporates the experience gained in executing primitive behaviors as well as the overall task. The implementation of this autonomous learning approach has been tested within OPMOR, a simulation environment for mobile robots and with our mobile platform, the UPM Robuter. Both, simulated and actual results are presented. The performance of the AHC neural network is adequate. Portions of this work has been implemented within the EEC ESPRIT 2483 PANORAMA Project.>