Engagement is a key concept in Human-Robot Interaction (HRI), as high engagement often leads to improved user experience and task performance. However, accurately estimating engagement during interactions is challenging. In this study, we propose a Dynamic Bayesian Network (DBN) to infer user engagement from various modalities, including head rotation, eye movements, facial expressions captured through visual sensors, as well as facial temperature variations measured by a thermal camera. Data was gathered from a human-robot interaction (HRI) experiment, where a robot guided participants and encouraged them to share their thoughts and insights on environmental issues. Our approach successfully combines these diverse features to offer a thorough assessment of user engagement. The network was tested on its capacity to classify participants as either engaged or not engaged, achieving an accuracy of 0.83 and an Area Under the Curve (AUC) of 0.82. These findings underscore the strength of our DBN in detecting user engagement during interactions.

Humor is a key element in human interactions, essential for building connections and rapport. To enhance human-robot communication, we developed a humor-aware chat framework that enables robots to deliver contextually appropriate humor. This framework takes into account the interaction environment, and user's profile as well as emotional state. Two GPT models are used to generate responses. The initial one, named sensor-GPT, processes contextual data from the sensor along with the user's response and conversation history to create prompts for the second one, chat-GPT. These prompts can guide the model on how to integrate appropriate humor elements into the conversation, ensuring that the dialogue is both contextually relevant and humorous. Our experiment compared the effectiveness of humor expression between our framework and the GPT-40 model. The results demonstrate that robots using our framework significantly outperform those using GPT-4o in humor expression, extending conversations, and improving overall interaction quality.

This paper investigates how different levels of information impact user trust and cognitive load in teleoperated robotic systems. Participants performed tasks under three conditions: (C1) minimal information after initial visual feedback, (C2) verbal guidance through a graphical interface, and (C3) a combination of visual and verbal guidance via a graphical user interface (GUI) that shows the direction in which the user should move the robot to complete the task. Measurements included physiological responses such as galvanic skin response (GSR), eye blink rate, and facial temperature, along with task performance. The findings revealed that increased cognitive load reduced user trust and performance. When only minimal information was provided, participants experienced the highest cognitive load and lowest trust levels. Verbal guidance significantly reduced cognitive load and increased trust, whereas the combination of visual and verbal guidance caused cognitive overload, counteracting the expected increase in trust. This study underscores the importance of balancing information quantity and quality to enhance user experience and the efficiency of teleoperated robotic systems.

Trust plays a crucial role in user performance during collaborative human-robot interaction. This study examines how varying levels of autonomy and system errors affect user trust and cognitive load in collaborative tasks between robots and humans. Participants performed a collaborative task using a UR5 robotic arm to place four bottles of different shapes into a box within a three-minute time frame under three conditions: (C1) full manual control by the user, (C2) autonomous operation with few errors—where the robot fails to correctly place one out of four bottles and the user can intervene upon detecting failures, and (C3) autonomous operation with frequent errors—where the robot fails to correctly place three out of four bottles, with user intervention allowed upon failure detection. Physiological indicators such as blink rate, galvanic skin response (GSR), and facial temperature, along with task performance metrics such as success rate and completion time were tracked. The results showed that participants experienced the highest cognitive load in Condition 1, as indicated by higher NASA-TLX scores, increased blink rates (average of 65 blinks per minute), elevated facial temperatures, and higher GSR readings. Trust levels were lowest in Condition 3, with 74% of participants reporting low trust, highlighting the significant impact of robot reliability on user’s trust. A strong negative correlation was found between cognitive load and trust in Condition 3 suggesting that increased cognitive load due to frequent robot errors leads to decreased trust. These findings contribute to understanding how system errors and autonomy levels influence cognitive load and trust in collaborative human-robot tasks. The insights gained can inform the design of collaborative robotic systems that balance autonomy and reliability, enhancing user experience and performance.

Social robots have shown significant potential in enhancing learning experiences, and humor has been proven to be beneficial for learning. This study investigates the impact of both the presence and timing of humor on students’ learning outcomes and overall learning experience. A total of 24 participants were randomly assigned to one of the three conditions: (C1) interact with a robot with no humor, (C2) interact with a robot with humor at pre-defined moments during the lesson, and (C3) interact with a robot that triggers humor based on engagement levels. The results revealed that the humor at pre-defined moments condition (C2) led to significantly better learning outcomes and longer interaction times compared to the other two conditions. While the adaptive humor in Condition C3 did not significantly outperform Condition C1, it showed positive effects on participants’ perceived learning effectiveness and engagement. These findings contribute to the understanding of how humor, when strategically timed, can enhance the effectiveness of social robots in educational settings.

Nonverbal communication plays a crucial role in both human-human and human-robot interactions (HRIs), where facial expressions convey emotions, intentions and trust. Enabling humanoid robots to generate human-like facial reactions in response to human speech and facial behaviours remains significant challenges. In this work, we leverage human-human interaction (HHI) datasets to train a humanoid robot, allowing it to learn and imitate facial reactions to both speech and facial expression inputs. Specifically, we extend a sequence-to-sequence (Seq2Seq)-based framework that enables robots to simulate human-like virtual facial expressions that are appropriate for responding to the perceived human user behaviours. Then, we propose a deep neural network-based motor mapping model to translate these expressions into physical robot movements. Experiments demonstrate that our facial reaction–motor mapping framework successfully enables robotic self-reactions to various human behaviours, where our model can best predict 50 frames (two seconds) of facial reactions in response to the input user behaviour of the same duration, aligning with human cognitive and neuromuscular processes. Our code is provided at https://github.com/mrsgzg/Robot_Face_Reaction.

Driving autonomously in diverse environments remains a significant challenge, especially when transitioning between regions with distinct traffic cultures and regulations. While human drivers exhibit remarkable adaptability through experiential learning and cognitive modeling, current data-driven autonomous systems often struggle with environmental adaptation, interpretability, and continuous learning capabilities. In this work, we present LoDriver (Local Driver), a novel knowledge-based architecture that enhances the conventional scene understanding-decision-planning paradigm through cognitive-inspired dual-process reasoning for path planning. LoDriver integrates a reactive process and a deliberative mechanism, processing multi-modal scene descriptions through parallel pathways. It maintains a structured memory module that dynamically accumulates driving experiences, traffic regulations, and knowledge, enabling experience-based decision-making and continuous learning through systematic memory updates. Experimental evaluation on the nuScenes dataset demonstrates LoDriver's interpretability and enhanced performance compared to existing knowledge-driven models, highlighting its advantage in operating across different environments.

This study addresses the critical need for refined, reliable, and complete real-world trajectory data in the de-velopment of Advanced Driver Assistance Systems (ADAS), particularly for Adaptive Cruise Control (ACC) functions. We conducted a comprehensive comparison of car-following and deceleration scenarios across ten open-source datasets from multiple countries, encompassing both highway and urban environments. Focusing on key kinematic variables crucial for longitudinal behavior, we employed statistical measures and safety metrics to compare data sets across different driving regulations and road designs. Our findings reveal substantial overlaps in the distributions of logical parameters, despite the varied data sources and cultural contexts. However, we noted significant differences in safety-critical metrics, such as Time Headway and Time To Collision (TTC), highlighting culture-specific driving behaviors. Interestingly, Chinese datasets consistently exhibited the smallest distance head ways across all scenarios, yet maintained high TTC values (around 16s) compared to other datasets, suggesting a unique approach to risk management. To quantify these differences, we calibrated the Intelligent Driver Model using U.S. data and evaluated its transferability, demonstrating remarkable performance degradation when applied to non-U.S. datasets. These results provide crucial insights for developing globally applicable, yet culturally sensitive safety assessment methodologies for next-generation automated vehicles, highlighting the need for adaptive ADAS technologies that can accommodate regional driving norms while maintaining consistent safety standards. The code and extracted Longitudinal Trajectory data used in this study are available: https://github.com/imanetaourarti/Car-Following-analysis.

Trust plays a crucial role in user performance during teleoperated human-robot interaction. This study presents a reinforcement learning (RL) model that adapts to dynamic trust levels using physiological data and task performance metrics. Participants completed a complex teleoperation task under three conditions: (C1) limited feedback, (C2) AI-generated verbal guidance, and (C3) AI guidance paired with real-time RViz visualization. Physiological indicators, such as blink rate, galvanic skin response (GSR), and facial temperature along with task performance metrics like success rate and completion time were tracked. Statistical analyses revealed that increased task complexity in C1 reduced trust and increased cognitive load, leading to poorer performance. AI-generated guidance in C2 improved task understanding and performance, supporting Hypothesis H2. In C3, combining AI guidance with RViz visualization further boosted trust and reduced cognitive load, partially confirming Hypothesis H3. The RL model successfully adapted guidance strategies based on real-time user states, and additional testing showed that the agent’s adaptive strategies significantly increased user trust and improved performance. These results underscore the potential of adaptive RL models to enhance trust and efficiency in teleoperated human-robot systems.

The seamless transition of control between drivers and autonomous systems remains a critical challenge in automated driving, affecting both safety outcomes and overall user experience. To address this challenge, our study examines the effectiveness of two distinct haptic feedback approaches—pneumatic and vibrotactile—when implemented as intelligent interface components for takeover requests (TORs) during these transition periods. We specifically investigate how these haptic modalities can effectively signal drivers when human intervention is required, facilitating smoother control transitions from automated to manual driving. We designed a comprehensive experimental setup integrating these haptic modalities with audio and visual cues and evaluated their performance across nine interaction tasks to understand how multi-modal feedback influences driver responsiveness during takeover scenarios. Our findings reveal that multi-modal approaches incorporating either pneumatic or vibrotactile feedback, combined with standard visual cues, substantially outperform audio-only alerts in both response time and accuracy metrics for takeover requests (TORs). Notably, pneumatic feedback offered more natural sensation and smoother transitions than vibrotactile feedback, with pneumatic systems excelling in comfort while vibrotactile feedback better serves urgent takeovers. This first systematic comparison provides valuable insights for developing interfaces that balance effectiveness with comfort in human-machine systems.

Teleoperated robots, especially in hazardous environments, integrate human cognition with machine efficiency, but can increase cognitive load, causing stress and reducing task performance and safety. This study examines the impact of the information available to the operator on cognitive load, physiological responses (e.g., GSR, blinking, facial temperature), and performance during teleoperation in three conditions: C1 - in presence, C2 - remote with Visual feedback, and C3 - remote with telepresence robot. The findings from our user study involving 20 participants show that information availability significantly impacts perceived cognitive load, as evidenced by the differences observed between conditions in our analysis. Furthermore, the results indicated that blinking rates varied significantly among the conditions. The results also underline that individuals with higher error scores on the spatial orientation test (SOT), reflecting lower spatial ability, are more likely to experience failure in conditions 2 and 3. The results show that information availability significantly affects cognitive load and teleoperation performance, especially depth perception of the robot’s actions. Additionally, the thermal and GSR data findings indicate an increase in stress and anxiety levels when operators perform conditions 2 and 3, thus corroborating an increase in the user’s cognitive load.

This paper examines various non-linear state estimators for accuracy and robustness, estimating vehicle sideslip angle which is crucial for improving vehicle handling, stability, and safety in modern vehicle dynamic control systems. The study compares the performance of different state estimators under various driving conditions and driving scenarios, with a particular focus on a novel two-stage observer (of order n = 2) that combines a super-twisting sliding mode observer and a conventional sliding mode filter. The results demonstrate the effectiveness of the proposed observer, which outperforms other state estimators in terms of accuracy and robustness.

Advanced Driver Assistance Systems (ADAS) are rapidly becoming a standard feature in modern road vehicles, enhancing safety and driver comfort. As ADAS adoption expands across diverse geographical and cultural regions, the performance of camera-based perception systems may vary significantly due to environmental and expected social behaviour of the different actors. This paper explores the referred factors and evaluates the traffic environment complexity for vehicles with different levels of automation. In particular, we propose a novel modeling and quantitative assessment approach for environment complexity. Specifically, we compare a perception model trained on United States dataset with a dataset from India, a nation characterized by unique traffic patterns, signage conventions, and cultural norms to assess its performance variation, and to lay the basis for proposing influencing factors of traffic environment complexity. We establish a scheme of referential and additional static factors and based on an expert evaluation, environment complexity is established. The effectiveness of the proposed approach is testified by naturalistic driving data. These findings pave the way for future research in intelligent driving and emphasize the importance of addressing cultural nuances as vehicle automation levels increase.

Social robots have become increasingly prevalent in the context of one-on-one tutoring, serving as effective educational aids. In response to this trend, the present study was devised to conduct a comparative analysis between human tutors and robot tutors. Additionally, the study aims to investigate how varying previous knowledge in robots influence students’ tendencies for help-seeking. By examining the performance and physiological signals of participants, this research seeks to provide valuable insights into the effectiveness of social robots in educational contexts. 21 participants were divided into three groups, each seeking assistance from a human tutor (HT), seeking help from a robot without any prior knowledge of robots (RT1), and seeking help from a robot after gaining some understanding of its capabilities (RT2). Our results demonstrated that participants sought more help from robot than from human and participants in Group RT2 performed better than participants in Group RT1. However, participants experienced greater cognitive load when interacting with a robot tutor compared to interacting with a human tutor. Future work could focus on developing interventions to alleviate students’ cognitive load during interactions with robot tutors.

Expressing humor in social interactions presents a significant challenge for humans due to its intricate linguistic nature. This complexity is further magnified when teaching robots to express humor appropriately. Among the various expressions of humor, jokes are one of the most commonly used. Therefore, a well-annotated joke dataset holds significant promise in enhancing a robot’s ability to express humor effectively. This paper introduces a dataset comprising over two thousand jokes, with the aim of providing rich material and multidimensional selection criteria for the humor expression of the robot. The creation of this joke dataset involved a collaborative effort among robot experts studying HRI, psychologists with rich humor research experience, and GPT-3. The annotation process primarily concentrated on four dimensions within the dataset: the humor style of jokes, semantic words (aligned with semantic gestures), keywords, and ratings of joke funniness. We additionally outline several prospective applications of this dataset. We introduce a BERT-based neural network model trained on the dataset with semantic word labels. This model aims to empower robots to choose suitable semantic words from jokes and articulate them alongside corresponding semantic gestures. Moreover, we offer suggestions for utilizing jokes from this dataset to facilitate the adaptive expression of humor by social robots. These endeavors will further enhance the multi-modal humor expression capability of social robots.

Laughter serves as a subtle social signal in human interaction, playing an essential role in expressing emotions and facilitating social connections. However, laughter comes in various forms and is usually accompanied by different non-verbal expressions, such as facial expressions and gestures. These accompanying factors can significantly influence the effect of laughter in diverse contexts, thus complicating the research on laughter, especially in understanding its role in social dynamics. Consequently, endowing robots with the ability to appropriately use laughter in interactions with humans is still a big challenge. Our current study focuses on the effect of robot laughter on robot humor expression. Our objective is to investigate whether and how two factors, the type of laughter and the robot laughter gesture, impact the overall humor performance. In this study, we selected four types of laughter (sarcastic, joyful, embarrassed, and relieved laughter) from a laughter corpus based on four specific types of jokes (Affiliative, Aggressive, Self-enhancing, and Self-defeating). For each type of laughter, we designed distinct robot gestures. During the humor performance, the robot NAO delivered jokes accompanied by matching or mismatching laughter, with or without corresponding gestures. To enhance the quantity and diversity of experimental data, we conducted an online survey utilizing recordings of the robot’s humor performance. The experimental findings indicate that when the robot’s laughter matches the type of humor in the joke, participants rate the humor performance significantly higher compared to situations where there is a mismatch. Additionally, the results confirm the positive impact of robot laughter gestures on humor performance.

The Covid-19 pandemic has massively developed the use of distance learning. The limits of this practice have gradually come to light, both for students and for teachers. It is now crucial to design alternative solutions to overcome the shortcomings of videoconferencing in terms of involvement, concentration, learning, and equity. Social robots are increasingly used as tutors in the educational context and help improve teaching efficiency. Many psychology-based principles have been applied in education to guide instructional strategies, motivate students, and create a positive and productive learning environment. In this work, we use Regulatory Focus Theory (RFT), which categorizes an individual’s motivation into two types: Promotion and Prevention. Promotion-focused individuals are motivated by the potential for growth and achievement, whereas prevention-focused individuals are motivated by the potential for avoiding negative outcomes. Based on RFT, we aim to explore if and how the regulatory-focused behavior of the tutor robot can affect participants’ learning outcomes. In this work, a language learning scenario was designed with two conditions: (1) a robot tutor with promotion-focused behavior, (2) a robot tutor with prevention-focused behavior. The results are encouraging and support that promotion robot tutor can increase the learning efficiency of promotion participants and prevention robot tutor will enhance the learning interest of prevention participants.

To develop robots that can show cognitive functions, we must learn from the knowledge of human cognition. Existing biological and psychological evidence suggests that self-face perception and sensorimotor learning mechanisms play a crucial role in self-recognition. However, one of the most important self-identity cues – facial information – has not been extensively studied in the robot self-recognition task. Current research on robot self-recognition primarily relies on the recognition of high-precision targets and tracking of manipulator motions, where the self-perception of facial information is not well studied. In this work, we propose a novel approach to achieve self-recognition via self-perception of facial expressions. Specifically, we developed a Conditional Generative Adversarial Network (CGAN) model using the knowledge on human cognitive and sensorimotor functions. It allows the robot to be aware of self-face (i.e., off-line model). Passing the observed visual variations in a mirror and comparing them to self-perceptive information, the robot can recognize the self through an online Bayesian learning regression. The results of our first experiment show that the robot can recognize itself in a mirror. The results from the second experiment show that our algorithm could be tricked by a similar robot with the same facial expressions, which is similar to the rubber hand illusion (RHI).

Some robots for human-robot interaction are designed with female or male physical appearance. Other robots are endowed with no gender characteristics, namely genderless robots, such as Pepper and NAO robot. A robot with male or female physical appearance should possess the mapped speech gender style during a natural human-robot interaction, which can be learned from humans' male or female speech. In this paper, we make a new trial to synthesis gender-free speeches for physically genderless robots, which is promising in order to improve a more natural human-robot interaction with genderless robots. Our gender style-controlled speech synthesizer takes the speech text and gender style embedding as inputs to generate speech audio. A speech gender encoder network is used to extract the embedding of the speech gender style with female and male speeches as input. Based on the distribution of the female and male gender style embedding, we explore the gender-free speech style embedding space where we sample some gender-free embedding vectors to generate genderless speech audio. This is a preliminary work where we show how the genderless speech audio wave will be synthesized from text.

To be seamlessly integrated in human-centered environments, robots are expected to have intelligent social capabilities on top of their physical abilities. To this end, research in artifi-cial intelligence and human-robot interaction face two major challenges. Firstly, robots need to cope with uncertainty during interaction, especially when dealing with factors that are not fully observable and hard to infer (latent variables) such as the states representing the dynamic environment and human behavior (e.g., intents, goals, preferences). Secondly, robots need to communicate their behaviors to agents (humans and other robots in the environment) in a clear and understandable manner. Therefore, robots need to be context-aware: being able to perceive and understand their surroundings, and adapt their functionalities accordingly.

Humor usually plays a positive role in social activities. We posit that endowing a social robot with humor ability can enhance expressive human-robot interaction. People’s perception on humor is different, and therefore, making the robot expressing humor in an appropriate way is a challenge. The main aim of this paper is to explore the correlation between people’s perception on different types of jokes and their humor styles (Affiliative, Self-enhancing, Self-defeating, Aggressive). In the experiment, we used the humanoid robot Pepper to perform different types of jokes. Both subjective (jokes rating) and objective measures (RGB and thermal images) were used. The latter method was employed to extract facial features (facial action unit and facial temperature). After extracting and analyzing the data of both measurement methods, we found that the Self-defeating humor style positively affects people’s rating on all types of jokes. In addition, there is also a positive correlation between people’s humor style scores and the degree of happiness.

The human gestures occur spontaneously and usually they are aligned with speech, which leads to a natural and expressive interaction. Speech-driven gesture generation is important in order to enable a social robot to exhibit social cues and conduct a successful human-robot interaction. In this paper, the generation process involves mapping acoustic speech representation to the corresponding gestures for a humanoid robot. The paper proposes a new GAN (Generative Adversarial Network) architecture for speech to gesture generation. Instead of the fixed mapping from one speech to one gesture pattern, our end-to-end GAN structure can generate multiple mapped gestures patterns from one speech (with multiple noises) just like humans do. The generated gestures can be applied to social robots with arms. The evaluation result shows the effectiveness of our generative model for speech-driven robot gesture generation.

Many Advanced Driver Assistance Systems (ADAS) and chassis systems can be found within the same passenger car. Most of these systems are controlled separately using standalone controllers. Several studies show the benefits of coordinating several systems especially in simultaneous operations. However, most of the studies tend to ignore the time-delays that exist in the control loop. This time-delays can destabilize the vehicle in severe situations. This paper provides a solution to face time-delays in the context of integrated ADAS and chassis systems. Results show the benefits of the control logic proposed in terms of stability and performance. This brings one step closer towards standardization of global vehicle motion control.

Trust is related to the performance of human teams, making it a significant characteristic, which needs to be analyzed inside human-robot teams. Trust was researched for a long time in other domains like social sciences, psychology, and economics. Building trust within a team is formed through common tasks and it depends on team performance and communication. By applying an online game based tasks for human-robot teams, the effects of three communication conditions (communication without text and verbal interaction, communication with text and verbal interaction related/not related to the task) on trust are analyzed. Additionally, we found that the participants’ background is linked to the trust in the interaction with the robot. The results show that in a human-robot team the human trust will increase more over time when he/she is working with a robot that uses text and verbal interaction communication related to the task. They further suggest that human trust will decrease to a lower extent when the robot fails in doing the tasks if it uses text and verbal communication with the human.

Our work focuses on detecting deception in Human-Robot Interactions (HRI) by using measurement techniques that are appropriate for such interactions. In our previous research works, we obtained interesting results by using thermal and RGB-D cameras. In this paper, we approached this aspect from a different angle and used a lab-designed armband to accurately measure the participants' heart rate and skin conductance. We also developed a deception card game scenario that entices human participants to lie either to a robot or a human game partner, allowing us to monitor and understand the correlations between human physiological manifestations and their trustworthiness. Our results show the existence of statistically significant correlations between the participants' deceptive behaviour and their heart rate, skin conductance, face position, and face orientation. These results allow us to improve robots' ability to detect deception in HRI.

For socially assistive robots, being able to recognize basic human actions is an important capability. The sensors, which are frequently mounted on most recent robots, such as RGB-D and thermal cameras, as well as the advances in deep learning have enabled the research on activity recognition to grow. In this paper, we collected our own dataset of actions in a home-like scenario, which contains thermal imagery in addition to RGB-D data and we proposed a method based on Long-term Recurrent Convolutional Networks (LRCN). We showed that our method has an accuracy comparable with the state-of-the-art. We also proved that thermal information can improve the recognition accuracy. Furthermore, we tested the real-time capability of our system and conducted a real-time experiment with a robot (Pepper robot from Softbank Robotics) so as to investigate the effect of a robot enabled with action recognition capability in a human-robot interaction.

Companion robots are more and more taking the role of caregivers for elderly people. Elderly people sometimes take the advice given by their family members or caregivers as a criticism. In this context, persuasive communication skills could be helpful. A social psychology theory called Regulatory Focus states that people have one of two inclinations when taking decisions: Promotion or Prevention Focus. Also, based on these inclinations, people can be influenced by the way the message is sent, including the speed of the speech and the amplitude of body gestures. In this paper, we analyze the influence of Regulatory Focus on a negotiation scenario, using 3 conditions: (1) a robot with a promotion behavior, (2) a robot with a prevention behavior, and (3) a robot with a neutral behavior. Our results support the results found in the psychology literature related to Regulatory Focus, suggesting that Promotion participants were more influenced by the robot showing a Promotion based behavior. Moreover, Prevention participants were more relaxed on the condition with the robot showing a Prevention based behavior, and accepted the biggest concession between the initial and final offer.

This paper investigates how individuals react in a situation when an experimenter (human or robot) either tells them to stop in the middle of playing the Jenga game, or accidentally bumps into a table and makes the tower fall down. The mood of the participants and different physiological parameters (i.e., galvanic skin response (GSR) and facial temperature variation) are extracted and analysed based on the condition, experimenter, and psychological questionnaires (i.e., TEQ, TEIQ, RST-PQ). This study was a between participants study with 23 participants. Our results show that multiple GSR parameters (e.g., latency, amplitude, number of peaks) differ significantly based on the condition and the experimenter the participants interacted with. The temperature variation in three regions of interest (i.e., forehead, left, and right periorbital regions) are good indicators of how ready an individual is to react in an unforeseen situation.

Most of automotive researches focus on autonomous vehicles. Studies regarding trajectory planning and trajectory tracking became preponderant. As in case of commercial ground vehicles there is a driver in the loop, one should raise the important question of how the trajectory should be tracked. In this paper, we investigate the influence of controlling integrated chassis systems on the vehicle's behavior. A fixed Model Predictive Control is used to track the trajectory. Tunable vehicle motion control is however used to provide different motion feelings. Results show that a specific trajectory could be followed in different manners. Therefore, vehicle dynamics can be and should be controlled in such a way to generate adaptive trust feelings to passengers in case of autonomous driving.

Enabling robots to determine how physiological parameters vary in relation to the profile of an individual can lead to a better adaptation of the behavior of the robot to the needs of the individual it interacts with. This paper investigates how physiological parameters (i.e., blinking, galvanic skin response (GSR), facial temperature variation) vary in a news reading task, based on various types of stimuli (i.e., auditory and visual) that TIAGo robot uses to present the news. The results from a within participant study with 11 participants are reported. We also consider the impact of personality and the user sensory profile (based on Adult and Adolescent Sensory Profile (AASP)) for the physiological parameters variation. Our results show that blinking is the main physiological parameter that varies in the non-stressing task of news reading.

In the context of a worldwide aging population, it is important to find solutions to help the elderly maintain their cognitive functions. This research was done in the context of the ENRICHME. We investigate the outcome of a 5 day intensive cognitive stimulation with an elderly individual. Each day was composed of two sessions (one at 11am, and one at 3pm). During each session the participant played three cognitive games (i.e., digit cancellation, integer matrix task, Stroop game), two of them having two difficulty levels. The mood of the participant was also recorded before and after each interaction session. Evidence was found that even after a few sessions, the performance of the participant increased for all games. The performance of each game and each difficulty level was analyzed based on the interaction time (1lam or 3pm) and the interaction day. A detailed analysis of the performance is presented together with a discussion of these results.

Social robots are expected to be part of everyday life of people. This will generate interactions between humans and robots that may have positive or negative effects on the users. In order to minimize the negative effects and increase robot persuasiveness, robots should behave in an appropriate manner by adapting to their users. How to achieve this adaptation remains a challenge. We propose the usage of the Regulatory Focus Theory, user physiological state, and game-performance information in order to detect user stress and adapt the behavior of the robot. We present a longitudinal experiment conducted with 35 participants in a game-like scenario. The robot was trained for adapting to the regulatory focus of the users and decreasing their stress while they were playing the game. For this reason, we trained the robot with 12 participants with Chronic Promotion State and with 12 participants with Chronic Prevention State. We used a Q-Learning algorithm based on the Regulatory Focus of the participants, user stress, and task performance. The model obtained was tested with 2 groups (6 and 5 participants, respectively) according to their Chronic Regulatory Focus. Results show that our system was able to generate a robot behavior capable of increasing robot persuasiveness and reducing user stress, which is of great importance for social robots.

The following topics are dealt with: human-robot interaction; mobile robots; humanoid robots; learning (artificial intelligence); service robots; medical robotics; control engineering computing; motion control; manipulators; handicapped aids.

With the development of compliant mechanisms and tactile skins, social robots are more inclined to physically interact with humans. Touch has been proved highly beneficial in interpersonal social interaction. This motivates social robots to exhibit affective touch abilities. However, most of the studies tackle the consequences and the perception of touch instead of how touching is performed. Understanding the touch manner enables to control robots in a more rich and natural way, but also to detect the intention of the user while touching the robot. This paper focuses on one specific social interaction based on touch, the handshake, and it analyses the pressure exerted by the participants depending on their mood. A corpus of daily repeated handshakes of 11 participants, during 16 days in Human-Human and Human-Robot interactions was produced. A handshake descriptor of 6 variables was computed and allowed to discriminate the participants' behavior. One of the variables (i.e., the maximum pressure on the fingers) enabled to discriminate the mood “Bored” from the others.

Imitation is of major importance during social interactions, would it be between humans or between a human and a robot. This is even more true when considering users with special needs. In this paper, we describe an experimental imitation task protocol using a robot Nao that we designed to assess whether sensory profiles of children with Autistic Spectrum Disorder (ASD) influence their capabilities to imitate or to initiate gestures that are going to be imitated. We based our work on the hypothesis that children with an overreliance on proprioceptive cues and hyporeactivity to visual cues have a greater difficulty imitating and improve their skills more slowly than children with an overreliance on visual cues and hyporeactivity to proprioceptive cues. Twelve children and teenagers with ASD participated in seven imitation sessions over eight weeks. As expected, we observed that children with an overreliance on proprioceptive cues and hyporeactivity to visual cues had more difficulties imitating the robot than the other children. Moreover, the repeated sessions revealed to have positive effects on social behaviors displayed by all children (gaze to the partner, imitations) toward a human partner after the sessions with the robot. We conclude on the possible impacts of such results on the design of social human-robot interactions for users with ASD.

Robots are expected to interact with persons in their everyday activities and should learn the preferences of their users in order to deliver a more natural interaction. Having a memory system that remembers past events and using them to generate an adapted robot's behavior is a useful feature that robots should have. Nevertheless, robots will have to face unknown situations and behave appropriately. We propose the usage of user's personality (introversion/extroversion) to create a model to predict user's preferences so as to be used when there are no past interactions for a certain robot's task. For this, we propose a framework that combines an Emotion System based on the OCC Model with an Episodic-Like Memory System. We did an experiment where a group of participants customized robot's behavior with respect to their preferences (personal distance, gesture amplitude, gesture speed). We tested the obtained model against preset behaviors based on the literature about extroversion preferences on interaction. For this, a different group of participants was recruited. Results shows that our proposed model generated a behavior that was more preferred by the participants than the preset behaviors. Only the group of introvert-female participants did not present any significant difference between the different behaviors.

For service robots to be well received in our daily lives, it is desirable that they appear as friendly as possible rather than some unfriendly characters. While a robot's physical appearance influences this perception, its behavior also has an impact. In order to be sure that a specific robot behavior will be correctly perceived, we propose to its potential users to shape the robot's behavior. In this paper, a specific behavior, “approaching a person”, is evaluated with a Robosoft Kompaï robot. To avoid logistics issues associated with having large groups of novice users performing demonstrations on a physical robot, a web-based approach built around a simulation of the actual robot is proposed. The relationship between the robot and the person is described by the two dimensions of the interpersonal circumplex: communion (hostile or friendly) and agency (submissive or dominant). The users can adjust three parameters of the approach behavior (i.e., distance, trajectory curvature, and deceleration) in a manner that corresponds the best to the described relationship. An analysis of the data from 69 users is presented, along with a verification experiment done with 10 participants and the real robot. Results suggest that users associate hostile robots with straight trajectories, and submissive robots with smoother deceleration.

My PhD research aims to develop a general framework for a behavior control architecture that will provide customized interaction between the robot and an elderly individual suffering from mild cognitive impairment (MCI). This framework will enable the robot to learn from past events and to adapt its behavior to the specific needs of the person that it interacts with. For this purpose, models will be created for both verbal and non-verbal communication. The user profile (e.g., personality, cognitive disability level, emotional internal states, preferences) will provide the input based on which the robot will adapt its behavior.

Individuals suffering from Autistic Spectrum Disorder (ASD) have impaired skills in social communication and joint attention. In this paper, we explain how we designed and evaluated a Joint Attention (JA) task for individuals with ASD using the Nao humanoid robot. The interaction was tested in children and teenagers with ASD (N=11). Their proprioceptive and visual integration of cues were first assessed, with the hypothesis that individuals with an overreliance on proprioceptive cues and with a hyporeactivity to visual cues would have more difficulties conducting successful interactions with the robot. We observed that participants with an overreliance on proprioceptive cues and hyporeactivity to visual cues showed different behaviors in responding to joint attention. They followed the prompting of the Nao robot more slowly than individuals with an overreliance on visual cues and a hyporeactivity to proprioceptive cues. Defining such individual profiles prior to the social interaction with a robot and working closely with caregivers could provide promising strategies for designing successful and adapted Human-Robot Interaction (HRI) for individuals with ASD.

This study presents an experiment highlighting how participants combine facial expressions and haptic feedback to perceive emotions when interacting with an expressive humanoid robot. Participants were asked to interact with the humanoid robot through a handshake behavior while looking at its facial expressions. Experimental data were examined within the information integration theory framework. Results revealed that participants combined Facial and Haptic cues additively to evaluate the Valence, Arousal, and Dominance dimensions. The relative importance of each modality was different across the emotional dimensions. Participants gave more importance to facial expressions when evaluating Valence. They gave more importance to haptic feedback when evaluating Arousal and Dominance.

Socially assistive robots are envisioned to provide social and cognitive assistance where they will seek to motivate and engage people in therapeutic activities. Due to their physicality, robots serve as a powerful technology for motivating people. Prior work has shown that effective motivation requires adaption to user needs and characteristics, but how robots might successfully achieve such adaptation is still unknown. In this paper, we present work on matching a robot's personality-expressed via its gaze behavior-to that of its users. We confirmed in an online study with 22 participants that the robot's gaze behavior can successfully express either an extroverted or introverted personality. In a laboratory study with 40 participants, we demonstrate the positive effect of personality matching on a user's motivation to engage in a repetitive task. These results have important implications for the design of adaptive robot behaviors in assistive human-robot interaction.

This paper deals with the haptic affective social interaction during a greeting handshaking between a human and a humanoid robot. The goal of this work is to study how the haptic interaction conveys emotions, and more precisely, how it in'uences the perception of the dimensions of emotions expressed through the facial expressions of the robot. Moreover, we examine the bene'ts of the multimodality (i.e., visuo-haptic) over the monomodality (i.e., visual-only and haptic-only). The experimental results with Meka robot show that the multimodal condition presenting high values for grasping force and joint stiffness are evaluated with higher values for the arousal and dominance dimensions than during the visual condition. Furthermore, the results corresponding to the monomodal haptic condition showed that participants discriminate well the dominance and the arousal dimensions of the haptic behaviours presenting low and high values for grasping force and joint stiffness.

In human-human interaction, three modalities of communication (i.e., verbal, nonverbal, and paraverbal) are naturally coordinated so as to enhance the meaning of the conveyed message. In this paper, we try to create a similar coordination between these modalities of communication in order to make the robot behave as naturally as possible. The proposed system uses a group of videos in order to elicit specific target emotions in a human user, upon which interactive narratives will start (i.e., interactive discussions between the participant and the robot around each video's content). During each interaction experiment, the humanoid expressive ALICE robot engages and generates an adapted multimodal behavior to the emotional content of the projected video using speech, head-arm metaphoric gestures, and/or facial expressions. The interactive speech of the robot is synthesized using Mary-TTS (text to speech toolkit), which is used - in parallel - to generate adapted head-arm gestures [1]. This synthesized multimodal robot behavior is evaluated by the interacting human at the end of each emotion-eliciting experiment. The obtained results validate the positive effect of the generated robot behavior multimodality on interaction.

In this paper, we describe the elaboration and the validation of a body and face database1, of 96 videos of 1 to 2 seconds of duration, expressing 4 emotions (i.e., anger, happiness, fear, and sadness) elicited through 4 platforms of increased visual complexity and level of embodiment. The final aim of this database is to develop an individualized training program designed for individuals suffering of autism in order to help them recognize various emotions on different test platforms: two robots, a virtual agent, and a human. Before assessing the recognition capabilities of individuals with ASD, we validated our video database on typically developed individuals (TD). Moreover, we also looked at the relationship between the recognition rate and their personality traits (extroverted (EX) vs. introverted (IN)). We found that the personality of our TD participants did not lead to a different recognition behavior. However, introverted individuals better recognized emotions from less visually complex characters than extroverted individuals.

A common goal of robot control architecture designers is to create systems that are sufficiently generic to be adapted to different robot hardware. Beyond code re-use from a software engineering standpoint, having a common architecture could lead to long-term experiments spanning multiple robots and research groups. This paper presents a first step toward this goal with HBBA, a Hybrid Behavior-Based Architecture first developed on the IRL-1 humanoid robot and integrating an Adaptive Resonance Theory-based episodic memory (EM-ART). This paper presents the first step of the adaptation of this architecture to two different robots, a Meka M-1 and a NAO from Aldebaran, with a simple scenario involving learning and sharing objects' information between both robots. The experiment shows that episodes recorded as sequences of people and objects presented to one robot can be recalled in the future on either robot, enabling event anticipation and sharing of past experiences.

The eight papers in this special section focus on computer-based intelligent technologies for improving the quality of life.

Robotics is moving towards real world applications, beyond the well-structured environment of industrial robotics. In the world of assistant robots and medical robots, Human-Robot Interaction is essential. Also in emerging industrial scenarios there is a need of the human to be closely included in the loop. The companies are confronted with the lack of guidelines and of standards on how the higher features of HRI may be safely incorporated. Although the scientific research is burgeoning and worthy of praise, it is clear that its results are scattered and not capable of giving a clear input to be easily taken up by companies and standardization organizations like ISO and IEC. The workshop aims at the integration of empirical findings into complex real world robot systems by focusing on three typical sectors (industrial, service and medical) to develop systematic approaches to benchmark and evaluate experimental systems so that normative results can be realized rapidly. The present workshop focuses on bringing together scientists, representative of robotics companies and of standardization working groups to foster discussion in the definition of experimental scenarios and protocols in HRI, so to be able to prime real world set-ups and help realize the robotic products of the future.

In a typical Human-Machine Interaction (HMI) system, a task is performed by cooperation of the human and the automation component. The system adopts a cognitive architecture to model human psychology and makes optimum decisions on dynamic task allocation between human and the machine counterpart depending on the context. However, such architectures do not define how those systems are implemented in software. Various models involved in Model Driven Software Development (MDSD) approach in developing HMI systems is presented. This paper proposes a metamodel for modeling Non-Functional Properties (NFP) in HMI systems and provides a case study on assistive lane keeping in automobiles to demonstrate the approach.

In recent times, researchers in robotics have arrived at a consensus that there is no single efficient architecture or framework that satisfies all aspects of robotic system design. A majority of robotic architectures currently in use are developed in-house to meet the specific objectives of the research group. However, these frameworks have been designed in an ad-hoc manner and thus restricts reusability and scalability. In this paper, we propose an architecture description language `Architecture Modeling and Analysis Language (AMAL)' that homogenizes the framework development process in a formal way. AMAL is a radical approach that enables framework development based on custom requirements or by integrating existing heterogeneous architectural paradigms. The Open Semantics Framework in AMAL facilitates domain knowledge integration and promotes separation of concerns to build complex systems. We also position our approach in the SafeRobots framework, a model-driven software development tool-chain for robotics.

Recently, there is an encouraging trend in adopting model-driven engineering approaches for software development in robotics research. In this paper, currently available model-driven techniques in robotics are analyzed with respect to the domain-specific requirements. A conceptual overview of our software development approach called 'Self Adaptive Framework for Robotic Systems (SafeRobots)' is explained and we also try to position our approach within this model ecosystem.

A robotic system is a software intensive system that is composed of distributed, heterogeneous software components interacting in a highly dynamic, uncertain environment. However, no systematic software development process is followed in robotics research. In this paper, we present the core concepts that drive our framework `Self Adaptive Framework for Robotic Systems (SafeRobots)' for developing software for robotic systems. Two motivating examples are discussed: one discusses a system integration and knowledge representation problem, and the other explicates the issues associated with robotic system development in an industrial scenario. We also report on our progress on designing a meta-model based language - Solution Space Modeling Language, for problem-specific knowledge representation.

In HRI, many researches emphasize the impact of the human user's personality (expressed mainly through the Extroversion dimension) over the perception of the robot's behavior. In our experiment, where participants interacted/used a novel driving assistance system, we focused on analyzing the role of each BigFive Personality dimension in people's task performance and in their reaction towards the vocal assistance system. The results show that three of the BigFive Personality dimensions (i.e., Extroversion, Openness, and Agreeableness) present a certain influence towards participants' performance. We also found that inexperienced users made better progress in using the driving interface than the experienced users. A detailed discussion about the contribution of our current work and future perspectives is provided.

In multimodal human-robot interaction(HRI), the process of communication can be established through verbal, non-verbal, and/or para-verbal cues. The linguistic literature [3] shows that para-verbal and non-verbal communications are naturally synchronized. This research focuses on the relation between non-verbal and para-verbal communication by mapping prosody cues to the corresponding arm gestures. Our approach for synthesizing arm gestures uses the coupled hidden Markov models (CHMMs), which could be seen as a collection of HMMs modeling the segmented prosodic characteristics' stream and the segmented rotation characteristics' streams of the two arms' articulations [4][1]. Nao robot was used for tests.

In human-robot interaction, a social intelligent robot should be capable of understanding the emotional internal state of the interacting human so as to behave in a proper manner. The main problem towards this approach is that human internal states can't be totally trained on, so the robot should be able to learn and classify emotional states online. This research paper focuses on developing a novel online incremental learning of human emotional states using Takagi-Sugeno (TS) fuzzy model. When new data is present, a decisive criterion decides if the new elements constitute a new cluster or if they confirm one of the previously existing clusters. If the new data is attributed to an existing cluster, the evolving fuzzy rules of the TS model may be updated whether by adding a new rule or by modifying existing rules according to the descriptive potential of the new data elements with respect to the entire existing cluster centers. However, if a new cluster is formed, a corresponding new TS fuzzy model is created and then updated when new data elements get attributed to it. The subtractive clustering algorithm is used to calculate the cluster centers that present the rules of the TS models. Experimental results show the effectiveness of the proposed method.

In human-robot interaction, gender and internal state detection play an important role in making the robot reacting in an appropriate manner. This research focuses on the important features to extract from a voice signal in order to construct successful gender and internal state detection systems, and shows the benefits of combining both systems together on the total average recognition score. Moreover, it consists a foundation on an ongoing approach to estimate the human internal state online via unsupervised clustering algorithms.

In this paper, we present the first steps towards a new concept of robot self-awareness that can be implemented into embodied robot systems. Our concept of "the self" is inspired by already existing approaches and aims to provide a cognitive system with meta-cognitive capabilities. We believe that robot self-awareness is a crucial factor in the improvement of HRI.

This paper addresses gait analysis for a mobility assistance robot designed for the elderly people. Six patients and ten healthy peoples were invited to be part of our first pilot experiment. We designed two experiments so as to firstly detect gait parameters and secondly to identify a change of speed. For the first trial, we compared the temporal-distance parameters of the healthy people and of individuals suffering of mobility problems. The percentages of the gait cycle for duration of stance are higher for people with mobility impairment than for healthy people. In the second experiment, we detected the change in walking speed from the ten healthy peoples. Two different metrics derived from the Kullback-Leibler (KL) divergence and from the Generalized Likelihood Ratio (GLR) were employed for walking change detection. The Receiver Operating Characteristic (ROC) curves show a better performance for the signal obtained with the accelerometer sensor than that obtained with the infrared distance sensor. Nevertheless, the results of our experiments demonstrated that both methodologies (KL and GLR) can be used to detect the change points during walking at high or slow speed.

Human-robot interaction (HRI) takes place especially through interfaces. The design of such interfaces is a very delicate and crucial phase because it influences the robot accessibility and usability by the user. In this paper, we describe and analyze the results of 2 tests conducted so as to understand some of the optimal features that should characterize the robot voice and graphical-based user interfaces. Our test platform is an assistive robot developed for the elderly with mild cognitive impairments. Therefore, the user interfaces must be clear and simple. The ambiguities must be eliminated so as to facilitate the use of the robot and hence not to discourage the elderly population to use new technologies.

Currently the 2 percent growth rate for the world's older population exceeds the 1.2 percent rate for the world's population as a whole. This difference is expected to increase rather than diminish so that by 2050, the number of individuals over the age 85 is projected to be three times what it is today. Most of these individuals will need physical, emotional, and cognitive assistance. In this paper, we present a new system based on the socially assistive robotics (SAR) technology that will play the role of a music therapist and will try to provide a customized help protocol through motivation, encouragements, and companionship to users suffering from cognitive changes related to aging and/or Alzheimer's disease.

This research focuses on studying the possible role of a socially interactive robot as a tool for monitoring and encouraging cognitive activities of the elderly and/or individuals suffering from dementia. One of the aims of this work is to show the benefits of the robot's physical embodiment in human-robot social interactions. The social therapist robot tries to provide customized cognitive stimulation by playing a music game with the user. The results of the 8-month pilot study depict a more efficient, natural, and preferred interaction with the robot rather than with the simulated robot.

This paper presents a vision-based approach for mobile robot localization. The environmental model is topological. The new approach uses a constellation of different types of affine covariant regions to characterize a place. This type of representation permits a reliable and distinctive environment modeling. The performance of the proposed approach is evaluated using a database of panoramic images from different rooms. Additionally, we compare different combinations of complementary feature region detectors to find the one that achieves the best results. Our experimental results show promising results for this new localization method. Additionally, similarly to what happens with single detectors, different combinations exhibit different strengths and weaknesses depending on the situation, suggesting that a context-aware method to combine the different detectors would improve the localization results.

This paper describes a hands-off therapist robot that monitors, assists, encourages, and socially interacts with post-stroke users in the process of rehabilitation exercises. We developed a behavior adaptation system that takes advantage of the users introversion-extroversion personality trait and the number of exercises performed in order to adjust its social interaction parameters (e.g., interaction distances/proxemics, speed, and vocal content) toward a customized post-stroke rehabilitation therapy. The experimental results demonstrate the robot's autonomous behavior adaptation to the user's personality and the resulting user improvements of the exercise task performance.

In this work we address the problem of perception, spatial cognition and topological navigation for a mobile robot. The objective of this work is to enable the navigation of an autonomous mobile robot (or vehicle) in an indoor (or outdoor) structured environment without relying on maps a priori learned and without using artificial landmarks. A new method for incremental and automatic topological mapping and global localization using fingerprints of places is presented. The fingerprint-based representation permits a reliable, compact and distinctive environment-modeling. Experimental results for mapping indoor and outdoor environments with a mobile robot and a "SMART" vehicle, both equipped with a multi-sensor system composed of two 180deg laser range finders and an omnidirectional camera are also reported

Abstract − Even today, robot mapping is one of the biggest challenges in mobile robotics. Geometric or topological maps can be used by a robot to navigate in the environment. Automatic creation of such maps is still problematic if the robot tries to map large environments. This paper presents a new method for incremental mapping using fingerprints of places. This type of representation permits a reliable, compact, and distinctive environment-modeling and makes navigation and localization easier for the robot. Experimental results for incremental mapping using a mobile robot equipped with a multi-sensor system composed of two 180 ° laser range finders and an omni-directional camera are also reported.

This work presents a localization algorithm for indoor environments. The environmental model is topological and the approach describes how a multimodal perception increases the reliability for the topological localization problem for mobile robots, by using the Bayesian programming formalism. For the topological framework the fingerprint concept is used. This type of representation permits a reliable and distinctive environment modeling. Experimental results of a mobile robot equipped with a multi sensor system composed of two 180/spl deg/ laser range finders and an omni-directional camera are reported.

This paper proposes an approach allowing topology learning and recognition in indoor environments by using a probabilistic approach called Bayesian Programming. The main goal of this approach is to cope with the uncertainty, imprecision and incompleteness of handled information. The Bayesian Program for topology recognition and door detection is presented. The method has been successfully tested in indoor environments with the BIBA robot, a fully autonomous robot. The experiments address both the topology learning and topology recognition capabilities of the approach.

In this paper a perception approach allowing for high distinctiveness is presented. The method works in accordance to the fingerprint concept. Such representation allows using a very flexible matching approach based on the minimum energy algorithm. The whole extraction and matching approach is presented in details and viewed in a topological optic, where the matching result can directly be used as observation function for a topological localization approach. The experimentation section will validate the fingerprint approach and present different set of experiments in order to explain practically the choice of different types of features.

This paper presents both the theory and the first experimental results of a new method which allows simultaneous estimation of the robot configuration and the odometry error (both systematic and non-systematic) during the mobile robot navigation. The estimation of the non-systematic components is carried out through an augmented Kalman filter which estimates a state containing the robot configuration and the parameters of the odometry error. It uses encoder readings as inputs and the readings from a laser range finder as observations. The estimation of the non-systematic components is carried out through another Kalman filter where the observations are obtained by two subsequent robot configurations provided by the previous augmented Kalman filter.