Humor is a key element in human interactions, essential for building connections and rapport. To enhance human-robot communication, we developed a humor-aware chat framework that enables robots to deliver contextually appropriate humor. This framework takes into account the interaction environment, and user's profile as well as emotional state. Two GPT models are used to generate responses. The initial one, named sensor-GPT, processes contextual data from the sensor along with the user's response and conversation history to create prompts for the second one, chat-GPT. These prompts can guide the model on how to integrate appropriate humor elements into the conversation, ensuring that the dialogue is both contextually relevant and humorous. Our experiment compared the effectiveness of humor expression between our framework and the GPT-40 model. The results demonstrate that robots using our framework significantly outperform those using GPT-4o in humor expression, extending conversations, and improving overall interaction quality.

This paper investigates how different levels of information impact user trust and cognitive load in teleoperated robotic systems. Participants performed tasks under three conditions: (C1) minimal information after initial visual feedback, (C2) verbal guidance through a graphical interface, and (C3) a combination of visual and verbal guidance via a graphical user interface (GUI) that shows the direction in which the user should move the robot to complete the task. Measurements included physiological responses such as galvanic skin response (GSR), eye blink rate, and facial temperature, along with task performance. The findings revealed that increased cognitive load reduced user trust and performance. When only minimal information was provided, participants experienced the highest cognitive load and lowest trust levels. Verbal guidance significantly reduced cognitive load and increased trust, whereas the combination of visual and verbal guidance caused cognitive overload, counteracting the expected increase in trust. This study underscores the importance of balancing information quantity and quality to enhance user experience and the efficiency of teleoperated robotic systems.

Trust plays a crucial role in user performance during collaborative human-robot interaction. This study examines how varying levels of autonomy and system errors affect user trust and cognitive load in collaborative tasks between robots and humans. Participants performed a collaborative task using a UR5 robotic arm to place four bottles of different shapes into a box within a three-minute time frame under three conditions: (C1) full manual control by the user, (C2) autonomous operation with few errors—where the robot fails to correctly place one out of four bottles and the user can intervene upon detecting failures, and (C3) autonomous operation with frequent errors—where the robot fails to correctly place three out of four bottles, with user intervention allowed upon failure detection. Physiological indicators such as blink rate, galvanic skin response (GSR), and facial temperature, along with task performance metrics such as success rate and completion time were tracked. The results showed that participants experienced the highest cognitive load in Condition 1, as indicated by higher NASA-TLX scores, increased blink rates (average of 65 blinks per minute), elevated facial temperatures, and higher GSR readings. Trust levels were lowest in Condition 3, with 74% of participants reporting low trust, highlighting the significant impact of robot reliability on user’s trust. A strong negative correlation was found between cognitive load and trust in Condition 3 suggesting that increased cognitive load due to frequent robot errors leads to decreased trust. These findings contribute to understanding how system errors and autonomy levels influence cognitive load and trust in collaborative human-robot tasks. The insights gained can inform the design of collaborative robotic systems that balance autonomy and reliability, enhancing user experience and performance.

Trust plays a crucial role in user performance during teleoperated human-robot interaction. This study presents a reinforcement learning (RL) model that adapts to dynamic trust levels using physiological data and task performance metrics. Participants completed a complex teleoperation task under three conditions: (C1) limited feedback, (C2) AI-generated verbal guidance, and (C3) AI guidance paired with real-time RViz visualization. Physiological indicators, such as blink rate, galvanic skin response (GSR), and facial temperature along with task performance metrics like success rate and completion time were tracked. Statistical analyses revealed that increased task complexity in C1 reduced trust and increased cognitive load, leading to poorer performance. AI-generated guidance in C2 improved task understanding and performance, supporting Hypothesis H2. In C3, combining AI guidance with RViz visualization further boosted trust and reduced cognitive load, partially confirming Hypothesis H3. The RL model successfully adapted guidance strategies based on real-time user states, and additional testing showed that the agent’s adaptive strategies significantly increased user trust and improved performance. These results underscore the potential of adaptive RL models to enhance trust and efficiency in teleoperated human-robot systems.

Teleoperated robots, especially in hazardous environments, integrate human cognition with machine efficiency, but can increase cognitive load, causing stress and reducing task performance and safety. This study examines the impact of the information available to the operator on cognitive load, physiological responses (e.g., GSR, blinking, facial temperature), and performance during teleoperation in three conditions: C1 - in presence, C2 - remote with Visual feedback, and C3 - remote with telepresence robot. The findings from our user study involving 20 participants show that information availability significantly impacts perceived cognitive load, as evidenced by the differences observed between conditions in our analysis. Furthermore, the results indicated that blinking rates varied significantly among the conditions. The results also underline that individuals with higher error scores on the spatial orientation test (SOT), reflecting lower spatial ability, are more likely to experience failure in conditions 2 and 3. The results show that information availability significantly affects cognitive load and teleoperation performance, especially depth perception of the robot’s actions. Additionally, the thermal and GSR data findings indicate an increase in stress and anxiety levels when operators perform conditions 2 and 3, thus corroborating an increase in the user’s cognitive load.

Expressing humor in social interactions presents a significant challenge for humans due to its intricate linguistic nature. This complexity is further magnified when teaching robots to express humor appropriately. Among the various expressions of humor, jokes are one of the most commonly used. Therefore, a well-annotated joke dataset holds significant promise in enhancing a robot’s ability to express humor effectively. This paper introduces a dataset comprising over two thousand jokes, with the aim of providing rich material and multidimensional selection criteria for the humor expression of the robot. The creation of this joke dataset involved a collaborative effort among robot experts studying HRI, psychologists with rich humor research experience, and GPT-3. The annotation process primarily concentrated on four dimensions within the dataset: the humor style of jokes, semantic words (aligned with semantic gestures), keywords, and ratings of joke funniness. We additionally outline several prospective applications of this dataset. We introduce a BERT-based neural network model trained on the dataset with semantic word labels. This model aims to empower robots to choose suitable semantic words from jokes and articulate them alongside corresponding semantic gestures. Moreover, we offer suggestions for utilizing jokes from this dataset to facilitate the adaptive expression of humor by social robots. These endeavors will further enhance the multi-modal humor expression capability of social robots.