Developments in the assistive capabilities of robots have allowed them to be more prevalent across domains from manufacturing to healthcare to education. To achieve more efficient and fluent collaborations between robots and humans, it has become necessary to explore how humans trust robot partners during collaborations. Current measurements of human trust in a robot partner typically rely on reflective, subjective survey measures, which are difficult to incorporate and update in the robot’s decision-making models for real-time deployment. To address this challenge, we aim to identify potential objective measures of trust, which can be incorporated into the robot’s decision-making process. We examine physiological data (Electrodermal Activity (EDA), Skin Temperature (TEMP), and Blood Volume Pulse (BVP)) and facial expression in an in-person ( \(n=30\) ) human–robot collaboration study and report the effect of trust on these objective measures. The results indicate that the participants who distrusted the robot partner experienced higher EDA, higher TEMP, and lower BVP variations than participants who trusted the robot. For facial expression, participants who distrusted the robot had more intense facial expressions than participants who trusted the robot. Ultimately, these findings can be used to further research on developing a real-time model of human trust in robots.

With robots becoming increasingly prevalent in various domains, it has become crucial to equip them with tools to achieve greater fluency in interactions with humans. One of the promising areas for further exploration lies in human trust. A real-time, objective model of human trust could be used to maximize productivity, preserve safety, and mitigate failure. In this work, we attempt to use physiological measures, gaze, and facial expressions to model human trust in a robot partner. We are the first to design an inperson, human-robot supervisory interaction study to create a dedicated trust dataset. Using this dataset, we train machine learning algorithms to identify the objective measures that are most indicative of trust in a robot partner, advancing trust prediction in human-robot interactions. Our findings indicate that a combination of sensor modalities (blood volume pulse, electrodermal activity, skin temperature, and gaze) can enhance the accuracy of detecting human trust in a robot partner. Furthermore, the Extra Trees, Random Forest, and Decision Trees classifiers exhibit consistently better performance in measuring the person's trust in the robot partner. These results lay the groundwork for constructing a real-time trust model for human-robot interaction, which could foster more efficient interactions between humans and robots.

Recent advancements in robot capabilities have enabled them to interact with people in various human-social environments (HSEs). In many of these environments, the perception of the robot often depends on its capabilities, e.g., task competency, language fluency, etc. To enable fluent human-robot interaction (HRI) in HSEs, it is crucial to understand the impact of these capabilities on the perception of the robot. Although many works have investigated the effects of various robot capabilities on the robot’s perception separately, in this paper, we present a large-scale HRI study (n = 60) to investigate the combined impact of both language fluency and task competency on the perception of a robot. The results suggest that while language fluency may play a more significant role than task competency in the perception of the verbal competency of a robot, both language fluency and task competency contribute to the perception of the intelligence and reliability of the robot. The results also indicate that task competency may play a more significant role than language fluency in the perception of meeting expectations and being a good teammate. The findings of this study highlight the relationship between language fluency and task competency in the context of social HRI and will enable the development of more intelligent robots in the future.

Social robots are being deployed to interact with people in various scenarios, where they are expected to in-corporate human-like conversational strategies to achieve flu-ency in interactions. For example, current robots are designed to perform advanced communication strategies (i.e., personal anecdotes, explanations, and apologies) to recover from task failure. However, these tactics are not always sufficient for failure recovery as they can be lengthy and insufficient for encouraging future interactions. In human-human interactions, people often use humor as a low-risk and engaging method for managing failures. Thus, the successful execution of advanced, human-like humor could enable robots to recover from task failures more efficiently. In this paper, we present a human-robot interaction study exploring how a robot's utilization of various human-like humor types (i.e., affiliative, aggressive, self-enhancing, and self-defeating) are perceived by human teammate (n = 32) and an external observer of the interaction (n = 256). Additionally, we have explored the effects of performance, humor type, perspective, and previous experience with robots on the participants' perceptions of warmth, competence, and the robot as a teammate. Our results indicate that dyadic participants rated the successful robot to be more competent and a better teammate than the bystander participants. Additionally, the results indicate that participants with less experience with robots found the successful robot to be more competent than participants with high levels of experience. These findings will enable the human-robot interaction community to develop more engaging robots for fluent interactive experiences in the future.

Humans naturally use verbal utterances and nonverbal gestures to refer to various objects (known as $\textit{referring expressions}$) in different interactional scenarios. As collecting real human interaction datasets are costly and laborious, synthetic datasets are often used to train models to unambiguously detect relationships among objects. However, existing synthetic data generation tools that provide referring expressions generally neglect nonverbal gestures. Additionally, while a few small-scale datasets contain multimodal cues (verbal and nonverbal), these datasets only capture the nonverbal gestures from an exo-centric perspective (observer). As models can use complementary information from multimodal cues to recognize referring expressions, generating multimodal data from multiple views can help to develop robust models. To address these critical issues, in this paper, we present a novel embodied simulator, CAESAR, to generate multimodal referring expressions containing both verbal utterances and nonverbal cues captured from multiple views. Using our simulator, we have generated two large-scale embodied referring expression datasets, which we have released publicly. We have conducted experimental analyses on embodied spatial relation grounding using various state-of-the-art baseline models. Our experimental results suggest that visual perspective affects the models' performance; and that nonverbal cues improve spatial relation grounding accuracy. Finally, we will release the simulator publicly to allow researchers to generate new embodied interaction datasets.