Humans and robots are increasingly working in personal and professional settings. In workplace settings, humans and robots may work together as colleagues, potentially leading to social expectations—or violation thereof. Extant research has primarily sought to understand social interactions and expectations in personal rather than professional settings, and none of these studies have examined negative outcomes arising from violations of social expectations (i.e., nonaligned, or unexpected, behaviors). This article reports the results of a 2 × 3 online experiment (human behavior: polite/rude; robot behavior: agreeable/neutral/“retaliatory”) that used a unique “first-person perspective video” to immerse participants in a workplace setting to examine perceptions of appropriate robot responses to human norm violation. The results are nuanced and reveal that while robots are expected to act in accordance with social expectations despite human behavior, there are benefits for robots perceived as “being the bigger person” in the face of human rudeness. Theoretical and practical implications are provided, which discuss the importance of these findings for the design of social robots.

When humans work together to complete a joint task, each person builds an internal model of the situation and how it will evolve. Efficient collaboration depends on how these individual models overlap to form a shared mental model among team members; shared models are also important for collaborative processes in human–robot teams. The development and maintenance of an accurate shared mental model requires bidirectional communication of individual intent and the ability to interpret the intent of other team members. To enable effective human–robot collaboration, this article investigates the use of augmented reality (AR) technology and user eye gaze to enable bidirectional communication of intent in a joint action task. We tested this approach through a user study with 37 participants and found that this communication improves task efficiency, trust, as well as task fluency. We conclude that using AR and eye gaze to enable bidirectional communication and support shared mental models is a promising means for improving collaboration between humans and robots.

Human operators in human-robot teams are commonly perceived to be critical for mission success. To explore the direct and perceived impact of operator input on task success and team performance, 16 real-world missions (10 h) were conducted based on the DARPA Subterranean Challenge. Missions involved deploying a heterogeneous team of robots to locate and identify artefacts such as climbing rope, drills and a mannequin representing a human survivor. Two conditions were evaluated: human operators that could control the robot team with state-of-the-art autonomy (Human-Robot Team) compared to autonomous missions without human operator input (Robot-Autonomy). Human interventions included creating waypoints to prioritise high-yield areas, and to navigate through error-prone spaces. Human-Robot Teams were often in directed autonomy mode (70% of mission time), found more items ( \(+\) 10.52%), traversed more distance ( \(+\) 12.71%), covered more unique ground ( \(+\) 10.56%), and longer time between safety-related events (34%). In routine conditions, both condition scores were comparable for artefacts, distance and coverage. Human-Robot Teams were faster at finding the first artefact but slower to respond to information from the robot team. Overall, operators contribute to mission-based outcomes, help to overcome environmental situations that can impede progress, and can assist robots to recover faster from difficult events.

Robots are inherently imperfect, and collaborating with an error-prone robotic teammate can deteriorate perceptions of trust and the willingness of users to continue working with the robot. Evidence-based trust repair strategies can be implemented into a robot’s design to mitigate the decline of trust in human-robot relationships following errors. It is not yet clear what trust repair strategies are most effective. To address this shortcoming, this study investigates two novel trust repair strategies: offered and automatic physical repair. A between-subjects repeated measures study was performed to determine the extent to which each type of physical trust repair was successful in restoring participants’ perceptions of trust. The results indicated that, where the no-repair condition experienced a significant decrease in trust score, only the automatic repair was consistently successful in bypassing the trust decline. Detailed analysis showed that participants from the offered repair condition did not view the robot as providing the appropriate information, meaning that the offer itself may have confused them. Participants’ response rate to the MultiDimensional Measure of Trust also revealed that users were less willing to associate moral terms with robotic teammates, though this hesitancy may reduce over time. These results contribute to research on human-robot trust repair by uncovering that physical repair is effective when it is automatic, but not when it is offered. This finding will help to further elucidate what repair strategies work to mitigate trust decline and thus help inform robot design.

Robotic vision, otherwise known as computer vision for robots, is a critical process for robots to collect and interpret detailed information related to human actions, goals, and preferences, enabling robots to provide more useful services to people. This survey and systematic review presents a comprehensive analysis on robotic vision in human-robot interaction and collaboration (HRI/C) over the past 10 years. From a detailed search of 3,850 articles, systematic extraction and evaluation was used to identify and explore 310 papers in depth. These papers described robots with some level of autonomy using robotic vision for locomotion, manipulation, and/or visual communication to collaborate or interact with people. This article provides an in-depth analysis of current trends, common domains, methods and procedures, technical processes, datasets and models, experimental testing, sample populations, performance metrics, and future challenges. Robotic vision was often used in action and gesture recognition, robot movement in human spaces, object handover and collaborative actions, social communication, and learning from demonstration. Few high-impact and novel techniques from the computer vision field had been translated into HRI/C. Overall, notable advancements have been made on how to develop and deploy robots to assist people.

Curriculum learning allows complex tasks to be mastered via incremental progression over 'stepping stone' goals towards a final desired behaviour. Typical implementations learn locomotion policies for challenging environments through gradual complexification of a terrain mesh generated through a parameterised noise function. To date, researchers have predominantly generated terrains from a limited range of noise functions, and the effect of the generator on the learning process is underrepresented in the literature. We compare popular noise-based terrain generators to two indirect encodings, CPPN and GAN. To allow direct comparison between both direct and indirect representations, we assess the impact of a range of representation-agnostic MAP-Elites feature descriptors that compute metrics directly from the generated terrain meshes. Next, performance and coverage are assessed when training a humanoid robot in a physics simulator using the PPO algorithm. Results describe key differences between the generators that inform their use in curriculum learning, and present a range of useful feature descriptors for uptake by the community.

The use of hand-based gestures has been proposed as an intuitive way for people to communicate with robots. Typically the set of gestures is defined by the experimenter. However, existing works do not necessarily focus on gestures that are communicative, and it is unclear whether the selected gesture are actually intuitive to users. This paper investigates whether different people inherently use similar gestures to convey the same commands to robots, and how teaching of gestures when collecting demonstrations for training recognizers can improve resulting accuracy. We conducted this work in two stages. In Stage 1, we conducted an online user study (n=190) to investigate if people use similar gestures to communicate the same set of given commands to a robot when no guidance or training was given. Results revealed large variations in the gestures used among individuals With the absences of training. Training a gesture recognizer using this dataset resulted in an accuracy of around 20%. In response to this, Stage 2 involved proposing a common set of gestures for the commands. We taught these gestures through demonstrations and collected ~ 7500 videos of gestures from study participants to train another gesture recognition model. Initial results showed improved accuracy but a number of gestures had high confusion rates. Refining our gesture set and recognition model by removing those gestures, We achieved an final accuracy of 84.1 ± 2.4%. We integrated the gesture recognition model into the ROS framework and demonstrated a use case, where a person commands a robot to perform a pick and place task using the gesture set.

Embodied conversational agents (ECAs) are often designed to produce nonverbal behavior to complement or enhance their verbal communication. One such form of the nonverbal behavior is co-speech gesturing, which involves movements that the agent makes with its arms and hands that are paired with verbal communication. Co-speech gestures for ECAs can be created using different generation methods, divided into rule-based and data-driven processes, with the latter, gaining traction because of the increasing interest from the applied machine learning community. However, reports on gesture generation methods use a variety of evaluation measures, which hinders comparison. To address this, we present a systematic review on co-speech gesture generation methods for iconic, metaphoric, deictic, and beat gestures, including reported evaluation methods. We review 22 studies that have an ECA with a human-like upper body that uses co-speech gesturing in social human-agent interaction. This includes studies that use human participants to evaluate performance. We found most studies use a within-subject design and rely on a form of subjective evaluation, but without a systematic approach. We argue that the field requires more rigorous and uniform tools for co-speech gesture evaluation, and formulate recommendations for empirical evaluation, including standardized phrases and example scenarios to help systematically test generative models across studies. Furthermore, we also propose a checklist that can be used to report relevant information for the evaluation of generative models, as well as to evaluate co-speech gesture use.

Social robots have been used to promote health education and coaching to provide health information. Important behaviors to address and monitor include actions that can be modified, such as physical activity. These behaviours often require different personalised recommendations. Robots could be an effective way to give personalised health feedback based on scores, including in acute medical settings. This trial involved an automated social robot interaction in a health clinic to collect health data and provide personalized feedback on four key factors: exercise, diet, alcohol and cigarette use. Patients completed an 20-minute health questionnaire with a Pepper Robot in a clinic room during a health visit. The interaction was programmed to run autonomously with automatic scoring and feedback based on health scores. Instructions were delivered using co-verbal speech and detailed text on the tablet. Questions also included ratings on comfort to discuss health topics with a human or robot. Patients could choose to receive an optional follow-up in four weeks’ time. A total of 47 patients completed the session. Patients reported being as comfortable to discuss health-related topics with a robot or human for exercise, diet, alcohol, cigarette use, and mental health. Program evaluation received moderate ratings for the robot on ease of use, usefulness and motivation to change a health behavior. No significant health changes were found 30 days later due to high initial health scores, leaving little room for improvement. This initial proof-of-concept trial found that a robot-delivered service could be deployed in a live health clinic in conjunction with patient visits.

Social robots have been used to help people to make healthy changes, and one setting that could benefit from having more support services offered includes the higher education sector. This trial involved an initial test to explore how a social robot could help to deliver a low-intensity problem-solving session for students around study-related issues and challenges. A Pepper Humanoid Robot was deployed in a student centre to help students to build a problem-solving plan on a specific issue. In the trial, 72 students gave detailed responses to session questions for issues such as procrastination, life/study balance and study workload. Students reported good ratings for emotional reaction to the robot, perceived utility, intention to use the robot again, confidence to use the robot, perceived helpfulness from the robot, likelihood to use the robot for a new higher education issue, and to recommend the robot to a friend. Robot evaluation scores were correlated with scores on perceived helpfulness of the robot and confidence to try an idea in the next week. Students who reported positive robot evaluation scores were also more willing to use the session content and rate the content as helpful. One week later, most students reported that the robot session helped them to fix their chosen issue, and that they used at least one idea from the session. Overall, this study found that a session run by a social robot could provide support for a study-related issue or challenge, and that some students did receive benefit from the session content. Future studies could include enhancements and adaptations to session length, technical refinement and capacity to address new issues during the session.

This paper reports the design and qualitative evaluation of a social robot programmed to deliver a talk-based treatment program to improve health behaviour change for food intake and weight loss. A qualitative study was conducted to investigate factors that influenced human-robot interaction and its relationship to health treatment outcomes. Semi-structured interviews were undertaken on completion of a randomised controlled trial that used an autonomous robot to deliver a 4-week behavioral intervention to help coach people to decrease the consumption of high calorie foods. Questions focused on individuals’ preferences, learnings and outcomes from their participation in the trial. Twenty participants completed the treatment, and 18 conducted an interview. Content analysis found that a social robot to deliver a psychotherapeutic treatment was effective and feasible. Participants did make changes to their health behaviour change with a >50% reduction in high calorie intake and average reduction of 4.4 kilograms in weight loss. The robot received positive evaluations on its interactive nature and sociable persona. Most participants made improvements that were aligned with their chosen health goal after completing the robot-delivered sessions, and reported that the robot sessions helped them to achieve their behaviour change goals, such as consuming fewer high calorie foods. Detailed recommendations are provided for the future design of healthcare interventions by robots, including key considerations for robot behaviour, treatment content, and presentation of the program. Future recommendations are presented for the development of robot personalization to more closely resemble techniques and skills from client-centred counselling.

An individual's self-efficacy to interact with a robot has important implications around the content, utility and success of the interaction. Individuals need to achieve a high level of self-efficacy in human robot-interaction in a reasonable time-frame for positive effects to occur in shortterm human-robot scenarios. This trial explored the impact of a 2-minute automated robot-delivered tutorial designed to teach people from the general public how to use the robot as a method to increase robot self-efficacy scores. This trial assessed scores before (T1) and after (T2) an interaction with the robot to investigate changes in self-efficacy, likability and willingness to use it. The 40 participants recruited had on average very low level of robotic experience. After the tutorial, people reported significantly higher robot self-efficacy with very large effect sizes to operate a robot and apply the robot to a task (η2p= 0.727 and 0.660). Significant increases in likability and willingness to interact with the robot were also found (η2p = 0.465 and 0.480). Changes in likability and self-efficacy contributed to 64% of the variance in changes to willingness to use the robot. Initial differences were found in robot self-efficacy for older people and those with less robotics and programming experience compared with other participants, but scores across these subgroups were similar after completion of the tutorial. This demonstrated that high levels of self-efficacy, likeability and willingness to use a social robot can be reached in a very short time, and on comparable levels, regardless of age or prior robotics experience. This outcome has significant implications for future trials using social robots, since these variables can strongly influence experimental outcomes.