In the Industry 5.0 era, the focus shifts from basic automation to fostering collaboration between humans and robots. Trust is crucial in this new paradigm, enabling smooth interaction, especially for users with limited robotics knowledge. This study presents a novel framework that uses human hand gestures and voice commands to control robot movements, aiming to enhance trust, reduce cognitive workload, and minimize task execution time-key for efficient manufacturing. In automated systems, swift completion of micromanagement tasks is essential to prevent process disruption. To evaluate this framework, we devised a testbed scenario within an automated carbon fiber transportation and draping process, focusing on a maintenance task as the micromanagement challenge. Participants inspected the gripper, guided the robot along a defined path, and performed maintenance, such as attaching cables. Two conditions were tested: gestures and voice commands versus a smartPAD. The results showed that gestures and voice commands increased trust, lowered cognitive load, and shortened execution times, improving overall manufacturing efficiency.

Forming friendship with peers from diverse backgrounds is key to children’s social emotional development. In this study, we explored the use of social robot, Haru, as mediator for remote communication in children group, to support connection and friendship building. We invited children from different countries aged from 10 to 15 to participate in two interaction sessions with peers from other countries, after which we conducted interviews with children from three countries, focusing on their experiences, and perceptions of the robot’s roles in the process. The findings indicated that social robot Haru effectively served as an icebreaker and entertainer; However, improvements are needed in conversation flow, transitions between different roles, and supporting children’s autonomy in guiding the conversation and the depth of their communication.

The integration of robots into daily life has increased significantly, spanning applications from social-care to industrial settings with collaborative robots. Ensuring a safe, secure environment and equitable workload distribution in human-robot collaborations is crucial. Trust is a key factor in these environments, essential for enhancing collaboration and achieving tasks while maintaining safety. Under-trusting robots can hinder productivity, while over-trusting them can lead to accidents. A comprehensive literature review of 100 publications from 2003 to 2023 analyzes trust and its influencing factors in industrial and social-care contexts. Findings reveal that in industrial settings, robot-related factors are more influential, whereas in social-care, human and environmental factors play a significant role. Furthermore, the review delves into gauging trust through observable behavior, while also exploring various trust evaluation methodologies. Results show that trust can be gauged through human behaviors, physical cues, and physiological measurements. Concerning trust evaluation methodologies, traditional questionnaires have limitations, opening new opportunities for machine learning and sensor-based approaches to real-time trust evaluation, as trust is a dynamic cognitive value that evolves over time. Notably, 97% of the reviewed articles were published in the last decade, underscoring a growing interest in human–robot interaction and trust within the scientific community.

Trust is a factor that is becoming more prominent in human robot interaction research. Only few approaches so far tackle the challenge of data-driven trust assessment. In this paper, we present a data set consisting of motion tracking data from an industrial human robot collaboration task. The data is collected during a trust manipulation experiment that has been designed to elicit different trust levels in the participants. Additionally, participants filled out a standard trust questionnaire. The data set allows for developing and testing data-driven trust assessment algorithms.

Our goal is to design a framework for holistic evaluation of human-robot collaboration systems. To this end we utilize several standardized questionnaires administered while participants perform collaborative tasks in robot work cells. We used Standard Usability Scale and the Usability metric for user experience questionnaires to access usability, NASA Task-Load for workload, two questionnaires for human-robot trust as well as the Unified theory of acceptance and use of technology questionnaires. We performed two pilot tests of our framework with human-robot collabotation work cells at two test sites as part of the DrapeBot project. The goal of the project is to enable human-robot collaboration in the process of carbon fiber draping the production of outer parts. After utilizing the evaluation framework at the two test sites we found that the collection of questionnaires were easy to adapt to each work cell and the practical limitation around running the experiments. Both work cells scored high in usability, expected increase of productivity, as well as high trust and low anxiety, but both work cells scored low on expectancy of use for work in the future at their current state of development.

In this paper, we investigate how a customer-facing service robot can support decision making in shopping interactions. In this role, a robot needs sometimes to reject a customer’s choice. Thus, we investigate different rejection strategies with the goal of changing customer behavior. The implemented strategies have been developed based on an ethnographic study on assisted shopping and tested in a lab experiment with 31 participants. The experiment showed significant differences in trust ratings and decision-making depending on the employed strategy.

Trust in automation is usually assessed with post-interaction questionnaires. For human robot collaboration it would be beneficial to assess the trust level during the interaction to adjust the robot’s collaboration behavior to the user expectations. In this paper we investigate if trust can be estimated from observable behavior like movements during the interaction with a large industrial manipulator. To this end, we report on a data collection for two tasks during collaborative draping, the transport of large cut pieces and the actual draping process in close proximity to the robot. The data is used to train and compare different deep learning models. Results show that automatic trust estimation is feasible, which opens up to using trust as a parameter for informing the interaction with robots.

Industry 5.0 signifies a transformative era where humans and robots collaborate closely, leading to advancements in manufacturing efficiency and personalization. In light of this, it becomes essential to assess the robot’s trustworthiness to ensure a secure environment and equitable workload distribution. The majority of trust assessments hinge on post-hoc questionnaires for the extent of trust experienced during the interaction. A data-driven approach is required to promptly assess trust levels in real-time, allowing for the adjustment of robot behavior to align with human needs. The paper proposes a chemical industry scenario where a robot assisted a human in the process of mixing chemicals. Several machine learning models, including deep learning, were developed using body motion data to categorize the level of trust exhibited by the human operator. The models achieve an accuracy exceeding 90%. The results clearly show the feasibility of data-driven trust assessment.

In the emerging phase of industrialization, Industry 5.0, humans will be working alongside advanced technologies such as Artificial Intelligence (AI) and robots to improve the manufacturing process. As a result, it is crucial to evaluate trust in the robot from a human perspective in order to provide a safe environment and balance workloads. Relevant trust indicators in the industrial context include proximity between human and robot, as well as risk associated with robot’s performance. In this study, a chemical industry scenario was developed, where a robot assists a human in mixing chemicals. An experiment was conducted for analysing how proximity and risk impact the trust level of the participants. According to the results, there was a higher average trust score in the low proximity (i.e. robot not close to the human) and low risk sections compared to the high proximity and high risk sections of the experiment, respectively. Moreover, statistical analysis indicates that risk had a higher impact on trust than proximity. The findings of this study encourage further research in this area since tools such as AI could be used to control the robot’s behavior according to the level of trust between the human and the robot.

Close-proximity human-robot collaboration (HRC) requires an appropriate level of trust from the operator to the robot to maintain safety and efficiency. Maintaining an appropriate trust level during robot-aided production requires non-obstructive real-time human-robot trust assessment. To this end we performed an experiment with 20 participants performing two types of HRC tasks in close proximity to a Kuka KR 300 R2500 ultra robot. The two tasks involved collaborative transport of textiles and collaborative draping, respectively. During the experiment we performed full body motion tracking and administered human-robot trust questionnaires in order investigate the correlation between trust and operator movement patterns. From the initial per-session analyses we see the effects of task types on movement patterns, but the correlations with trust are weak overall. Further analysis at higher temporal resolution and with correction for participants’ base movement patterns are required.

Nowadays the sensor-based human activity recognition is a key issue in the field of the physical exercises’ recognition. The purpose of this paper is to recognise exercise sports, such as squats, pull-ups and dips, using a dataset based on three ultra-wideband (UWB) sensors with additional inertial data and a deep learning approach in order to achieve an appropriate rate of the recognition of sports exercises with reduced computational effort. Firstly, a dataset, containing 273 samples of dips, 215 samples of pull-ups and 956 samples of squats, plus an additional 2024 samples of the input signals acquired during breaks, was created. Next, an optimal set of hyperparameters of Convolutional Neural Network (CNN) architecture in order to balance the accuracy rate and computational effort was achieved. Finally, it was discovered that acceleration signals have the highest energy and therefore, a comparison of the accuracy rate between CNN, using all signals and CNN, using acceleration only, was carried out (97.5% vs. 97.7%). Moreover, much less computational effort, expressed in number of multiplications, (1.8e4 vs. 1.2e5) was achieved. The practical usefulness is presented, by facilitating the implementation of the presented UWB sensor-based system for recognising such physical exercises as squats, pull-ups and dips on smartphones and commonly available wearable sensor devices.

The workshop investigates two major boundaries within HRI design and research: Firstly, we aim to cross the boundaries of engaging in interdisciplinary collaboration of such divergent disciplines as engineering, design, psychology, philosophy and sociology. Secondly, we aim to cross the boundaries of HRI design and social contexts of use - often referred to as ‘real world’ environments. This endeavor is not new, however we aim for approaching these two boarders of HRI research and design more systematically, e.g. by providing new methodological impulses. The idea of “configuring” has a long tradition in Science and Technology Studies (STS) to describe how potential users and use cases are shaped and in turn reshaped (configured) throughout technology design - be it explicitly or accidentally. Given HRI is becoming deeper integrated in ‘real world’ contexts, such as public spaces, homes and care facilities, we argue for the need for a re-configuration. This includes a critical reflection of material, procedural and methodological implications that shape future users within HRI design practices - for and together with people.

In order to test how explanations affect a user working together with a collaborative robot, we created a test scenario where a user sorts trash together with a collaborative robot. Sometimes the robot is not able to fulfill its part of the task. Different modalities (textual, graphical, both) for explaining this error to the user are tested in a between subjects design and the effects on task performance, cognitive load and trust are analyzed.

There is a growing interest in the retail industry to deploy service robots for customer interactions. Deploying such customer-facing robots raises the question of how we want to interact with these robots and reveals concerns that businesses and marketers could use robots to manipulate consumers. In this experiment, 67 study participants interacted with different virtual shopping robots that tried to impact "shoppers" purchasing decisions. The results indicate that a robot can increase consumer spending. The study exemplifies how a collaborative robot could be used as a customer-serving robot in a retail environment and investigates the impact of (i) different interaction strategies (human vs robot control) and (ii) dark patterns on shopping behavior (manipulative vs supportive robot).

This study explores how to best utilize service robots when deploying them in customer facing contexts for information requests. Many companies investing in such robots unfortunately fail to integrate them successfully in a way where they facilitate meaningful interactions with users. Copenhagen Visitor Service is one of those companies who have tried to integrate a Pepper robot into their service environment but failed due to several factors relating to both the environment, the robot and lack of a strategical implementation. Through an exploratory field study with in-situ feedback from the participants, it became evident that the robot in this specific setting needed a more focused purpose and an interaction flow that was not reliant on verbal communication. A prototype was developed consisting of an informative quiz targeting families. A subsequent field deployment demonstrated a better and more meaningful user experience, indicating that careful consideration of the use context is paramount for a successful integration as well as a strategical implementation.

Going beyond dyadic (one-to-one) interaction has been increasingly explored in HRI. Yet we lack a comprehensive view on non-dyadic interaction research in HRI. To map out 15 years of works investigating non-dyadic interaction, and thereby identifying the trend of the field and future research areas, we performed a literature review containing all 164 publications (2006-2020) from the HRI conference investigating non-dyadic interaction. Our approach is inspired by the 4C framework, an interaction framework focusing on understanding and categorising different types of interaction between humans and digital artefacts. The 4C framework consists of eight interaction principles for multi-user/multi-artefact interaction categorised into four broader themes. We modified the 4C framework to increase applicability and relevance in the context of non-dyadic human-robot interaction. We identify an increasing tendency towards non-dyadic research (36% in 2020), as well as a focus on simultaneous studies (85% from 2006-2020) over sequential. We also articulate seven interaction principles utilised in non-dyadic HRI and provide specific examples. Last, based on our findings, we discuss several salient points of non-dyadic HRI, the applicability of the modified 4C framework to HRI and potential future topics of interest as well as open-questions for non-dyadic research.

This study analysed work activity in a hospital basement where humans and robots interacted and cooperated on logistics tasks. The robots were deployed to automate parts of courier processes and improve the work environment for the hospital's kitchen staff. Human–robot cooperation was studied through ethnographic fieldwork relating to mobile service robots and hospital kitchen staff. The results highlighted problems arising through the assumption that the ‘plug and play’ service robots could effectively automate work tasks. The analysis revealed the complexity of human–robot interaction in dynamic work settings such as hospitals and identified contradictions between the envisioning and realisation of robots at work, as well as the visible and invisible procedures underpinning human–robot cooperation. Consequently, we emphasise the importance of considering robots as agents of change and draw attention to the new work practices that arise when robots assume the roles of workers in dynamic work settings.

Human-robot collaboration in industrial settings is an expanding research field in robotics. When working together, robot mistakes are an important factor to decrease trust and therefore interferes with cooperation. It is unclear whether explanations help to restore human-robot trust after a mistake. In our study, we investigate whether system explanations as a trust-repairing action after a robot makes a mistake in a collaborative task is helpful. Our pilot study revealed that users are more interested in solutions to errors than they are in just why the error happened. Therefore, in our main study, we evaluated three levels of mistake explanations (no explanation, explanation, and explanation with solution) after a robot in VR made a mistake in executing a shared objective. After testing with 30 participants we found that the robot making a mistake significantly affects trust toward the robot, compared to it completing the task successfully. While participants found the explanations helpful to trust or distrust the robot, the levels of the explanation did not lead to an increase in trust towards the robot after a mistake. In addition, we found no significant impact of explanations on self-efficacy and the emotional state of the participants. Our results show that explanations alone are not sufficient to increase human-computer trust after robot mistakes.

With the increased use of social robots in critical applications, like elder care and rehabilitation, it becomes necessary to investigate the user's trust in robots to prevent over- and under-utilization of the robotic systems. While several studies have shown how trust increases through personalised behaviour, there is a lack of research concerned with the influence of personalised physical appearance. This study explores the effect of personalised physical appearance on trust in human-robot-interaction (HRI). In an online game, 60 participants interacted with a robot, where half of the participants were asked to personalise the robot prior to the game. Trust was measured through a trust-related questionnaire as well as by evaluating user behaviour during the game. Results indicate that personalised physical appearance does not directly correlate to higher trust perceptions, however, there was significant evidence that players exhibit more trusting behaviours in a game against a personalised robot.

With increased interest in close-proximity human-robot collaboration in production settings it is important that we understand how robot behaviors and mistakes affect human-robot trust, as a lack of trust can cause loss in productivity and over-trust can lead to hazardous misuse. We designed a system for real-time human-robot trust assessment using a top-down depth camera tracking setup with the goal of using signs of physical apprehension to infer decreases in trust toward the robot. In an experiment with 20 participants we evaluated the tracking system in a repetitive collaborative pick-and-place task where the participant and the robot had to move a set of cones across a table. Midway through the tasks we disrupted the participants expectations by having the robot perform a trust-dampening action. Throughout the tasks we measured the participant’s preferred proximity and their trust toward the robot. Comparing irregular robot movements versus task execution mistakes as well simultaneous versus turn-taking collaboration, we found reported trust was significantly decreased when the robot performed an execution mistake going counter to the shared objective. This decrease was higher for participant working simultaneously as the robot. The effect of the trust-dampening actions on preferred proximity was inconclusive due to unexplained movement trends between tasks throughout the experiment. Despite being given the option to stop the robot in case of abnormal behavior, the trust-dampening actions did not increase the number of participant disruptions for the actions we tested.

In this study we set out to design a computer vision-based system to assess human-robot trust in real time during close-proximity human-robot collaboration. This paper presents the setup and hardware for an augmented reality-enabled human-robot collaboration cell as well as a method of measuring operator proximity using an infrared camera. We tested this setup as a tool for assessing trust through physical apprehension signals in a collaborative drawing task, where participants hold a piece of paper on a table while the robot draws between their hands. Midway through the test we attempt to induce a decrease in trust with an unexpected change in robot speed and evaluate subject motions along with self-reported trust and emotional arousal through galvanic skin response. After performing the experiment with forty participants, we found that reported trust was significantly affected when robot movement speed was increased. The galvanic skin response measurement were not significantly different between the test conditions. The motion tracking method used in this study did not suggest that subjects' motions were significantly affected by the decrease in trust.

We propose a method of human-robot trust assessment in close-proximity human-robot collaboration involving body tracking for recognition of physical signs of apprehension. We tested this by performing skeleton tracking on 30 participant while they repeated a shared task with a Sawyer robot while reporting trust between tasks. We tested different robot velocity and environment conditions with an unannounced increase in velocity midway through to provoke a dip trust. Initial analysis show significant effect for the test conditions on participant movements and reported trust as well as linear correlations between tracked signs of apprehension and reported trust.

Social signal extraction from the facial analysis is a popular research area in human-robot interaction. However, recognition of emotional signals from Traumatic Brain Injured (TBI) patients with the help of robots and non-intrusive sensors is yet to be explored. Existing robots have limited abilities to automatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even more challenging and complex due to unique, unusual and diverse ways of expressing their emotions. To tackle the disparity in a TBI patient's Facial Expressions (FEs), a specialized deep-trained model for automatic detection of TBI patients' emotions and FE (TBI-FER model) is designed, for robot-assisted rehabilitation activities. In addition, the Pepper robot's built-in model for FE is investigated on TBI patients as well as on healthy people. Variance in their emotional expressions is determined by comparative studies. It is observed that the customized trained system is highly essential for the deployment of Pepper robot as a Socially Assistive Robot (SAR).

Robots in the health care sector are often envisioned as a kind of social interaction partner. We suggest a different approach, where robots become adaptable tools for facilitating positive social interaction between and learning for special needs users. The paper presents the development and a series of field tests of a new robot game platform, which is envisioned to level the playing field for users with distinct motor and cognitive capacities by adapting the robots to their abilities. The series of field tests shows that the system is successful in triggering positive social interactions between the players.

Rehabilitation after traumatic brain injury (TBI) is very critical as it is largely unpredictable depending upon the nature of the injury. Rehabilitation process and recovery time also varies, as it takes months and years, depending upon the assessment of treatment, mental and physical conditions and strategies. Due to non-cooperative behaviour of patients, and increase in negative emotional expressions it is very beneficial to evaluate these expressions in a contactless way, and perform a rehabilitation physiotherapy, cognitive or other behavioral activities when the patient is in a positive mood. In this paper we have analyzed the methods for facial features extraction for TBI patients to determine optimal time to have aforementioned rehabilitation process on the basis of positive and negative facial expressions. We have employed a deep learning architecture based on convolutional neural network and long short term memory on RGB and thermal data that were collected in challenging scenarios from real patients. It automatically identifies the patient's facial expressions, and inform experts or trainers that “it is the time” to start rehabilitation session.

In earlier work it has been shown how culture can be used as a parameter influencing human robot interaction in general (e.g. [1]). While this is a good starting point, in our work with concrete application fields we encounter that culture in its usual definition as national culture (e.g. [2]; [3]) is too general a concept to be useful in these concrete applications. Thus, we shifted our focus instead to a concept of local cultural practices, which is derived from situated practices as in Wengers communities of practice [4] and grounded loosely in Sperbers idea of an epidemiology of representations [5], i.e. culture or rather cultural practices as an emergent phenomenon from learning processes in a given group. Developing this new kind of culture-aware robots can then not start from a general definition of culture like Hofstede [2], Schwartz and Sagiv [6], etc. but has to take the actual group of users (and stakeholders) into account. We exemplify this approach with our work in a residency for citizens with acquired brain damage.

The behavior of being disengaged in group work is commonly defined as social loafing. This disengagement can be reduced by various factors, such as group members having individual accountability in the form of unique tasks. This paper examines social loafing in a collaborative mobile game in groups of three users. The game promotes individual accountability in role differentiations on a shared tablet, and these roles are compared to a condition without roles. Another condition tested if a shared tablet would reduce social loafing, compared to individual tablets for each group member. The game was tested on 41 students from 7th to 8th grade. Using a tablet each showed significantly more instances of social loafing compared to sharing one. The results show no significant difference for role assignment.

No abstract available.

The vision of social robotics sees robots moving more and more into unrestricted social environments, where robots interact closely with users in their everyday activities, maybe even establishing relationships with the user over time. In this paper we present a field trial with a robot in a semipublic place. Our analysis of the interactions with casual users shows that it is not enough to focus on modeling behavior that is similar to successful human interactions but that we have to take more deviant ways of interaction like abuse and impoliteness into account when we send robots into the users' environments. The analysis uses impoliteness theory as an analytical toolbox and exemplifies which strategies are employed by users in unexpected encounters with a humanoid robot.

This paper presents how to gain insights into children's navigation of an interactive virtual learning environment and how that would benefit their educators. A prototype for logging user information as quantifiable data has been developed and deployed in an in-situ evaluation of the system. The system collects a hidden corpus of data, which is visualized in order to assess if and how the interaction with the system reflects on the learning gains of the individual learners.

An approach to intercultural training of nonverbal behavior is presented that draws from research on role-plays with virtual agents and ideas from situated learning. To this end, a mobile serious game is realized where the user acquires knowledge about German emblematic gestures and tries them out in role-plays with virtual agents. Gesture performance is evaluated making use of build-in acceleration sensors of smart phones. After an account of the theoretical background covering diverse areas like virtual agents, situated learning and intercultural training, the paper presents the GAME approach along with details on the gesture recognition and content authoring. By its experience-based role-plays with virtual characters, GAME brings together ideas from situated learning and intercultural training in an integrated approach and paves the way for new m-learning concepts.

In this paper, we investigate the mutual-complementary functionality of accelerometer (ACC) and electromyogram (EMG) for recognizing seven word-level sign vocabularies in German sign language (GSL). Results are discussed for the single channels and for feature-level fusion for the bichannel sensor data. For the subject-dependent condition, this fusion method proves to be effective. Most relevant features for all subjects are extracted and their universal effectiveness is proven with a high average accuracy for the single subjects. Additionally, results are given for the subject-independent condition, where subjective differences do not allow for high recognition rates. Finally we discuss a problem of feature-level fusion caused by high disparity between accuracies of each single channel classification.

The workshop is centered around three main research challenges: 1.) Computationally viable models of cultural aspects of conversations: Cultural norms and values penetrate all our communications and interactions by giving us heuristics how to behave and how to interpret the verbal and nonverbal behavior of others. To make such a notion like culture available for computation, we need a very specific theory of culture that takes its effects on communication and interaction into account.2.) Reliable empirical data on cultural/cross-cultural interaction: To realize technical systems that take cultural influences on behavior into account, precise data analysis on how this influence manifests itself is necessary. In the literature, this information is often given in very general forms without to the precise data on which the observations are based.3.) Enculturating conversational interfaces: Having identified cultural influences on verbal/nonverbal communicative behaviors, it remains to be shown how this can be applied to the development of human-computer interfaces, for instance in an interface reflecting cultural norms and values of communication.

A multiplayer dice game was realized which is played by two users and one embodied conversational agent. During the game, the players have to lie to each other to win the game and the longer the game commences the more probable it is that someone is lying, which creates highly emotional situations. We ran a number of evaluation studies with the system. The specific setting allows us to compare user–user interactions directly with user–agent interactions in the same game. So far, the users’ gaze behavior and the users’ verbal behavior towards one another and towards the agent have been analyzed. Gaze and verbal behavior towards the agent partly resembles patterns found in the literature for human–human interactions, partly the behavior deviates from these observations and could be interpreted as rude or impolite like continuous staring, insulting, or talking about the agent. For most of these seemingly abusive behaviors, a more thorough analysis reveals that they are either acceptable or present some interesting insights for improving the interaction design between users and embodied conversational agents.

Recently, there has been a significant amount of work on the recognition of emotions from speech and biosignals.Most approaches to emotion recognition so far concentrate on a single modality and do not take advantage of the fact that an integrated multimodal analysis may help to resolve ambiguities and compensate for errors.In this paper, we describe various methods for fusing physiological and voice data at the feature-level and the decision-level as well as a hybrid integration scheme.The results of the integrated recognition approach are then compared with the individual recognition results from each modality.