Recent research suggests that conversation systems using voices resembling a user’s close contacts, such as family and friends, may be perceived as more persuasive, though this area remains understudied. However, the impact of voices resembling familiar characters on interactions with autonomous systems remains not fully understood. To facilitate communication, conversational systems could tap into users’ existing perceptions of familiar characters. This article explores how character-inspired voices, based on well-known personalities, influence user perception and interaction with AI systems. The study involved 26 participants interacting with an autonomous conversation platform featuring three voice profiles inspired by famous figures. Participants’ preferences for these real characters (most liked, neutral, and most disliked) were assessed before interaction, and their perceptions of the AI system were evaluated afterward. Results showed that users’ preferences for specific personalities influenced their perception of the AI system in terms of engagement, interest, and knowledge, suggesting that familiar voices can improve user experience and provide insights for enhancing voice-based AI systems.

Collaborative robots (cobots) and AI technologies are increasingly adopted in industrial settings to enhance productivity and efficiency. While cobots equipped with AI capabilities can enable more collaborative work between people and machines, they also face worker acceptance challenges. Understanding future workers’ perceptions, attitudes, and experiences with cobots and conversational AIs can inform robot designers and developers to design systems that promote collaboration, trust, and acceptance. In this study, we gathered quantitative and qualitative data from 37 participants enrolled in a vocational training program for industrial factory workers, who interacted with an AI-empowered, voice-enabled cobot during a simulated smart-factory assembly task and visited an art exhibition featuring industrial robots and cobots. While these participants are not currently employed in factories, they are considered proxy users —individuals with relevant domain knowledge and training who represent future factory workers. The art exhibition functioned as a design probe to illicit discussion and prompt critical reflection about automation and the role of artificial emotions in HRI. The smart-factory task offered participants a concrete example of how AI-empowered virtual assistants might be combined with cobots on the factory floor. In contrast with some of the HRI literature, participants expressed a strong preference for robots without emotional displays and social behaviors, challenging the view that anthropomorphism and human-like emotions promote robot acceptance. Based on our study, we propose design recommendations for developing AI-empowered, voice-enabled cobots based on five themes generated from the qualitative data.

In the Industry 5.0 era, the focus shifts from basic automation to fostering collaboration between humans and robots. Trust is crucial in this new paradigm, enabling smooth interaction, especially for users with limited robotics knowledge. This study presents a novel framework that uses human hand gestures and voice commands to control robot movements, aiming to enhance trust, reduce cognitive workload, and minimize task execution time-key for efficient manufacturing. In automated systems, swift completion of micromanagement tasks is essential to prevent process disruption. To evaluate this framework, we devised a testbed scenario within an automated carbon fiber transportation and draping process, focusing on a maintenance task as the micromanagement challenge. Participants inspected the gripper, guided the robot along a defined path, and performed maintenance, such as attaching cables. Two conditions were tested: gestures and voice commands versus a smartPAD. The results showed that gestures and voice commands increased trust, lowered cognitive load, and shortened execution times, improving overall manufacturing efficiency.

Accurate evaluation of human attention in Human-Robot Collaboration (HRC) is essential to ensure intuitive and safe interactions. Although recent research has made progress in predicting human attention in social contexts, accurately estimating attention in industrial settings remains a challenge, particularly in industrial settings where it is crucial for error prevention, productivity optimization, and maintaining a secure work environment. In this paper, we present a multimodal deep fusion framework, ELLISON, designed to predict human attention lapses during HRC activities in manufacturing assembly tasks. First, we introduce a multimodal attention tracking pipeline featuring dual backbone Transformer Encoder models, which is trained to identify areas of interest related to human attention incorporating operator’s head pose and gaze features. Secondly, a deep fusion method is applied to consolidate the outputs of both modalities using a learned weighted strategy to provide a unified estimate of human attention throughout the tracking process. Experimental results demonstrate the effectiveness of our approach in tracking human attention in identifying instances where workers paid close attention or became distracted while working alongside robots, during collaborative tasks. The proposed method has potential applications in enhancing safety measures, optimizing workflows, and improving efficiency of HRC in industrial environments.

An emerging trend in the field of human-robot collaboration is the disassembly of end-of-life products. Safety is a crucial requirement of the disassembly process since worn-out or damaged products could break, possibly resulting in dangerous behavior of the robot. To protect the user from such behavior, this work addresses this challenge through the implementation of an energy-aware Cartesian impedance controller, combined with virtual workspace restrictions. Hereby, the passivity of the robotic system is ensured. The paper proposed two approaches to ensure the passivity of the system when subjected to workspace restrictions due to unplanned interactions and contact loss. The first approach employs an augmented energy tank with restricted energy flow. The second approach monitors the overall energy flow, regulating and separating non-passive behavior, caused by workspace restrictions. The approaches are evaluated and compared with each other, by using a KUKA LBR iiwa robot. The results highlight the potential of virtual workspace restrictions in human-robot collaborative disassembly tasks.

Industry 5.0 signifies a transformative era where humans and robots collaborate closely, leading to advancements in manufacturing efficiency and personalization. In light of this, it becomes essential to assess the robot’s trustworthiness to ensure a secure environment and equitable workload distribution. The majority of trust assessments hinge on post-hoc questionnaires for the extent of trust experienced during the interaction. A data-driven approach is required to promptly assess trust levels in real-time, allowing for the adjustment of robot behavior to align with human needs. The paper proposes a chemical industry scenario where a robot assisted a human in the process of mixing chemicals. Several machine learning models, including deep learning, were developed using body motion data to categorize the level of trust exhibited by the human operator. The models achieve an accuracy exceeding 90%. The results clearly show the feasibility of data-driven trust assessment.

Human-robot collaborative disassembly is an emerging trend in the sustainable recycling process of electronic and mechanical products. It requires the use of advanced technologies to assist workers in repetitive physical tasks and deal with creaky and potentially damaged components. Nevertheless, when disassembling worn-out or damaged components, unexpected robot behaviors may emerge, so harmless and symbiotic physical interaction with humans and the environment becomes paramount. This work addresses this challenge at the control level by ensuring safe and passive behaviors in unplanned interactions and contact losses. The proposed algorithm capitalizes on an energy-aware Cartesian impedance controller, which features energy scaling and damping injection, and an augmented energy tank, which limits the power flow from the controller to the robot. The controller is evaluated in a real-world flawed unscrewing task with a Franka Emika Panda and is compared to a standard impedance controller and a hybrid force-impedance controller. The results demonstrate the high potential of the algorithm in human-robot collaborative disassembly tasks.

skrl is an open-source modular library for reinforcement learning written in Python and designed with a focus on readability, simplicity, and transparency of algorithm implementations. In addition to supporting environments that use the traditional interfaces from OpenAI Gym/Farama Gymnasium, DeepMind and others, it provides the facility to load, configure, and operate NVIDIA Isaac Gym, Isaac Orbit, and Omniverse Isaac Gym environments. Furthermore, it enables the simultaneous training of several agents with customizable scopes (subsets of environments among all available ones), which may or may not share resources, in the same run. The library's documentation can be found at https://skrl.readthedocs.io and its source code is available on GitHub at https://github.com/Toni-SM/skrl.

This paper presents a natural language-enabled virtual assistant (VA), named Max, developed to support flexible and scalable human–robot interactions (HRI) with industrial robots. Regardless of the numerous natural language interfaces already proposed for intuitive HRI on the industrial shop floor, most of those interfaces remain tightly bound with a specific robotic system. Besides, the lack of a natural and efficient human–robot communication protocol hinders the user experience. Therefore, three key elements characterize the proposed framework. First, a Client-Server style architecture is introduced so Max can provide a centralized solution for managing and controlling various types of robots deployed on the shop floor. Second, inspired by human-human communication, two conversation strategies, lexical-semantic and general diversion strategies, are used to guide Max’s response generation. These conversation strategies were embedded to improve the operator’s engagement with the manufacturing tasks. Third, we fine-tuned the state-of-the-art (SOTA) pre-trained model, Bidirectional Encoder Representations from Transformers (BERT), to support a highly accurate prediction of requested intents from the operator and robot services. Multiple experiments were conducted using the latest iteration of our autonomous industrial mobile manipulator, “Little Helper (LH)”, to validate Max’s performance in a real manufacturing environment.

Equipping robots with the ability to identify who is talking to them is an important step towards natural and effective verbal interaction. However, speaker identification for voice control remains largely unexplored compared to recent progress in natural language instruction and speech recognition. This motivates us to tackle text-independent speaker identification for human-robot interaction applications in industrial environments. By representing audio segments as time-frequency spectrograms, this can be formulated as an image classification task, allowing us to apply state-of-the-art convolutional neural network (CNN) architectures. To achieve robust prediction in unconstrained, challenging acoustic conditions, we take a data-driven approach and collect a custom dataset with a far-field microphone array, featuring over 3 hours of "in the wild" audio recordings from six speakers, which are then encoded into spectral images for CNN-based classification. We propose a shallow 3-layer CNN, which we compare with the widely used ResNet-18 architecture: in addition to benchmarking these models in terms of accuracy, we visualize the features used by these two models to discriminate between classes, and investigate their reliability in unseen acoustic scenes. Although ResNet-18 reaches the highest raw accuracy, we are able to achieve remarkable online speaker recognition performance with a much more lightweight model which learns lower-level vocal features and produces more reliable confidence scores. The proposed method is successfully integrated into a robotic dialogue system and showcased in a mock user localization and authentication scenario in a realistic industrial environment: https://youtu.be/IVtZ8LKJZ7A.

To this day, most robots are installed behind safety fences, separated from the human. New use-case scenarios demand for collaborative robots, e.g. to assist the human with physically challenging tasks. These robots are mainly installed in work-environments with limited space, e.g. existing production lines. This brings certain challenges for the control of such robots. The presented work addresses a few of these challenges, namely: stable and safe behaviour in contact scenarios; avoidance of restricted workspace areas; prevention of joint limits in automatic mode and manual guidance. The control approach in this paper extents an Energy-aware Impedance controller by repulsive potential fields in order to comply with Cartesian and joint constraints. The presented controller was verified for a KUKA LBR iiwa 7 R800 in simulation as well as on the real robot.

This paper defines a framework for virtual recommissioning in reconfigurable manufacturing systems. The need for virtual recommissioning arises with the multiple commissioning tasks in the life span of a reconfigurable manufacturing system. A classification of reconfiguration complexity and elementary abilities are combined in a reconfiguration matrix. The reconfiguration matrix serves as a framework for future research in virtual recommissioning. Lastly a preliminary exploration of virtual recommissioning is conducted on models of an Industry 4.0 Smart Factory demonstrator.