The shift in research focus from Industry 4.0 (I4.0) to Industry 5.0 (I5.0) promises a human-centric workplace, with social and well-being values at the centre of technological implementation. Human-Robot Collaboration (HRC) is a core aspect of I5.0 development, with an increase in adaptive and personalised interactions and behaviours. This review investigates recent advancements towards personalised HRC, where user-centric adaptation is key. There is a growing trend for adaptable HRC research, however there lacks a consistent and unified approach. The review highlights key research trends on which personal factors are considered, the design of human-robot interaction, and adaptive task completion. This raises various key considerations for future developments, particularly around the ethical and regulatory development of personalised systems, which are discussed in detail.

Modern industry is increasingly moving away from mass manufacturing, towards more specialized and per-sonalized products. As manufacturing tasks become more complex, full automation is not always an option, human involvement may be required. This has increased the need for advanced human robot collaboration (HRC), and with it, improved methods for interaction, such as voice control. Recent advances in natural language processing, driven by artificial intelligence (AI), have the potential to answer this demand. Large language models (LLMs) have rapidly developed very impressive general reasoning capabilities, and many methods of applying this to robotics have been proposed, including through the use of code generation. This paper presents Language Model Program Voice Control (LMPVC), an LLM-based prototype voice control architecture with integrated policy programming and teaching capabilities, built for use with Robot Operating System 2 (ROS2) compatible robots. The architecture builds on prior works using code generation for voice control by implementing an additional programming and teaching system, the Policy Bank. We find this system can compensate for the limitations of the underlying LLM, and allow LMPVC to adapt to different downstream tasks without a slow and costly training process. The architecture and additional results are released on GitHub (https://github.com/ozzyuni/LMPVC).

Pointing gestures are a common interaction method used in Human-Robot Collaboration for various tasks, ranging from selecting targets to guiding industrial processes. This study introduces a method for localizing pointed targets within a planar workspace. The approach employs pose estimation to detect shoulder and wrist keypoints, and uses linear extrapolation to extract gesturing data from an RGB-D stream. The study proposes a rigorous methodology and comprehensive analysis for evaluating pointing gestures and target selection in typical robotic tasks. In addition to evaluating accuracy, the gesturing method is integrated into a proof-of-concept robotic system, which includes object detection, speech transcription, and speech synthesis to demonstrate the integration of multiple modalities in a collaborative application. Finally, a discussion over method limitations and performance is provided to understand its role in multimodal robotic systems. All developments are available at: https://github.com/NMKsas/gesture_pointer.git

Robot control for tactile feedback based manip-ulation can be difficult due to modeling of physical contacts, partial observability of the environment, and noise in perception and control. This work focuses on solving partial observability of contact-rich manipulation tasks as a Sequence-to-Sequence (Seq2Seq) Imitation Learning (IL) problem. The proposed Seq2Seq model first produces a robot-environment interaction sequence to estimate the partially observable environment state variables, and then, the observed interaction sequence is transformed to a control sequence for the task itself. The proposed Seq2Seq IL for tactile feedback based manipulation is experimentally validated on a door-open task in a simulated environment and a snap-on insertion task with a real robot. The model is able to learn both tasks from only 50 expert demonstrations while state-of-the-art reinforcement learning and imitation learning methods fail.

Non-anthropomorphic robots can be cost-effective and efficient choice in certain context in comparison to social or humanoid robots. However, introduction of nonanthropomorphic robots can evoke uncertainty and anxiety due to novelty of technology. The goal of this paper is to explore personality design space for non-anthropomorphic wellness robots in office environment to foster acceptance among users. Through Participatory Design approach, we explored appropriate personalities for a well-being robot, which would detect employees’ sitting posture and suggest small wellness interventions. We addressed the following research questions: (i) How can personalities be designed and integrated to non-anthropomorphic wellness robots to promote users’ acceptance? (ii) How do the users perceive designed personalities of non-anthropomorphic wellness robot in the office context? We conducted one contextual inquiry (n=5) and one co-design workshop (n=15) followed by evaluation (n=5) in IT office environment with office employees. As a contribution to the paper, we present personalities and design implications for non-anthropomorphic wellness robot in the office context. Our contribution will serve as a guideline for designers to explore and expand their knowledge on designing robot personalities for non-anthropomorphic robots in the context.

Existing Deep Learning (DL) frameworks typically do not provide ready-to-use solutions for robotics, where very specific learning, reasoning, and embodiment problems exist. Their relatively steep learning curve and the different methodologies employed by DL compared to traditional approaches, along with the high complexity of DL models, which often leads to the need of employing specialized hardware accelerators, further increase the effort and cost needed to employ DL models in robotics. Also, most of the existing DL methods follow a static inference paradigm, as inherited by the traditional computer vision pipelines, ignoring active perception, which can be employed to actively interact with the environment in order to increase perception accuracy. In this paper, we present the Open Deep Learning Toolkit for Robotics (OpenDR). OpenDR aims at developing an open, non-proprietary, efficient, and modular toolkit that can be easily used by robotics companies and research institutions to efficiently develop and deploy AI and cognition technologies to robotics applications, providing a solid step towards addressing the aforementioned challenges. We also detail the design choices, along with an abstract interface that was created to overcome these challenges. This interface can describe various robotic tasks, spanning beyond traditional DL cognition and inference, as known by existing frameworks, incorporating openness, homogeneity and robotics-oriented perception e.g., through active perception, as its core design principles.

Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid obstacles". A straightforward solution in the context of reinforcement learning (RL) is to combine multiple objectives into a multi-term reward function and train a single monolithic controller. Recently, a hybrid solution based on pre-trained single objective controllers and a switching rule between them was proposed. In this work, we compare these two approaches in the multi-objective setting of a robot manipulator to reach a target while avoiding an obstacle. Our findings show that the training of a hybrid controller is easier and obtains a better success-failure trade-off than a monolithic controller. The controllers trained in simulator were verified by a real set-up.

The integration of collaborative robots (cobots) is changing manufacturing and production processes in factories. When cobots are designed to be efficient, skillful and safe to interact with, workers can collaborate with them conveniently. As workers often work with cobots intensively, it is crucial to explore the user experience (UX) of cobots. The goal of our research is to explore how factory cobots could be used in ways that support pleasurable worker experiences. We adapted "research through design" (RtD) to conduct exploratory research on novel interactions related to affectiveness, playfulness and relatedness in human-robot collaboration (HRC) using collaborative robot arm, Panda. RtD is a method that utilizes methods and practices of design to produce new knowledge. We conducted an exploratory study with 33 participants to evaluate three HRC storyboards scenarios in two complementary remote workshops. The findings report suitability of affective and playful behavior of cobots in an industrial setting. In addition, we deduced that personality of the robot plays a crutial role in HRC.

The modelling of cognition is playing a major role in robotics. Indeed, robots need to learn, adapt and plan their actions in order to interact with their environment. To do so, approaches like embodiment and enactivism propose to ground sensorimotor experience in the robot's body to shape the development of cognition. In this work, we focus on the role of memory during learning in a closed loop. As sensorimotor contingencies, we consider a robot arm that moves a baby mobile toy to get visual reward. First, the robot explores the continuous sensorimotor space by associating visual stimuli to motor actions through motor babbling. After exploration, the robot uses the experience from its memory and exploits it, thus optimizing its motion to perceive more visual stimuli. The proposed approach uses Dynamic Field Theory and is integrated in the GummiArm, a 3D printed humanoid robot arm. The results indicate a higher visual neural activation after motion learning and show the benefits of an embodied babbling strategy.

In the past years human-robot collaboration has gained interest among industry and production environments. While there is interest towards the topic, there is a lack of industrially relevant cases utilizing novel methods and technologies. The feasibility of the implementation, worker safety and production efficiency are the key questions in the field. The aim of the proposed work is to provide a conceptual safety system for context-dependent, multi-modal communication in human-robot collaborative assembly, which will contribute to safety and efficiency of the collaboration. The approach we propose offers an addition to traditional interfaces like push buttons installed at fixed locations. We demonstrate an approach and corresponding technical implementation of the system with projected safety zones based on the dynamically updated depth map and a graphical user interface (GUI). The proposed interaction is a simplified two-way communication between human and the robot to allow both parties to notify each other, and for the human to coordinate the operations.

Robotic systems developed for support can provide assistance in various ways. However, regardless of the service provided, the quality of user interaction is key to adoption by the general public. Simple communication difficulties, such as terminological differences, can make or break the acceptance of robots. In this work we take into account these difficulties in communication between a human and a robot. We propose a system that allows to handle unknown concepts through symbol manipulation based on natural language interactions. In addition, ontologies are used as a convenient way to store the knowledge and reason about it. To demonstrate the use of our system, two scenarios are described and tested with a Care-O-Bot 4. The experiments show that confusions and difficulties in communication can effectively be resolved through symbol manipulation.

We introduce the modelling and control of a rolling microrobot. The microrobot is capable of manipulating micro-objects through the use of a magnetic visual control system. This system consists of a rod-shaped microrobot, a magnetic actuation system and a visual control system. Motion of the rolling microrobot on a supporting surface is induced by a rotating magnetic field. As the robot is submerged in a liquid this motion creates a rising flow in front, a sinking flow behind, and a vortex above the robot, thus enabling non-contact transportation of micro-objects. Besides this fluid-vortex approach, the microrobot is also able to manipulate micro-objects via a pushing strategy. We present the design and modelling of the 50×60×300 μm micro-agent, the visual control system, and an experimental analysis of the micromanipulation and control methods.

In this paper, we present a holistic system for automating the motion of a rolling microrobot for protein crystal harvesting. The RodBot, which was introduced in previous work, is able to perform noncontact manipulation of microscopic objects such as fragile crystals by trapping them in an induced vortex fluid flow. Here, we are concerned with navigating the RodBot autonomously in a liquid environment containing obstacles such as crystals. A literature review shows existing approaches to untethered microrobot control are limited and cluttered environments are often not considered. We demonstrate real-time tracking of the RodBot and surrounding obstacles, kinematic obstacle-free path planning, and nonholonomic path following. The system was evaluated in qualitative and quantitative experiments, shows satisfactory performance, and presents itself as a first step towards fully automated crystal harvesting.

In this work, a magnetic visual control system for automated protein crystal harvesting is proposed. The system consists of a rod-shaped microrobot, a magnetic actuation system and a visual control system. A rotating magnetic field induces the microrobot to roll on the supporting surface, thereby creating a vortex in a liquid environment. This vortex enables the robot to trap and transport even delicate objects in a non-contact manner to a pre-defined position. We present the micro-agent, the actuation system and the visual control system to achieve this automated procedure.

This paper presents a hybrid magnetic-mechanical manipulation system for automated capsulorhexis utilizing a flexible catheter with a sharp edge magnetic tip. Vision based closed loop control is implemented to guide the tip on a circular path in the anterior eye segment. A continuous motion with high repeatability is achieved. The system shows the first catheter-based application of the electromagnetic manipulation system, OctoMag, for fast and safe ophthalmic surgery that potentially reduces the risk of complications and improves precision.

This paper presents direct methods for vision-based control for the application of industrial inkjet printing. In this, visual control is designed with a direct coupling between camera measurements and joint motion. Traditional visual servoing commonly has a slow visual update rate and needs an additional local joint controller to guarantee stability. By only using the product as reference and sampling with a high update rate, direct visual measurements are sufficient for controlled positioning. The proposed method is simpler and more reliable than standard motor encoders, despite the tight real-time constraints. This direct visual control method is experimentally verified with a 2D planar motion stage for micrometer positioning. To achieve accurate and fast motion, a balance is found between frame rate and image size. With a frame rate of 1600 fps and an image size of 160 × 100 pixels we show the effectiveness of the approach.

A new visual servoing method is proposed which uses position based visual servoing (PBVS) in combination with an additional image based control layer on the target pose to maintain fixation on an object. The proposed method (denoted feed forward PBVS) does not require trajectory generation but instead uses via-points to explore the object. It exploits the advantages of PBVS without the disadvantages of image based visual servoing (IBVS) as occurs in hybrid approaches. The proposed method is experimentally validated with a redundant 7-DOF manipulator. Comparison with existing visual servoing methods (PBVS and one partitioned approach) shows the effectiveness of the method.