In the Industry 5.0 era, the focus shifts from basic automation to fostering collaboration between humans and robots. Trust is crucial in this new paradigm, enabling smooth interaction, especially for users with limited robotics knowledge. This study presents a novel framework that uses human hand gestures and voice commands to control robot movements, aiming to enhance trust, reduce cognitive workload, and minimize task execution time-key for efficient manufacturing. In automated systems, swift completion of micromanagement tasks is essential to prevent process disruption. To evaluate this framework, we devised a testbed scenario within an automated carbon fiber transportation and draping process, focusing on a maintenance task as the micromanagement challenge. Participants inspected the gripper, guided the robot along a defined path, and performed maintenance, such as attaching cables. Two conditions were tested: gestures and voice commands versus a smartPAD. The results showed that gestures and voice commands increased trust, lowered cognitive load, and shortened execution times, improving overall manufacturing efficiency.

Despite the importance of human-robot collaborative transportation of flexible material in many industrial scenarios, many works in the literature assume a passive role for the robot during the collaboration. The robot can only follow the human partner, without providing assistance in the more challenging phase of the collaboration such as precise material positioning. This work presents a framework for co-transportation, proposing a distance-based policy for dynamic leader role allocation through the task. For large distances from the target pose, the robot is mainly controlled by vision-based manual guidance exploiting haptic feedback and 3D human pose information; instead, close to the target material position, the robot acts as a leader guiding the human operator. The proposed framework is evaluated considering a carbon fiber draping task, which requires both co-transportation and precise positioning of flexible materials. Experimental results demonstrate how the robot leading the task in the final stage allows to achieve high task efficiency and alleviates human stress in the execution of the task.

Robot grasping has been widely studied in the last decade. Recently, Deep Learning made possible to achieve remarkable results in grasp pose estimation, using depth and RGB images. However, only few works consider the choice of the object to grasp. Moreover, they require a huge amount of data for generalizing to unseen object categories. For this reason, we introduce the Few-shot Semantic Grasping task where the objective is inferring a correct grasp given only five labelled images of a target unseen object. We propose a new deep learning architecture able to solve the aforementioned problem, leveraging on a Few-shot Semantic Segmentation module. We have evaluated the proposed model both in the Graspnet dataset and in a real scenario. In Graspnet, we achieve 40,95% accuracy in the Few-shot Semantic Grasping task, outperforming baseline approaches. In the real experiments, the results confirmed the generalization ability of the network.

In this paper, we present a control system for the continuous teleoperation of a robotic manipulator via brain-machine interface (BMI). The proposed solution is based on shared control approach that allows the user to only focus on the operational tasks, while the low-level control details are automatically handled by the robotic intelligence. The user drives the manipulator through the imagination of limb movements (both hands vs. both feet) and a parameterized mapping function is implemented to convert the continuous BMI outputs into robot velocity commands which are sent to the shared control framework. The latter consists in: (i) a target predictor module, to infer the most probable target objects from the sequence of BMI commands; (ii) a control module based on an improved version of artificial potential fields (APF) to assist the user in reaching the target while avoiding collisions with obstacles in the environment. The system has been tested with a sample subject in a tabletop reach-to-grasp experiment with multiple target objects and obstacles achieving a success rate of 80%. The proposed system could be used in the future to help people with severe motor disabilities in performing daily life operations, such as drinking, feeding or manipulating objects.

In shared control teleoperation, the robot assists the user in accomplishing the desired task. Rather than simply executing the user’s command, the robot attempts to integrate it with information from the environment, such as obstacle and/or goal locations, and it modifies its behavior accordingly. In this article, we propose a real-time shared control teleoperation framework based on an artificial potential field approach improved by the dynamic generation of escape points around the obstacles. These escape points are virtual attractive points in the potential field that the robot can follow to overcome the obstacles more easily. The selection of which escape point to follow is done in real time by solving a soft-constrained problem optimizing the reaching of the most probable goal, estimated from the user’s action. Our proposal has been extensively compared with two state-of-the-art approaches in a static cluttered environment and a dynamic setup with randomly moving objects. Experimental results showed the efficacy of our method in terms of quantitative and qualitative metrics. For example, it significantly decreases the time to complete the tasks and the user’s intervention, and it helps reduce the failure rate. Moreover, we received positive feedback from the users that tested our proposal. Finally, the proposed framework is compatible with both mobile and manipulator robots.