To meet the demands in terms of energy-efficient and fast production and delivery of goods, robotic fleets began to populate warehouses and industrial environments. To maximize the profitability of the operations, multi-robot systems are required to coordinate agents and avoid downtime efficiently. In this paper, agent coordination is formulated as a multi-robot task allocation (MRTA) problem with time and precedence constraints. The method capitalizes on a graph method to build a measure graph reflecting the sparsity of tasks and a precedence graph, which includes the task constraints, to group the tasks into batches. A batch solver is provided to obtain the final solutions to the MRTA. In this way, the sustainability and environmental impact of logistics operations can be improved by reducing the number of robots needed to complete tasks and also by assigning tasks closest to the robot location, reducing the amount of time and the total energy required for the robots to complete the job. Extensive experiments on both uniformly distributed and sparse data sets prove the effectiveness of the proposed algorithm compared to state-of-the-art algorithms such as MIP and TePSSI.Note to Practitioners—This paper was motivated by the problem of minimizing the energy consumption of multi-robot systems in the execution of complex tasks, which requires, in the most general case, the motion of the robot to a target location and further on-site operations. This scenario is particularly relevant in smart, automated warehouses, where mobile robots are repeatedly demanded to store or dispatch goods in a structured environment, where operation duration and future requests are known a priori. The paper formulates this problem by means of a batched multi-robot task allocation (BMRTA) optimization, which can include time windows and precedence constraints jointly. First, the task constraints are encoded into two graphs and then combined to group subtasks together in batches. Then, each batch is solved separately, minimizing the overall energy required to achieve the tasks in the batch. Although the optimality of the solution is ensured only locally, i.e., within the same batch, the task clustering improves the computational efficiency with respect to global approaches, especially in large-sized problems. Experimental results demonstrated that when comparing BMRTA with literature approaches such as MIP and TePSSI, not only the energy consumption but also the total travel distance can be minimized while the total duration of the tasks remains comparable.

Observational learning is a promising approach to enable people without expertise in programming to transfer skills to robots in a user-friendly manner, since it mirrors how humans learn new behaviors by observing others. Many existing methods focus on instructing robots to mimic human trajectories, but motion-level strategies often pose challenges in skills generalization across diverse environments. This article proposes a novel framework that allows robots to achieve ahigher-levelunderstanding of human-demonstrated manual tasks recorded in RGB videos. By recognizing the task structure and goals, robots generalize what observed to unseen scenarios. We found our task representation on Shannon's Information Theory (IT), which is applied for the first time to manual tasks. IT helps extract the active scene elements and quantify the information shared between hands and objects. We exploit scene graph properties to encode the extracted interaction features in a compact structure and segment the demonstration into blocks, streamlining the generation of behavior trees for robot replicas. Experiments validated the effectiveness of IT to automatically generate robot execution plans from a single human demonstration. In addition, we provide HANDSOME, an open-source dataset of HAND Skills demOnstrated by Multi-subjEcts, to promote further research and evaluation in this field.

In the context of telehealth, robotic approaches have proven a valuable solution to in-person visits in remote areas, with decreased costs for patients and infection risks. In particular, in ultrasonography, robots have the potential to reproduce the skills required to acquire high-quality images while reducing the sonographer’s physical efforts. In this paper, we address the control of the interaction of the probe with the patient’s body, a critical aspect of ensuring safe and effective ultrasonography. We introduce a novel approach based on variable impedance control, allowing the real-time optimisation of compliant controller parameters during ultrasound procedures. This optimisation is formulated as a quadratic programming problem and incorporates physical constraints derived from viscoelastic parameter estimations. Safety and passivity constraints, including an energy tank, are also integrated to minimise potential risks during human-robot interaction. The proposed method’s efficacy is demonstrated through experiments on a patient’s dummy torso, highlighting its potential for achieving safe behaviour and accurate force control during ultrasound procedures, even in cases of contact loss.

In this paper, we propose a robot oriented knowledge representation system based on the use of the Prolog language. Our framework hinges on a special organisation of Knowledge Base (KB) that enables: 1) its efficient population from natural language texts using semi-automated procedures based on Large Language Models (LLMs); 2) the seamless generation of temporal parallel plans for multi-robot systems through a sequence of transformations; 3) the automated translation of the plan into an executable formalism. The framework is supported by a set of open source tools and its functionality is shown with a realistic application.

This paper introduces a new method for online estimating the penetration of the end-effector and the viscoelastic properties of a soft body, through palpation exams using a collaborative robotic arm. The estimator is based on the dimensionality reduction method that simplifies the nonlinear Hunt-Crossley model. In addition, in our algorithm, the model parameters can be found without a force sensor, leveraging only the robotic arm controller data. An extended Kalman filter is employed to achieve online estimation, which embeds the dynamic contact model. The algorithm is tested with various types of silicone, a material that resembles biological tissues, including samples with hard intrusions to simulate cancerous cells within a softer tissue. The results indicate that this technique can accurately determine the model parameters and estimate the penetration of the end-effector into the soft body. These promising preliminary results demonstrate robots’ potential to be an effective tool for early-stage cancer diagnostics.

Human-robot collaborative disassembly is an emerging trend in the sustainable recycling process of electronic and mechanical products. It requires the use of advanced technologies to assist workers in repetitive physical tasks and deal with creaky and potentially damaged components. Nevertheless, when disassembling worn-out or damaged components, unexpected robot behaviors may emerge, so harmless and symbiotic physical interaction with humans and the environment becomes paramount. This work addresses this challenge at the control level by ensuring safe and passive behaviors in unplanned interactions and contact losses. The proposed algorithm capitalizes on an energy-aware Cartesian impedance controller, which features energy scaling and damping injection, and an augmented energy tank, which limits the power flow from the controller to the robot. The controller is evaluated in a real-world flawed unscrewing task with a Franka Emika Panda and is compared to a standard impedance controller and a hybrid force-impedance controller. The results demonstrate the high potential of the algorithm in human-robot collaborative disassembly tasks.

This paper presents a new method to describe spatio-temporal relations between objects and hands, to recognize both interactions and activities within video demonstrations of manual tasks. The approach exploits Scene Graphs to extract key interaction features from image sequences while simultaneously encoding motion patterns and context. Additionally, the method introduces event-based automatic video segmentation and clustering, which allow for the grouping of similar events and detect if a monitored activity is executed correctly. The effectiveness of the approach was demonstrated in two multi-subject experiments, showing the ability to recognize and cluster hand-object and object-object interactions without prior knowledge of the activity, as well as matching the same activity performed by different subjects.

This paper presents a physical interface for collaborative mobile manipulators in industrial manufacturing and logistics applications. The proposed work builds on our earlier MOCA-MAN interface, through which an operator could be physically coupled to a mobile manipulator to be assisted in performing daily activities. The previous interface was based on a magnetic clamp attached to one arm of the user for the coupling stage, and a bracelet based on EMG sensors on the other arm for human-robot communication via gestures. The new interface instead presents the following additions: i) An industrial-like design that allows the worker to couple/decouple easily and to operate mobile manipulators locally; ii) A simplistic communication channel via a simple buttons board that allows controlling the robot with one hand only; iii) The interface offers enhanced loco-manipulation capabilities that do not compromise the worker mobility. In addition, an experimental evaluation with six human subjects is carried out to analyze the enhanced locomotion and flexibility of the proposed interface in terms of mobility constraint, usability, and physical load reduction.

Even though cobots have high potential in bringing several benefits in the manufacturing and logistic processes, their rapid (re-)deployment in changing environments is still limited. To enable fast adaptation to new product demands and to boost the fitness of the human workers to the allocated tasks, we propose a novel method that optimizes assembly strategies and distributes the effort among the workers in human-robot cooperative tasks. The cooperation model exploits AND/OR Graphs that we adapted to solve also the role allocation problem. The allocation algorithm considers quantitative measurements that are computed online to describe human operators' ergonomic status and task properties. We conducted preliminary experiments to demonstrate that the proposed approach succeeds in controlling the task allocation process to ensure safe and ergonomic conditions for the human worker.

Work-related musculoskeletal disorders (MSD) are one of the major cause of injuries and absenteeism at work. These lead to important cost in the manufacturing industry. Human-robot collaboration can help decreasing this issue by appropriately distributing the tasks and decreasing the workload of the factory worker. This paper proposes a novel generic task allocation approach based on hierarchical finite-state machines for human-robot assembly tasks. The developed framework decomposes first the main task into sub-tasks modelled as state machines. Based on capabilities considerations, workload, and performance estimations, the task allocator assigns the sub-task to human or robot agent. The algorithm was validated on the assembly of a crusher unit of a smoothie machine using the collaborative Franka Emika Panda robot and showed promising results in terms of productivity thanks to task parallelization, with improvement of more than 30% of the total assembly time with respect to a collaborative scenario, where the agents perform the tasks sequentially.

This paper proposes a novel integrated dynamic method based on Behavior Trees for planning and allocating tasks in mixed human robot teams, suitable for manufacturing environments. The Behavior Tree formulation allows encoding a single job as a compound of different tasks with temporal and logic constraints. In this way, instead of the well-studied offline centralized optimization problem, the role allocation problem is solved with multiple simplified online optimization sub-problem, without complex and cross-schedule task dependencies. These sub-problems are defined as Mixed-Integer Linear Programs, that, according to the worker-actions related costs and the workers' availability, allocate the yet-to-execute tasks among the available workers. To characterize the behavior of the developed method, we opted to perform different simulation experiments in which the results of the action-worker allocation and computational complexity are evaluated. The obtained results, due to the nature of the algorithm and to the possibility of simulating the agents' behavior, should describe well also how the algorithm performs in real experiments.

The objective of this paper is to create a new collaborative robotic system that subsumes the advantages of mobile manipulators and supernumerary limbs. By exploiting the reconfiguration potential of a MObile Collaborative robot Assistant (MOCA), we create a collaborative robot that can function autonomously, in close proximity to humans, or be physically coupled to the human counterpart as a supernumerary body (MOCA-MAN). Through an admittance interface and a hand gesture recognition system, the controller can give higher priority to the mobile base (e.g., for long distance co-carrying tasks) or the arm movements (e.g., for manipulating tools), when performing conjoined actions. The resulting system has a high potential not only to reduce waste associated with the equipment waiting and setup times, but also to mitigate the human effort when performing heavy or prolonged manipulation tasks. The performance of the proposed system, i.e., MOCA-MAN, is evaluated by multiple subjects in two different use-case scenarios, which require large mobility or close-proximity manipulation.

In this paper, a novel human-robot collaborative framework for mixed case palletizing is presented. The framework addresses several challenges associated with the detection and localisation of boxes and pallets through visual perception algorithms, high-level optimisation of the collaborative effort through effective role-allocation principles, and maximisation of packing density. A graphical user interface (GUI) is additionally developed to ensure an intuitive allocation of roles and the optimal placement of the boxes on target pallets. The framework is evaluated in two conditions where humans operate with and without the support of a Mobile COllaborative robotic Assistant (MOCA). The results show that the optimised placement can improve up to the 20% with respect to a manual execution of the same task, and reveal the high potential of MOCA in increasing the performance of collaborative palletizing tasks.

In this work, we propose a novel visuo-haptic guidance interface to enable mobile collaborative robots to follow human instructions in a way understandable by non-experts. The interface is composed of a haptic admittance module and a human visual tracking module. The haptic guidance enables an individual to guide the robot end-effector in the workspace to reach and grasp arbitrary items. The visual interface, on the other hand, uses a real-time human tracking system and enables autonomous and continuous navigation of the mobile robot towards the human, with the ability to avoid static and dynamic obstacles along its path. To ensure a safer human-robot interaction, the visual tracking goal is set outside of a certain area around the human body, entering which will switch robot behaviour to the haptic mode. The execution of the two modes is achieved by two different controllers, the mobile base admittance controller for the haptic guidance and the robot's whole-body impedance controller, that enables physically coupled and controllable locomotion and manipulation. The proposed interface is validated experimentally, where a human-guided robot performs the loading and transportation of a heavy object in a cluttered workspace, illustrating the potential of the proposed Follow-Me interface in removing the external loading from the human body in this type of repetitive industrial tasks.