Robots have surpassed humans in terms of strength and precision, yet humans retain an unparalleled ability for decision-making in the face of unpredictable disturbances. This article aims to combine the strengths of both entities within a singular task: human motion guidance under strict geometric constraints, particularly adhering to predetermined paths. To tackle this challenge, a modular haptic guidance law is proposed that takes the human-applied wrench as an input. Using an auxiliary variable called phase, the generated desired motion is guaranteed to consistently adhere to the constraint path. The guidance policy can be generalized into physically interpretable terms, adjustable either prior to initiating the task or dynamically while the task is in progress. An illustrative guidance adaptation policy is showcased that takes into account the human's manipulability. Passivity analysis is used to ensure overall system stability. Experiments, including a 20-participant user study, explore various aspects of the approach in practice.

Gaussian Processes (GPs), as a nonparametric learning method, offer flexible modeling capabilities and calibrated uncertainty quantification for function approximations. Additionally, GPs support online learning by efficiently incorporating new data with polynomial-time computation, making them well-suited for safety-critical dynamical systems that require rapid adaptation. However, the inference and online updates of exact GPs, when processing streaming data, incur cubic computation time and quadratic storage memory complexity, limiting their scalability to large datasets in real-time settings. In this paper, we propose a streaming kernel-induced progressively generated expert framework of Gaussian processes (SkyGP) that addresses both computational and memory constraints by maintaining a bounded set of experts, while inheriting the learning performance guarantees from exact Gaussian processes. Furthermore, two SkyGP variants are introduced, each tailored to a specific objective, either maximizing prediction accuracy (SkyGP-Dense) or improving computational efficiency (SkyGP-Fast). The effectiveness of SkyGP is validated through extensive benchmarks and real-time control experiments demonstrating its superior performance compared to state-of-the-art approaches.

Robotic assembly tasks remain an open challenge due to their long horizon nature and complex part relations. Behavior trees (BTs) are increasingly used in robot task planning for their modularity and flexibility, but creating them manually can be effort-intensive. Large language models (LLMs) have recently been applied to robotic task planning for generating action sequences, yet their ability to generate BTs has not been fully investigated. To this end, we propose LLM-as-BT-Planner, a novel framework that leverages LLMs for BT generation in robotic assembly task planning. Four in-context learning methods are introduced to utilize the natural language processing and inference capabilities of LLMs for producing task plans in BT format, reducing manual effort while ensuring robustness and comprehensibility. Additionally, we evaluate the performance of fine-tuned smaller LLMs on the same tasks. Experiments in both simulated and real-world settings demonstrate that our framework enhances LLMs' ability to generate BTs, improving success rate through in-context learning and supervised fine-tuning.

Large Language Models (LLMs) have gained popularity in task planning for long-horizon manipulation tasks. To enhance the validity of LLM-generated plans, visual demonstrations and online videos have been widely employed to guide the planning process. However, for manipulation tasks involving subtle movements but rich contact interactions, visual perception alone may be insufficient for the LLM to fully interpret the demonstration. Additionally, visual data provides limited information on force-related parameters and conditions, which are crucial for effective execution on real robots. In this paper, we introduce LEMMo-Plan, an in-context learning framework that incorporates tactile and force-torque information from human demonstrations to enhance LLMs' ability to generate plans for new task scenarios. We propose a bootstrapped reasoning pipeline that sequentially integrates each modality into a comprehensive task plan. This task plan is then used as a reference for planning in new task configurations. Real-world experiments on two different sequential manipulation tasks demonstrate the effectiveness of our framework in improving LLMs' understanding of multi-modal demonstrations and enhancing the overall planning performance. More materials are available on our project website: lemmo-plan.github.io/LEMMo-Plan/.

This study presents a comprehensive experimental analysis of Twisted String Actuators (TSA), focused on enhancing contraction modelling accuracy and establishing a baseline for TSA tension and impedance control efficacy. A novel TSA string radius function is introduced, computing effective radii for multi-strand bundles based on axial actuator tension. The proposed model was validated in physical experiments, resulting in a reduction of maximal errors between measured and simulated actuator contraction trajectories from up to 60 % in established models to around 10% in our work. Additionally, the tension-dependent radius modification effectively reduced errors between the estimated and the measured bundle tension by an order of magnitude, marking an essential step towards TSA control independent of bundle tension measurements. TSA tension control was assessed based on four metrics: accu-racy, precision, impact stability, and bandwidth, following ISO 9283:1998 standards. The quality of tension control was found to be dependent on bundle tension, twisting angle and strand quantity, whereas impact stability was maintained in all config-urations. Joint impedance control with TSA was evaluated for perturbation stability and position control bandwidth, where the latter was enhanced with increasing joint stiffness. The presented analysis informs designers about the capabilities of TSAs in different configurations, and their respective suitability for desired applications.

Robust perception of the environment is a critical challenge for robots, especially those that use mobile platforms or humanoid forms to perform manipulation tasks. Active vision, leveraging strategic camera movements and adaptive imaging parameters, holds great potential for addressing critical challenges such as achieving high accuracy in precise manipulation, ensuring low latency for rapid responsiveness, and overcoming occlusions and illumination variations in dynamic environments. This paper introduces a novel, cost-effective, and easily deployable active vision system designed to enhance visual perception for robotic applications. Integrated with a novel hybrid software setup, the system utilizes ArUco markers to achieve high-accuracy, low-latency performance, boasting sub-millimeter and sub-degree accuracy at 200 Hz with a latency of less than 15 ms. Additionally, a new measurement and evaluation procedure is presented, offering benchmarking for marker-based object detection systems that for the first time includes rotation measurements as well. The benchmarking results for the proposed system indicate that achieving the desired performance levels necessitates specialized active vision measurement strategies. For instance, to ensure high positional accuracy, the system needs precise object centering, while high rotational accuracy requires accounting for lateral or rotational offsets.

The increasing integration of robots in close human environments necessitates robust safety measures that can adapt to evolving tasks and conditions. Current standards rely on task-specific safety evaluations that are often inflexible, requiring repeated assessments whenever task parameters change. This work proposes MonLog, a data-driven, probabilistic method to automatically derive safety curves (SCs) from recent injury protection data sets. By leveraging non-linear modeling techniques, our approach addresses the limitations of conventional linear SCs, which often result in overly conservative speed restrictions. We present a comprehensive test routine to validate our method, highlighting improvements in both compliance with safety constraints and operational efficiency. Our findings demonstrate that the proposed approach not only enhances safety but also optimizes robotic performance, making it suitable for a wide range of applications.

Robot manipulation researchers reference human performance as a goal for their work, however, human data is seldom present in robotics benchmarks. We introduce a real-world benchmark targeting manipulation skills for performing electrical circuit inspection with a multimeter using an Internet-connected electronic task board. We present timing study results and an exemplary robot solution across six different tasks from the Robothon Grand Challenge at the automatica conference in 2023. Contributions from 16 robot teams were collected using task boards we manufactured and distributed as part of the 30-day international competition as an initial performance database. Our work systematically highlights the skill gap between the winning robot solution and the best human performance from a group of 30 subjects. Our goal is to chronicle progress over time in robot manipulation skills and provide a standardized, physical benchmark across the global community. Videos of the team submissions, the exemplary robot solution, as well as the project reproduction code are provided in the included repository.

In this paper, we propose a closed-loop sensitivity-based approach to enhance the robustness of robotic non-prehensile dynamic manipulation tasks. The proposed method aims at fulfilling the transportation of an object, that is free to move on a tray-shaped robot end-effector, in face of not perfectly known nominal dynamic parameters. The approach is built up on taking the parameterized reference trajectory to be tracked as the optimization variable minimizing a norm of the task closed-loop sensitivity. The resulting optimal reference trajectory is inherently more robust to the parametric variations of object dynamic properties compared to a baseline trajectory execution. The tracking performance is assessed and validated along hardware experiments and an extensive simulation campaign assessing the superior robustness of our approach.

Ensuring the sterility of medical equipment, particularly endoscopes used in environments teeming with diverse pathogens and drug-resistant bacteria, is crucial for safe medical procedures. However, the complexity of endoscope reprocessing, which involves numerous dexterous manual manipulations, poses significant challenges. Achieving certification for sterilization requires precise, repetitive execution with strict tolerances. In this study, we propose a framework that automates the handling and storage of endoscopes right after the sterilization process and employs compliant collaborative robots to address these dexterous manipulation challenges. In the first stage, we identified the key manipulation skills involved in the process through observations and feedback from medical personnel. In the second stage, we proposed a system that employs a high-level action planner to orchestrate the removal and storage of endoscopes, integrating two collaborative robots and a linear unit. Through real-time force measurements, compliant control, task knowledge, and safety protocols, we establish a system that ensures the safety of both medical equipment and personnel in proximity. In our first experiment, we conducted 50 trials with a 100 % reliability rate. Each trial had an execution time of 102 seconds, with a variance of 1.2 seconds. In our second experiment, we performed 10 trials with a human obstructing the transfer path, facing away from the robot. In all cases, the system successfully and promptly detected the collision. This work pioneers the automation of medical reprocessing in sterile environments using tactile robots and addresses the associated challenges.

Assembly is a crucial skill for robots in both modern manufacturing and service robotics. However, mastering transferable insertion skills that can handle a variety of high-precision assembly tasks remains a significant challenge. This paper presents a novel framework that utilizes diffusion models to generate 6D wrench for high-precision tactile robotic insertion tasks. It learns from demonstrations performed on a single task and achieves a zero-shot transfer success rate of 95.7% across various novel high-precision tasks. Our method effectively inherits the self-adaptability demonstrated by our previous work. In this framework, we address the frequency misalignment between the diffusion policy and the real-time control loop with a dynamic system-based filter, significantly improving the task success rate by 9.15%. Furthermore, we provide a practical guideline regarding the trade-off between diffusion models' inference ability and speed.

This paper is about generating motion plans for high degree-of-freedom systems that account for both static and dynamic collisions along the entire body. A particular class of mathematical programs with complementarity constraints become useful in this regard. Optimization-based planners can tackle confined space trajectory planning while being cognizant of robot and (mostly static) obstacle constraints. However, handling moving obstacles is non-trivial in a real-time setting. To this end, we present the FLIQC (Fast LInear Quadratic Complementarity based) motion planner. Our reactive planner employs a novel motion model that captures the entire rigid robot as well as the obstacle geometry and ensures nonpenetration between the surfaces due to the imposed constraint. We perform thorough comparative studies with the state-of-the-art, which demonstrate improved performance. Extensive simulation and hardware experiments validate our claim of generating continuous and real-time motion plans at 1 kHz for modern collaborative robots with constant minimal parameters.

Optimal path planning requires finding a series of feasible states from the starting point to the goal to optimize objectives. Popular path planning algorithms, such as Effort Informed Trees (EIT*), employ effort heuristics to guide the search. Effective heuristics are accurate and computationally efficient, but achieving both can be challenging due to their conflicting nature. This paper proposes Direction Informed Trees (DIT*), a sampling-based planner that focuses on optimizing the search direction for each edge, resulting in goal bias during exploration. We define edges as generalized vectors and integrate similarity indexes to establish a directional filter that selects the nearest neighbors and estimates direction costs. The estimated direction cost heuristics are utilized in edge evaluation. This strategy allows the exploration to share directional information efficiently. DIT* convergence faster than existing single-query, sampling-based planners on tested problems in$\mathbb{R}^{4}$to$\mathbb{R}^{16}$and has been demonstrated in real-world environments with various planning tasks. A video showcasing our experimental results is available at: https://youtu.be/2SX6QT2NOek.

Laboratory automation is a key driver for higher efficiency and reproducibility of experiments and measurements in natural science laboratories. One process that is particularly susceptible to both manual errors in the physical handling of labware, faulty data analyses, and incomplete reporting is the quantitative Polymerase Chain Reaction (qPCR). It is a ubiquitous analysis method in biolaboratories to amplify and measure the amount of a specific DNA sequence in a sample. Our system, which we call the qPCRBot, addresses these issues through three key pillars: automating data analysis and handling processes, standardizing data management and system communication protocols, and utilizing a robotic manipulator for labware transport. To achieve this, we developed a SiLA 2-based client-server architecture for unified and standardized access to both the qPCR device and the robot. For the manipulator, we implemented a Cartesian motion generator to ensure proper labware transport. We transform all experiment data to a standardized, XML-based format and integrate a widely-used Laboratory Information Management System for its storage. These developments collectively enable streamlined qPCR measurements without human interaction, thus enhancing both efficiency and reproducibility.

Learning from Demonstration (LfD) techniques enable robots to learn and generalize tasks from user demonstrations, eliminating the need for coding expertise among end-users. One established technique to implement LfD in robots is to encode demonstrations in a stable Dynamical System (DS). However, finding a stable dynamical system entails solving an optimization problem with bilinear matrix inequality (BMI) constraints, a non-convex problem which, depending on the number of scalar constraints and variables, demands significant computational resources and is susceptible to numerical issues such as floating-point errors. To address these challenges, we propose a novel compositional approach that enhances the applicability and scalability of learning stable DSs with BMIs.

In an aging society, the need for rehabilitation treatment is expected to rise. As current healthcare systems have limited capacity and personnel, access to rehabilitation devices usable in households can help address the demand. A lower-limb rehabilitation robot designed for home use must be adaptable to accommodate acute and chronic rehabilitation phases. Existing devices are mechanically complex and require intricate, patient-specific adjustments. To address this, we propose a single degree of freedom (DoF) mechanism based on a chain drive that can be used in multiple configurations, inside and outside a patient bed. We model the gait pattern and construct a custom cost function that captures key features of natural human walking. This cost function is then used to optimize the design parameters of the robot via a direct-search solver to accommodate patients of varying sizes and achieve effective rehabilitation with a fixed trajectory. The outcome is validated experimentally by comparing two robot configurations with five healthy subjects.

This paper presents a novel approach for robot navigation in environments containing deformable obstacles. By integrating Learning from Demonstration (LfD) with Dynamical Systems (DS), we enable adaptive and efficient navigation in complex environments where obstacles consist of both soft and hard regions. We introduce a dynamic modulation matrix within the DS framework, allowing the system to distinguish between traversable soft regions and impassable hard areas in real-time, ensuring safe and flexible trajectory planning. We validate our method through extensive simulations and robot experiments, demonstrating its ability to navigate deformable environments. Additionally, the approach provides control over both trajectory and velocity when interacting with deformable objects, including at intersections, while maintaining adherence to the original DS trajectory and dynamically adapting to obstacles for smooth and reliable navigation.

In this work, we model and control a cable-driven, remote-actuated system that includes both friction and compliance in its dynamics. The control objective is to solve a regulation problem using a Model Predictive Controller (MPC). Unlike the flexible-joint robot models, which typically assume frictionless compliant elements, the proposed model incorporates friction forces between two compliant cable-sheaths that connect the motor to the driven link. Three controllers are developed based on the cascade control principles integrated with the MPC framework. Their performance is evaluated through both simulations and experiments on a custom-designed testbed. The results demonstrate that the MPC-Cascade control scheme achieves the best overall performance, with fast convergence and low control effort.

In this paper, we extend the Unified Force-Impedance Control (UFIC) framework for the whole-body control of service mobile robots subject to non-holonomic constraints. This enables the robot to execute complex service tasks that demand both force and impedance control within its whole body workspace. The task space of the robot is defined as the pose of both end-effectors. Following the concept of UFIC, both impedance and force tracking commands are applied in the task space of the whole-body controller, with augmented energy tanks incorporated to ensure passivity. To enable smooth transitions between force tracking and impedance control—particularly in cases of contact loss—a shaping function is used to modulate the force control command. Additionally, the robot’s redundancy is exploited to shape the posture, while satisfying joint limits, avoiding singularities, and preventing self-collisions between the arms. The effectiveness of the proposed whole-body UFIC controller is validated through simulations and real-world experiments with the service robot GARMI performing several daily tasks.

Contact estimation and force sensing are fundamental requirements for sensitive manipulation and safe physical human-robot interaction. The robot controllers that enable these functions rely on accurate and precise sensing. The performance of external force estimation is influenced by the design of the robot’s sensory system. And similar to how humans prefer specific arm configurations for performing precise and delicate tasks, e.g., drawing a thin, straight line, robots also have "sweet spots" that allow for the most accurate performance of tasks based on their sensing capabilities. To fully exploit a robot’s proprioceptive force sensing, it is essential to provide robot integrators, designers, and simulations with knowledge about these optimal settings including factors such as joint configurations, temperatures, and many more. This paper first investigates which of these factors are most relevant and how they can be best measured and based on that introduces force sensing error maps as a tool for structured research on robot force sensing performance and future developments of tactile robot applications. We first investigate the factors influencing the force sensing performance of 7-degree-of-freedom robots on the example of a Kinova Gen3 and then derive 2-dimensional Cartesian force sensing error maps for this robot, an LWR iiwa 14, and a Franka Emika robot. These maps enable comparison of robot sensing capabilities, revealing patterns and weak spots to guide application design toward more tactile areas.

Teleoperation with haptic feedback allows users to interact with remote environments while retaining a sense of touch. However, the stability and transparency of these systems are compromised under communication network delay. This paper presents an augmented Model-Mediated Teleoperation with 3D object and dynamic environment tracking (MMT-DET) by a vision-based algorithm, enabling users to receive haptic feedback in structured dynamic environments while maintaining robustness against network delays. A user study comparing the proposed method with teleoperation using the Time Domain Passivity Approach (TDPA) was conducted. The results demonstrate that our MMT-DET exhibits robustness to varying delays in task performance and outperforms TDPA at higher delay levels.

Robots with intrinsic joint elasticity can perform highly dynamic manoeuvres by leveraging energy storage and release, enabling explosive motions such as throwing. By augmenting elastic robots with clutch mechanisms, link decoupling can be used to fully exploit inertial coupling effects and gravitational acceleration in motion while effectively circumventing spring deflection limits. However, braking such systems in a decoupled state presents a challenge, as re-engaging the link risks damaging the joint. While optimal control strategies could be applied, they are not inherently safe due to model uncertainties. To address this, we propose a feedback-based two-stage method that coordinates the transition through the hybrid modes of the system. These modes are characterized by underactuated and actuated dynamics. First, a decoupled link is braked via inertial coupling until a safe velocity for clutching is reached, after which the link is re-coupled and actively braked. We demonstrate the effectiveness of this method through simulations comparing it with optimal control and validate it experimentally using a physical prototype.

This paper presents GARMI’s successful outdoor demonstration during the Kandahar Ski World Cup, where it performed trophy handovers in sub-zero temperatures. The event highlighted challenges in deploying robots in extreme conditions, including fluctuating temperatures and uneven terrain. GARMI achieved and completed the trophy handover during the live event, streamed to 60 million viewers. This experience raised two key research questions: the feasibility of high-precision robotics in harsh weather and strategies to compensate for environmental effects. To address them, we extended our previous framework to estimate the mass of the lifted trophy in real-time, incorporating IMU data and conducting experiments under varying temperatures and orientations. Experimental results showed that even slight variations in the robot’s base orientation had a significant impact on the accuracy of mass estimation. For instance, a 5° tilt in the robot’s base orientation resulted in a more than 100% increase in mass estimation error. Online mass estimation, performed using a quasi-static model, demonstrated improved accuracy when incorporating IMU-based corrections for base orientation. Additionally, temperature variations were found to affect robot control performance, with tracking errors increasing outside the manufacturer’s recommended temperature range. The findings highlighted the need for real-time corrections and compensations for base orientation and temperature in robot dynamics, ensuring safe human-robot interaction.

Robotic system development must adopt a holistic approach for tactile and dynamic tasks, shifting from the decoupled design of end-effectors and robot manipulators for traditional sequential tasks. Although established metrics exist for traditional tasks, such as pick-and-place, they lack the nuanced evaluation required for dynamic and tactile operations. Accordingly, this paper introduces an integrated framework that defines and unifies decoupled and coupled gripper metrics into a single perspective. We categorise gripper metrics based on their interaction with the robot manipulator, which can be entirely decoupled, coupled by time-sequence, or coupled. Using this classification, we propose 16 metrics to evaluate force control, force reaction, and efficiency. We introduce three new experimental setups and describe the corresponding procedures to quantify these metrics. Results from three commercial finger grippers demonstrate the efficacy of the proposed metrics, revealing each gripper’s strengths and limitations when integrated into different manipulator systems. Incorporating these metrics into performance reviews provides a comprehensive evaluation of robotic system fitness, considering dynamic, real-time challenges. This supports informed design choices and enhances tactile manipulation tasks.

This paper introduces Context Informed Trees (CIT*), a sampling-based motion planning algorithm that enhances exploration efficiency by biasing sampling based on uncertainty estimation from local samples and connectivity information obtained during the search process. CIT* is based on Flexible Informed Trees (FIT*) and incorporates three key components: region-based sampling, uncertainty-driven weighting, and connection-greedy prioritization (CGP). It generates regions from sampled states based on local obstacle proximity, assigning weights to these regions using probability uncertainty estimation via kernel density estimation (KDE) classification. To further refine the sampling focus, CGP prioritizes regions that exhibit strong connectivity in previous searches, ensuring that exploration is directed toward unknown and critical areas that have a higher likelihood of contributing to feasible and efficient paths. The sampling process is then guided by a mixture of Gaussian distributions centered on weighted regions, where the weighting biases sampling toward more critical regions, thereby improving search efficiency and accelerating convergence. Benchmark evaluations demonstrate that CIT* improves efficiency by reducing reliance on random sampling, which often leads to slower solution discovery and higher path costs. With biased sampling, CIT* maintains strong performance in solving complex motion planning problems in ${\mathbb{R}^4}$ to ${\mathbb{R}^{16}}$ and has been demonstrated on a real-world manipulation task. A video showcasing our method and experimental results is available at: https://youtu.be/SG2cy9WmjD0.

Recent studies underscore the importance of the patient’s active contribution and voluntary effort in enhancing therapy outcomes in physical rehabilitation. This paper presents an adaptive control scheme to implement active robotic rehabilitation. The primary goal is to dynamically regulate robotic assistance based on the patient’s performance and individual conditions, encouraging active participation, and effective therapy. To achieve this, a Lyapunov-based adaptive algorithm is developed that dynamically adjusts the admittance parameters by balancing the error and effort minimization. A novel performance index based on human energy input enables real-time identification of the intended human sharing role. This index is used as an adaptive rate in the proposed algorithm to enhance the control system’s dynamic responsiveness to changes in human performance. The proposed approach achieves two main rehabilitation objectives. First, it encourages active and safe human participation. Second, it enhances the therapy by providing personalized assistance, tailored to individual abilities and conditions, and thus reduces the need for therapist intervention. The performance of the proposed approach is illustrated in experimental studies. The results demonstrate the adaptability of the algorithm, ensuring compliant and safe interaction and effective task completion. Note to Practitioners—In a human-robot cooperation (HRC) framework, the automatic adaptation of the robot’s role as well as safe and stable interaction are crucial. These aspects are amplified in the context of robotic rehabilitation due to the special conditions of the human participants. Classic control methods, in shared control, lack system intelligence and automation in role allocation. However, the shared role of humans in HRC, particularly in rehabilitation applications, introduces real-time and unpredictable variations. This study addresses the shortcomings of classic control methods, by integrating intelligence into the control system through an adaptive Neural Network algorithm in shared autonomy. To emulate human-like adaptability, two crucial aspects are considered. Firstly, it incorporates safety assurance embedded in the adaptive algorithm via Lyapunov-based adaptation. Secondly, it detects the human’s role within the control loop through a novel energy-based performance index, which views the human as an active contributor to the system’s dynamic energy flow. This ensures robust behavior by dynamically adjusting the trade-off between task completion and minimal robot intervention. A standout feature of our algorithm lies in its expendability to exoskeleton systems, making it highly versatile for use in robotic rehabilitation and assistive technologies. The algorithm’s design allows for straightforward integration with exoskeletons, requiring only interaction force measurements in the joint space. It facilitates monitoring of a patient’s performance in each joint using the proposed performance index based on the human energy entry into the system. Beyond rehabilitation, the algorithm’s ability to adjust autonomy levels through adaptation makes it applicable to a wide range of Human-Robot Cooperation scenarios where automatic role allocation is necessary. Preliminary experiments underscore the adaptive algorithm’s robust responsiveness to changes in human performance. Future investigations should involve clinical experiments addressing real-life challenges associated with various movement deficiencies and responding to real-time issues that may arise during rehabilitation sessions.

This paper addresses motion replanning in human-robot collaborative scenarios, with an emphasis on reactivity and safety-compliant efficiency. While existing human-aware motion planners perform well in structured environments, they often struggle with unpredictable human behavior. This can result in safety measures that hinder the robot’s performance and overall throughput. This study combines reactive path replanning and a safety-aware cost function, enabling the robot to adapt its path to the changes in the scene in real-time. This solution reduces the execution time and trajectory slowdowns while ensuring safety. Simulations and real-world experiments show the method’s effectiveness compared to standard human-robot cooperation approaches, with efficiency enhancements of up to 60%.

Path planning in robotics often involves solving continuously valued, high-dimensional problems. Popular informed approaches include graph-based searches, such as A*, and sampling-based methods, such as Informed RRT*, which utilize informed set and anytime strategies to expedite path optimization incrementally. Informed sampling-based planners define informed sets as subsets of the problem domain based on the current best solution cost. However, when no solution is found, these planners re-sample and explore the entire configuration space, which is time-consuming and computationally expensive. This article introduces Multi-Informed Trees (MIT*), a novel planner that constructs estimated informed sets based on prior admissible solution costs before finding the initial solution, thereby accelerating the initial convergence rate. Moreover, MIT* employs an adaptive sampler that dynamically adjusts the sampling strategy based on the exploration process. Furthermore, MIT* utilizes length-related adaptive sparse collision checks to guide lazy reverse search. These features enhance path cost efficiency and computation times while ensuring high success rates in confined scenarios. Through a series of simulations and real-world experiments, it is confirmed that MIT* outperforms existing single-query, sampling-based planners for problems in$\mathbb {R}^{4}$to$\mathbb {R}^{16}$and has been successfully applied to real-world robot manipulation tasks. A video showcasing our experimental results is available at:https://youtu.be/30RsBIdexTUNote to Practitioners—The motivation for this work stems from the challenges faced by existing informed path planners in high-dimensional, continuously valued environments, particularly when an initial feasible solution is difficult to find. Traditional asymmetric bidirectional planners rely on the best current solution to define problem subsets. When a lazy path has been found through lazy reverse search, these planners tend to re-sample and explore the entire problem space, which could hinder the path planning process. Our proposed MIT* algorithm addresses this issue by constructing an estimated informed set based on prior admissible solution costs before finding the initial solution. This estimated set helps to narrow the search area, thereby accelerating the initial convergence rate. MIT* also integrates an adaptive sampling strategy that dynamically adjusts based on the ongoing exploration process, enhancing the planner’s ability to efficiently navigate through challenging spaces. Furthermore, MIT* employs adaptive sparse collision checks, which guide the lazy reverse search that balances computational efficiency with accuracy in pathfinding. The proposed algorithm can be applied to industrial robots, humanoid robots, or service robots to achieve efficient path planning.

Robots have surpassed humans in terms of strength and precision, yet humans retain an unparalleled ability for decision-making in the face of unpredictable disturbances. This article aims to combine the strengths of both entities within a singular task: human motion guidance under strict geometric constraints, particularly adhering to predetermined paths. To tackle this challenge, a modular haptic guidance law is proposed that takes the human-applied wrench as an input. Using an auxiliary variable called phase, the generated desired motion is guaranteed to consistently adhere to the constraint path. It is demonstrated how the guidance policy can be generalized into physically interpretable terms, adjustable either prior to initiating the task or dynamically while the task is in progress. Additionally, an illustrative guidance adaptation policy is showcased that takes into account the human's manipulability. Leveraging passivity analysis, potential sources of instability are pinpointed, and subsequently, overall system stability is ensured by incorporating an augmented virtual energy tank. Lastly, a comprehensive set of experiments, including a 20-participant user study, explores various aspects of the approach in practice, encompassing both technical and usability considerations.

In this work, we advance robotic grasping by incorporating wrist compliance in a unified hand-arm system inspired by human limb coordination. This integration improves grasping reliability and robustness through impedance and force learning in robotic arms. The compliant wrist system effectively compensates for uncertainties in object position and orientation. Employing a combined impedance-force control approach, we address diverse grasping and manipulation tasks in simulation. Successfully transferring the learned policy to a service humanoid mobile robot enables the seamless execution of grasping and opening tasks for various doors and handles without additional learning, using both fully actuated and underactuated robotic hands. Remarkably, our robust strategies yielded only one failure in 30 trials for the underactuated hand, even with up to 8 cm translation normal to the handle and$33^\circ$rotation errors, and no failures for the fully actuated one with up to 12 cm translation and$30^\circ$rotation. This significantly outperforms state-of-the-art end-to-end reinforcement learning approaches. Furthermore, we successfully tested and validated our approach across various constrained everyday tasks in different environments. Our proposed framework represents an advancement in the learning and execution of power grasping with compliant manipulation, achieving practically relevant performance.

Recent advances in control and planning allow for seamless physical human–robot interaction (pHRI). At the same time, novel challenges appear in orchestrating intelligent decision-making and ensuring safe control of robots. Particularly in scenarios involving unforeseen or unintended collisions, robots face the imperative of reacting judiciously to avert potential risks to humans, other robots, obstacles, or themselves. At the same time, they need to maintain focus on their primary task or be able to safely resume it. Collision detection and identification algorithms are now well established in industry, yet complex collision reflexes have not transitioned into industrial applications beyond basic stopping reactions. Despite the introduction of numerous advanced high-performance reflex controllers over the past decades, their real-world adoption has remained a challenge. This work establishes a systematic framework to address that gap. For this, thereflex control problemis defined,reflex behaviorsare systematically classified and categorized, and relevantsafety datais acquired followingexisting international standards. We argue that this foundational step is crucial for improving the safety and capabilities of robots in both complex industrial and domestic environments. We validate our approach within the system class of articulated manipulators through a state-of-the-art cooperative pick-and-place task, providing a blueprint for future implementations for other robot classes.

Controlling the shape of deformable linear objects using robots and constraints provided by environmental fixtures has diverse industrial applications. In order to establish robust contacts with these fixtures, accurate estimation of the contact state is essential for preventing and rectifying potential anomalies. However, this task is challenging due to the small sizes of fixtures, the requirement for real-time performances, and the infinite degrees of freedom of the deformable linear objects. In this paper, we propose a real-time approach for estimating both contact establishment and subsequent changes by leveraging the dependency between the applied and detected contact force on the deformable linear objects. We seamlessly integrate this method into the robot control loop and achieve an adaptive shape control framework which avoids, detects and corrects anomalies automatically. Real-world experiments validate the robustness and effectiveness of our contact estimation approach across various scenarios, significantly increasing the success rate of shape control processes.

This work focuses on the agile transportation of liquids with robotic manipulators. In contrast to existing methods that are either computationally heavy, system/container specific or dependant on a singularity-prone pendulum model, we present a real-time slosh-free tracking technique. This method solely requires the reference trajectory and the robot’s kinematic constraints to output kinematically feasible joint space commands. The crucial element underlying this approach consists on mimicking the end-effector’s motion through a virtual quadrotor, which is inherently slosh-free and differentially flat, thereby allowing us to calculate a slosh-free reference orientation. Through the utilization of a cascaded proportional-derivative (PD) controller, this slosh-free reference is transformed into task space acceleration commands, which, following the resolution of a Quadratic Program (QP) based on Resolved Acceleration Control (RAC), are translated into a feasible joint configuration. The validity of the proposed approach is demonstrated by simulated and real-world experiments on a 7 DoF Franka Emika Panda robot.

Time-effective and accurate source localization with mobile robots is crucial in safety-critical scenarios, e.g. leakage detection. This becomes particular challenging in realistic cluttered scenarios, i.e. in the presence of complex current flows or wind. Traditional methods often fall short due to simplifications or limited onboard resources.We propose to combine source localization with a Gaussian Markov Random Field (GMRF). This allows to improve source localization hypotheses by building on the GMRF’s concentration and flow field belief that are continuously updated by gathered measurements. We introduce the upstream source proximity (USP) as a natural metric that exploits the joint knowledge represented in the field belief’s concentration and flow field, i.e. predicting sources upstream. As a result, our method yields a computationally efficient source localization and field belief module providing substantially more stable gradients than conventional concentration gradient-based methods.We demonstrate the suitability of our approach in a series of numerical experiments covering complex source location scenarios. With regard to computational requirements, the method achieves update rates of 10Hz on a RaspberryPi4B.

A drawing robot avatar is a robotic system that allows for telepresence-based drawing, enabling users to remotely control a robotic arm and create drawings in real-time from a remote location. The proposed control framework aims to improve bimanual robot telepresence quality by reducing the user workload and required prior knowledge through the automation of secondary or auxiliary tasks. The introduced novel method calculates the near-optimal Cartesian end-effector pose in terms of visual feedback quality for the attached eye-to-hand camera with motion constraints in consideration. The effectiveness is demonstrated by conducting user studies of drawing reference shapes using the implemented robot avatar compared to stationary and teleoperated camera pose conditions. Our results demonstrate that the proposed control framework offers improved visual feedback quality and drawing performance.

This paper investigates the problem of effective tool manipulation for motion planning in complex human-like scenarios. Vector-field-based real-time strategies, although widely used, usually do not account for unwieldy tools or incorporate systematic methods to handle these extra maneuvers needed. Instead, we formalize the problem and propose a novel field-based reactive planner that explicitly accounts for rotational forces for seamless maneuvers based on the tool’s geometry and featured points. Furthermore, we capture and encode robot performance through capability metrics and improve the same using an additional quality distribution method. This enables seamless integration of the robot’s embodiment with the reactive force-torque (wrench) field giving rise to flexible tool usage in non-stationary environments. Extensive simulation analysis on a 7 DoF collaborative robot manipulating a common tool in an unorganized table-top layout reinforces our claim of robustness in stationary and non-stationary scenarios.

This paper addresses the migration of high-frequency robot controllers to remote computing services, which are connected via a communication channel prone to delays and packet loss. The stability of the networked system is guaranteed by ensuring passivity of each subcomponent in the interconnection, as well as the Time-Domain-Passivity-Approach (TDPA) for the communication channel. We reduce conservatism of the TDPA using the model knowledge on both sides of the communication system to identify passivity excesses. This is further used to avoid over-dissipation of energy in the passivity controller by augmentation of a tolerable passivity-shortage. Tracking offsets are eliminated with a position drift compensation algorithm, for which convergence guarantees are provided. The experimental validation of the results conducted on a 7-DoF Franka Research 3 robot demonstrates a substantial enhancement in tracking performance due to the proposed modifications, particularly in scenarios with high communication delays.

Flexible manufacturing lines are required to meet the demand for customized and small batch-size products. Even though state-of-the-art tactile robots may provide the versatility for increased adaptability and flexibility, their potential is yet to be fully exploited. To support robotics deployment in manufacturing, we propose a task-based tactile robot programming paradigm that uses an object-centric tactile skill definition that directly links identified object constraints of the task to the definition of constraint-based unified force-impedance control. In this study, we first explain the basic concept of abstracting the task constraints experienced by the object and transferring them to the robot’s operational space frame. Second, using the object-centric tactile skill definition, we synthesize unified force-impedance control and formalized holonomic constraints to enable flexible task execution. Later, we propose the quantified analysis metrics for the process by analyzing them as a typical example of flexible manipulation disassembly skills, e.g., levering and unscrew-driving regarding their object requirements. Supported by realistic experimental evaluation using a Franka Emika robot, our tactile robot programming approach for the direct translation between task-level constraints and robot control parameter design is shown to be a viable solution for increased robotic deployment in flexible manufacturing lines.

The use of pointed or edged tools or objects is one of the most challenging aspects of today’s application of physical human-robot interaction (pHRI). One reason for this is that the severity of harm caused by such edged or pointed impactors is less well studied than for blunt impactors. Consequently, the standards specify well-reasoned force and pressure thresholds for blunt impactors and advise avoiding any edges and corners in contacts. Nevertheless, pointed or edged impactor geometries cannot be completely ruled out in real pHRI applications. For example, to allow edged or pointed tools such as screwdrivers near human operators, the knowledge of injury severity needs to be extended so that robot integrators can perform well-reasoned, time-efficient risk assessments. In this paper, we provide the initial datasets on injury prevention for the human hand based on drop tests with surrogates for the human hand, namely pig claws and chicken drumsticks. We then demonstrate the ease and efficiency of robot use using the dataset for contact on two examples. Finally, our experiments provide a set of injuries that may also be expected for human subjects under certain robot mass-velocity constellations in collisions. To extend this work, testing on human samples and a collaborative effort from research institutes worldwide is needed to create a comprehensive human injury avoidance database for any pHRI scenario and thus for safe pHRI applications including edged and pointed geometries.

Environmental monitoring via UAVs offers unprecedented aerial observation capabilities. However, the limited flight durations of typical multirotors and the demands on human attention in outdoor missions call for more autonomous solutions. Addressing the specific challenges of precise UAV landings – especially amidst wind disturbances, obstacles, and unreliable global localization – we introduce a mobile hub concept. This hub facilitates continuous mission cycling for unmodified off-the-shelf UAVs. Our approach centers on a small landing platform affixed to a robotic arm, adeptly correcting UAV pose errors in windy conditions. Compact enough for installation in an economy car, the system emphasizes two novel strategies. Firstly, external visual tracking of the UAV informs the landing controls for both the drone and the robotic arm. The arm compensates for UAV positioning errors and aligns the platform’s attitude with the UAV for stable landings, even on small platforms under windy conditions. Secondly, the robotic arm can transport the UAV inside the hub, perform maintenance tasks like battery replacements, and then facilitate direct relaunches. Importantly, our design places all operational responsibility on the hub, ensuring the UAV remains unaltered. This ensures broad compatibility with standard UAVs, only necessitating an API for attitude setpoints. Experimental results underscore the efficiency of our model, achieving safe landings with minimal errors (≤ 7 cm) in winds up to 5 Beaufort (8.1 m/s). In essence, our mobile hub concept significantly boosts UAV mission availability, allowing for autonomous operations even under challenging conditions.

Intrinsically elastic robots surpass their rigid counterparts in a range of different characteristics. By temporarily storing potential energy and subsequently converting it to kinetic energy, elastic robots are capable of highly dynamic motions even with limited motor power. However, the time-dependency of this energy storage and release mechanism remains one of the major challenges in controlling elastic robots. A possible remedy is the introduction of locking elements (i.e. clutches and brakes) in the drive train. This gives rise to a new class of robots, so-called clutched-elastic robots (CER), with which it is possible to precisely control the energy-transfer timing. A prevalent challenge in the realm of CERs is the automatic discovery of clutch sequences. Due to complexity, many methods still rely on pre-defined modes. In this paper, we introduce a novel contact-implicit scheme designed to optimize both control input and clutch sequence simultaneously. A penalty in the objective function ensures the prevention of unnecessary clutch transitions. We empirically demonstrate the effectiveness of our proposed method on a double pendulum equipped with two of our newly proposed clutch-based Bi-Stiffness Actuators (BSA).

Bowden cables serve as essential components in various mechanical systems, facilitating power transmission from remote actuators to specific destinations. The pretension of Bowden cables profoundly influences system performance, notably in terms of friction. This study investigates the effects of cable pretension and shape on friction and torque efficiency. A custom self-designed testbed, comprising integrated actuator units, pulleys, and a novel pretension mechanism connected by Bowden cables, is utilized to conduct experimental tests under varying parameters. This work adopts an integrated approach of experimentation, modeling, and validation, offering preliminary insights into the torque transmission characteristics of tendon driven actuator systems. Additionally, the precise model exhibits excellent conformity across a broad range of shapes and provides initial insights into hysteresis modeling attributable to cable material properties.

Established techniques that enable robots to learn from demonstrations are based on learning a stable dynamical system (DS). To increase the robots’ resilience to perturbations during tasks that involve static obstacle avoidance, we propose incorporating barrier certificates into an optimization problem to learn a stable and barrier-certified DS. Such optimization problem can be very complex or extremely conservative when the traditional linear parameter-varying formulation is used. Thus, different from previous approaches in the literature, we propose to use polynomial representations for DSs, which yields an optimization problem that can be tackled by sum-of-squares techniques. Finally, our approach can handle obstacle shapes that fall outside the scope of assumptions typically found in the literature concerning obstacle avoidance within the DS learning framework. Supplementary material can be found at the project webpage: https://martinschonger.github.io/abc-ds

In the field of Learning from Demonstration (LfD), Dynamical Systems (DSs) have gained significant attention due to their ability to generate real-time motions and reach predefined targets. However, the conventional convergence-centric behavior exhibited by DSs may fall short in safety-critical tasks, specifically, those requiring precise replication of demonstrated trajectories or strict adherence to constrained regions even in the presence of perturbations or human intervention. Moreover, existing DS research often assumes demonstrations solely in Euclidean space, overlooking the crucial aspect of orientation in various applications. To alleviate these shortcomings, we present an innovative approach geared toward ensuring the safe execution of learned orientation skills within constrained regions surrounding a reference trajectory. This involves learning a stable DS on SO(3), extracting time-varying conic constraints from the variability observed in expert demonstrations, and bounding the evolution of the DS with Conic Control Barrier Function (CCBF) to fulfill the constraints. We validated our approach through extensive evaluation in simulation and showcased its effectiveness for a cutting skill in the context of assisted teleoperation.

Integrating humanoid service mobile robots into human environments presents numerous challenges, primarily concerning the safety of interactions between robots and humans. To address these safety concerns, we propose a novel approach that leverages the capabilities of digital twin technology by tailoring it to incorporate comprehensive and robust safety concepts. This paper introduces a "safe-by-design" digital twin that operates alongside the real twin robot in the loop, engaging real-time safety framework during physical interactions with the surrounding environment, including humans.To validate the effectiveness of our proposed safe-by-design digital twin framework, we conducted experiments using a humanoid service mobile robot alongside simulated human counterparts. Our results demonstrate the capability of the integrated impact safety module within the proposed digital twin approach to limit the velocities of both the robot’s base and arms, adhering to injury biomechanics-based safety thresholds. These findings emphasize the promise of our proposed approach for ensuring the physical safety of humanoid service mobile robots operating in dynamic human environments. It enables the digital twin to preemptively identify potential safety hazards and formulate safe intervention actions to ensure the robot’s compliance with safety regulations, paving the way for safer and more widespread adoption of robotic systems in various service domains.

The basis for robotics skill learning is an adequate representation of manipulation tasks based on their physical properties. As manipulation tasks are inherently invariant to the choice of reference frame, an ideal task representation would also exhibit this property. Nevertheless, most robotic learning approaches use unprocessed, coordinate-dependent robot state data for learning new skills, thus inducing challenges regarding the interpretability and transferability of the learned models.In this paper, we propose a transformation from spatial measurements to a coordinate-invariant feature space, based on the pairwise inner product of the input measurements. We describe and mathematically deduce the concept, establish the task fingerprints as an intuitive image-based representation, experimentally collect task fingerprints, and demonstrate the usage of the representation for task classification. This representation motivates further research on data-efficient and transferable learning methods for online manipulation task classification and task-level perception.

This study presents an innovative test rig engineered to explore the kinematic and viscoelastic characteristics of human specimen hands. The rig features eight force-controlled motors linked to muscle tendons, enabling precise stimulation of hand specimens. Hand movements are monitored through an optical tracking system, while a force-torque sensor quantifies the resultant fingertip loads. Employing this setup, we successfully demonstrated a pincer grip using a cadaver hand and measured both muscle forces and grip strength. Our results reveal a nonlinear relationship between tendon forces and grip strength, which can be modeled by an exponential fit. This investigation serves as a nexus between biomechanical and robotics-focused research, providing critical insights for the advancement of robotic hand actuation and therapeutic interventions.

In robotics and biomechanics, accurately determining joint parameters and computing the corresponding forward and inverse kinematics are critical yet often challenging tasks, especially when dealing with highly individualized and partly unknown systems. This paper unveils a cutting-edge kinematic optimizer, underpinned by an autoencoder-based architecture, to address these challenges. Utilizing a neural network, our approach simulates inverse kinematics, converting measurement data into joint-specific parameters during encoding, enabling a stable optimization process. These parameters are subsequently processed through a predefined, differentiable forward kinematics model, resulting in a decoded representation of the original data. Beyond offering a comprehensive solution to kinematics challenges, our method also unveils previously unidentified joint parameters. Real experimental data from knee and hand joints validate the optimizer’s efficacy. Additionally, our optimizer is multifunctional: it streamlines the modeling and automation of kinematics and enables a nuanced evaluation of diverse modeling techniques. By assessing the differences in reconstruction losses, we illuminate the merits of each approach. Collectively, this preliminary study signifies advancements in kinematic optimization, with potential applications spanning both biomechanics and robotics.

Insertion is an essential skill for robots in both modern manufacturing and services robotics. In our previous study, we proposed an insertion skill framework based on forcedomain wiggle motion. The main limitation of this method lies in the robot’s inability to adjust its behavior according to changing contact state during interaction. In this paper, we extend the skill formalism by incorporating a behavior tree-based primitive switching mechanism that leverages highfrequency tactile data for the estimation of contact state. The efficacy of our proposed framework is validated with a series of experiments that involve the execution of tightly constrained peg-in-hole tasks. The experiment results demonstrate a significant improvement in performance, characterized by reduced execution time, heightened robustness, and superior adaptability when confronted with unknown tasks. Moreover, in the context of transfer learning, our paper provides empirical evidence indicating that the proposed skill framework contributes to enhanced transferability across distinct operational contexts and tasks.

This study addresses the absence of an identification framework to quantify a comprehensive dynamic model of human and anthropomorphic tendon-driven fingers, which is necessary to investigate the physiological properties of human fingers and improve the control of robotic hands. First, a generalized dynamic model was formulated, which takes into account the inherent properties of such a mechanical system. This includes rigid-body dynamics, coupling matrix, joint viscoelasticity, and tendon friction. Then, we propose a methodology comprising a series of experiments, for step-wise identification and validation of this dynamic model. Moreover, an experimental setup was designed and constructed that features actuation modules and peripheral sensors to facilitate the identification process. To verify the proposed methodology, a 3D-printed robotic finger based on the index finger design of the Dexmart hand was developed, and the proposed experiments were executed to identify and validate its dynamic model. This study could be extended to explore the identification of cadaver hands, aiming for a consistent dataset from a single cadaver specimen to improve the development of musculoskeletal hand models.

Signed Distance Fields are a common surface representation method widely used for both 3D mapping and obstacle avoidance. While the former traditionally uses projective Truncated Signed Distance Fields (TSDF), the latter often requires a complete Euclidean Signed Distance Field (ESDF) representation of the environment. In this paper, we propose a unified system by combining both methods to generate force vectors to nearby obstacles from a TSDF-based 3D reconstruction. We introduce a new merging scheme to better capture the geometry of the object, with no post-processing requirements, and a way to increase the effective range of the system. Validation experiments demonstrate the accuracy of the force vector calculation by comparing it against an ideal simulated environment. The flexibility of the system is demonstrated by implementing a haptic feedback teleoperation setup, which is validated through a user study in a teleoperation task. Through this, it is shown that the proposed method provides a statistically significant improvement to the task. Finally, a brief description on future improvements to the system is presented.

This paper introduces a comprehensive user-guided planning framework designed for robots operating in dynamic, human-centered environments – where the ability to execute sequential tasks flexibly and adaptively is paramount. Our planner enables robots to (i) encode object-centric constraints and user preferences via multiple demonstrations, (ii) transfer geometric features and implicit relaxations to novel scenarios while reacting to unforeseen events, and (iii) adapt to changing task conditions in real-time, including the real-time replanning and tracking of moving targets. Our approach relies on C1screw linear interpolation, which generates smooth paths satisfying the underlying geometric constraints that characterize the task. The prescribed path is combined with a hierarchical quadratic programming-based controller which explores the user demonstrations's stochastic variability to relax task constraints while ensuring real-time whole-body collision avoidance. Our framework continuously checks for dynamic changes in task targets, ensuring appropriate planning or control actions, and tending to the prescribed screw path. This comprehensive approach is deployed in different task conditions which are available at https://youtu.be/F0cMr1n1D9k.

Non-prehensile transportation of unstable objects presents a challenging task in robotics. To ensure the success of the transportation, it is necessary to consider both the object’s stability via contact dynamics and the motion constraints of the robot. We propose two novel trajectory planning methods derived from sampling and dynamic programming algorithms, tested on a 7-DoF Franka Emika robot against common strategies like Model Predictive Control (MPC) and S-curve planning, particularly under the constraint of a non-rotating tray. The results demonstrate the effectiveness of our methodologies in improving transportation speed. This research contributes to advancements in robotic manipulation techniques by tackling non-prehensile manipulation of dynamically unstable objects.

Markerless motion capture devices such as the Leap Motion Controller (LMC) have been extensively used for tracking hand, wrist, and forearm positions as an alternative to Marker-based Motion Capture (MMC). However, previous studies have highlighted the subpar performance of LMC in reliably recording hand kinematics. In this study, we employ four LMC devices to optimize their collective tracking volume, aiming to enhance the accuracy and precision of hand kinematics. Through Monte Carlo simulation, we determine an optimized layout for the four LMC devices and subsequently conduct reliability and validity experiments encompassing 1560 trials across ten subjects. The combined tracking volume is validated against an MMC system, particularly for kinematic movements involving wrist, index, and thumb flexion. Utilizing calculation resources in one computer, our result of the optimized configuration has a better visibility rate with a value of 0.05 ± 0.55 compared to the initial configuration with -0.07 ± 0.40. Multiple Leap Motion Controllers (LMCs) have proven to increase the interaction space of capture volume but are still unable to give agreeable measurements from dynamic movement.

In this work, an upgraded exoskeleton design is presented with enhanced trajectory tracking and mechanical transparency. Compared to the first version, the design features a 3-DoF actuated shoulder joint and a mechanism to regulate the pretension of Bowden cables. Force/torque sensors are installed to directly measure the interaction forces between the human arm and the exoskeleton at the connecting points. Three control strategies were evaluated to follow a desired trajectory; A PD controller, a PD controller with friction observer, and an adaptive controller based on Radial Basis Function (RBF). These strategies also form the basis for an admittance control, aimed at improving the exoskeleton’s mechanical transparency during interaction with the human arm. Simulations and experimental results demonstrate that the PD control, supported by friction estimation via a momentum observer, achieves superior tracking performance. Moreover, the system’s mechanical transparency is enhanced using the admittance RBF-based controller, showing marginally superior results.

Recent advancements in artificial hand development have primarily concentrated on enhancing adaptive grasping, dexterity, as well as the integration of biomimetic skin. However, few designs have successfully combined lightweight, cost-effective solutions, and tactile sensing along with adaptive grasping in a human-sized prototype. We propose, an open-source, highly integrated artificial hand. It leverages a compliant linkage mechanism for versatile grasping capabilities, featuring six degrees of actuation and MEMS-based tactile sensors on every fingertip.

This study addresses the critical need for diverse and comprehensive data focused on human arm joint torques while performing activities of daily living (ADL). Previous studies have often overlooked the influence of objects on joint torques during ADL, resulting in limited datasets for analysis. To address this gap, we propose an Object Augmentation Algorithm (OAA) capable of augmenting existing marker-based databases with virtual object motions and object-induced joint torque estimations. The OAA consists of five phases: (1) computing hand coordinate systems from optical markers, (2) characterising object movements with virtual markers, (3) calculating object motions through inverse kinematics (IK), (4) determining the wrench necessary for prescribed object motion using inverse dynamics (ID), and (5) computing joint torques resulting from object manipulation. The algorithm’s accuracy is validated through trajectory tracking and torque analysis on a 5+4 degree of freedom (DoF) robotic hand-arm system, manipulating three unique objects. The results show that the OAA can accurately and precisely estimate 6 DoF object motion and object-induced joint torques. Correlations between computed and measured quantities were > 0.99 for object trajectories and > 0.93 for joint torques. The OAA was further shown to be robust to variations in the number and placement of input markers, which are expected between databases. Differences between repeated experiments were minor but significant (p < 0.05). The algorithm expands the scope of available data and facilitates more comprehensive analyses of human-object interaction dynamics.

Prosthetic limb abandonment remains an unsolved challenge as amputees consistently reject their devices. Current prosthetic designs often fail to balance human-like performance with acceptable device weight, highlighting the need for optimised designs tailored to modern tasks. This study aims to provide a comprehensive dataset of joint kinematics and kinetics essential for performing activities of daily living (ADL), thereby informing the design of more functional and user-friendly prosthetic devices. Functionally required Ranges of Motion (ROM), velocities, and torques for the Glenohumeral (rotation), elbow, Radioulnar, and wrist joints were computed using motion capture data from 12 subjects performing 24 ADLs. Our approach included the computation of joint torques for varying mass and inertia properties of the upper limb, while torques induced by the manipulation of experimental objects were considered by their interaction wrench with the subject’s hand. Joint torques pertaining to individual ADL scaled linearly with limb and object mass and mass distribution, permitting their generalisation to not explicitly simulated limb and object dynamics with linear regressors (LRM), exhibiting coefficients of determination R = 0.99 ± 0.01. Exemplifying an application of data-driven prosthesis design, we optimise wrist axes orientations for two serial and two differential joint configurations. Optimised axes reduced peak power requirements, compared to anatomical configurations, by exploiting high torque correlations (r = −0.84, p < 0.05) between Ulnar deviation and wrist flexion/extension joints. This study offers critical insights into the functional requirements of upper limb prostheses, providing a valuable foundation for data-driven prosthetic design that addresses key user concerns and enhances device adoption.

Despite recent advancements in torque-controlled tactile robots, integrating them into manufacturing settings remains challenging, particularly in complex environments. Simplifying robotic skill programming for non-experts is crucial for increasing robot deployment in manufacturing. This work proposes an innovative approach, Vision-Augmented Unified Force-Impedance Control (VA-UFIC), aimed at intuitive visuo-tactile exploration of unknown 3D curvatures. VA-UFIC stands out by seamlessly integrating vision and tactile data, enabling the exploration of diverse contact shapes in three dimensions, including point contacts, flat contacts with concave and convex curvatures, and scenarios involving contact loss. A pivotal component of our method is a robust online contact alignment monitoring system that considers tactile error, local surface curvature, and orientation, facilitating adaptive adjustments of robot stiffness and force regulation during exploration. We introduce virtual energy tanks within the control framework to ensure safety and stability, effectively addressing inherent safety concerns in visuo-tactile exploration. Evaluation using a Franka Emika research robot demonstrates the efficacy of VA-UFIC in exploring unknown 3D curvatures while adhering to arbitrarily defined force-motion policies. By seamlessly integrating vision and tactile sensing, VA-UFIC offers a promising avenue for intuitive exploration of complex environments, with potential applications spanning manufacturing, inspection, and beyond.

Safety for physical human-robot interaction (pHRI) is a major concern for all application domains. While current standardization for industrial robot applications provide safety constraints that address the onset of pain in blunt impacts, these impact thresholds are difficult to use on edged or pointed impactors. The most severe injuries occur in constrained contact scenarios, where crushing is possible. Nevertheless, situations potentially resulting in constrained contact only occur in certain areas of a workspace and design or organisational approaches can be used to avoid them. What remains are risks to the human physical integrity caused by unconstrained accidental contacts, which are difficult to avoid while maintaining robot motion efficiency. Nevertheless, the probability and severity of injuries occurring with edged or pointed impacting objects in unconstrained collisions is hardly researched. In this paper, we propose an experimental setup and procedure using two pendulums modeling human hands and arms and robots to understand the injury potential of unconstrained collisions of human hands with edged objects. Pig feet are used as ex vivo surrogate samples - as these closely resemble the physiological characteristics of human hands - to create an initial injury database on the severity of injuries caused by unconstrained edged or pointed impacts. For the effective mass range of typical lightweight robots, the data obtained show low probabilities of injuries such as skin cuts or bone/tendon injuries in unconstrained collisions when the velocity is reduced to < 0.5 m/s. Additionally, distinct differences between injury probability of the finger substitutes and the back of the hand substitutes are observed. The proposed experimental setups and procedures should be complemented by sufficient human modeling, e.g. the effective masses of human body parts, and will eventually lead to a complete understanding of the biomechanical injury potential in pHRI.

Mobile manipulators (MM) have proven valuable in assisting humans in industrial settings. However, their strict separation from humans in controlled environments limits their effectiveness. Efforts have been made to bridge this gap for physical human-robot interaction (pHRI), leading to the development of collaborative mobile manipulators (CMM). Nonetheless, unpredictable environments continue to present challenges. This paper introduces an innovative suspension design for mobile bases (MBs) to enhance the safety and autonomy of CMMs. We propose an electromechanical approach leveraging variable stiffness and combining passive springs with adaptive transmission mechanisms. Through simulation, physical prototype development, and experimental validation, we demonstrate the effectiveness of our approach in stabilizing the MB against external disturbances. Our findings provide valuable insights for the development of CMMs in dynamic environments.

A typical manipulation task consists of a manipulator equipped with a gripper to grasp and move an object with constraints on the motion of the hand-held object, which may be due to the nature of the task itself or from object-environment contacts. In this paper, we study the problem of computing joint torques and grasping forces for time-optimal motion of an object, while ensuring that the grasp is not lost and any constraints on the motion of the object, either due to dynamics, environment contact, or no-slip requirements, are also satisfied. We present a second-order cone program (SOCP) formulation of the time-optimal trajectory planning problem that considers nonlinear friction cone constraints at the hand-object and object-environment contacts. Since SOCPs are convex optimization problems that can be solved optimally in polynomial time using interior point methods, we can solve the trajectory optimization problem efficiently. We present simulation results on three examples, including a non-prehensile manipulation task, which shows the generality and effectiveness of our approach.

Industrial robots have increased payload, repeatability, and reach compared to collaborative robots, however, they have a fixed position controller and low intrinsic admittance. This makes realizing safe contact challenging due to large contact force overshoots in contact transitions and contact instability when the environment and robot dynamics are coupled. To improve safe contact on industrial robots, we propose an admittance controller with inverse model compensation, designed and implemented outside the position controller. By including both the inner loop and outer loop dynamics in its design, the proposed method achieves expanded admittance in terms of increasing both gain and cutoff frequency of the desired admittance. Results from theoretical analyses and experiments on a commercial industrial robot show that the proposed method improves rendering of the desired admittance while maintaining contact stability. We further validate this by conducting actual assembly tasks of plug insertion with fine positioning, switch insertion onto the rail, and colliding the robot end effector with random objects and surfaces, as seen at https://youtu.be/8XfkdHEdWDs.

The intersection of robotics and artificial intelligence led to a profound paradigm shift in Robot Learning. Robots have the capacity to replicate human actions and also dynamically adapt, innovate, and excel across a spectrum of tasks. However, the heterogeneity in the deployment of robot platforms and software frameworks poses considerable challenges in terms of systematic testing and comparative analyses. Additionally, the data scarcity of especially force controlled robot manipulation is still restraining the development of advanced foundation models. A reference platform with default software stack can help to increase comparability, reducing development time and collect a large amount of tactile robot manipulation data. To address on this problem, we developed a Parallel and Distributed Robot AI (PD.RAI) framework, comprising a scalable ensemble of Robot Learning Units (RLUs), a global database, and the Robot Cluster Intelligence (RoCI). Each RLU is endowed with robot arms, cameras, and local computational units to autonomously engage in planning, control, and local machine learning of tactile manipulation skills. The RoCI system oversees the learning process and schedules the RLUs tasks. To show the functionality of the system, two black-box optimization algorithms are compared within the robot skill learning domain. An experiment with 24 different optimization tasks is conducted in parallel. The algorithms are incorporated into the same existing default modules acting as a reference environment. This allows for a realistic comparison without sacrificing diversity of possible configurations and testing environments.

Manipulability analysis is a methodology employed to assess the capacity of an articulated system, at a specific configuration, to produce motion or exert force in diverse directions. The conventional method entails generating a virtual ellipsoid using the system’s configuration and model. Yet, this approach poses challenges when applied to systems such as the human body, where direct access to such information is limited, necessitating reliance on estimations. Any inaccuracies in these estimations can distort the ellipsoid’s configuration, potentially compromising the accuracy of the manipulability assessment. To address this issue, this article extends the standard approach by introducing the concept of the manipulability pseudo-ellipsoid. Through a series of theoretical analyses, simulations, and experiments, the article demonstrates that the proposed method exhibits reduced sensitivity to noise in sensory information, consequently enhancing the robustness of the approach.

This study presents the development and validation of an innovative hand exoskeleton designed for the re-habilitation of patients with Complex Regional Pain Syndrome (CRPS), a condition frequently arising post-injury or surgeries. The prototype is tailored for the hand, a region commonly affected by CRPS, and is notable for its adaptability and a comprehensive sensor system for monitoring individual joint movements. Reliable sensor performance was defined through precise force measurements and stability over time, showing minimal drift. These features enable personalized rehabilitation and objective progress tracking, addressing limitations in traditional physiotherapy such as availability, cost, and time constraints. The contributions of this work lie in its innovative design and the potential for robotic systems to improve therapeutic outcomes in CRPS rehabilitation.

Performing human-like motion is crucial for service humanoid robots. Real-time motion retargeting allows clear observation of the robot’s pose and provides instant feedback during human demonstrator actions. This paper presents an optimization-based real-time anthropomorphic motion retargeting framework for transferring human arm motion to a robot arm. The framework is generic, applicable to both spherical-rotational-spherical (SRS) and non-SRS robot arms. We introduce the normalized normal vector of the arm plane as an anthropomorphic criterion within our framework. The method is validated on a service humanoid robot, with both static and dynamic evaluations. The statistical analysis show that our method maintains strong anthropomorphic features while ensuring accurate wrist pose tracking.

In path planning, anytime almost-surely asymptotically optimal planners dominate the benchmark of sampling-based planners. A notable example is Batch Informed Trees (BIT*), where planners iteratively determine paths to batches of vertices within the exploration area. However, utilizing a consistent batch size is inefficient for initial pathfinding and optimal performance, it relies on effective task allocation. This paper introduces Flexible Informed Trees (FIT*), a sampling-based planner that integrates an adaptive batch-size method to enhance the initial path convergence rate. FIT* employs a flexible approach in adjusting batch sizes dynamically based on the inherent dimension of the configuration spaces and the hypervolume of the n-dimensional hyperellipsoid. By applying dense and sparse sampling strategy, FIT* improves convergence rate while finding successful solutions faster with lower initial solution cost. This method enhances the planner’s ability to handle confined, narrow spaces in the initial finding phase and increases batch vertices sampling frequency in the optimization phase. FIT* outperforms existing single-query, sampling-based planners on the tested problems in R2to R8, and was demonstrated on a real-world mobile manipulation task.

Path planning has long been an important and active research area in robotics. To address challenges in high-dimensional motion planning, this study introduces the Force Direction Informed Trees (FDIT*), a sampling-based planner designed to enhance speed and cost-effectiveness in pathfinding. FDIT* builds upon the state-of-the-art informed sampling planner, the Effort Informed Trees (EIT*), by capitalizing on often-overlooked information in invalid vertices. It incorporates principles of physical force, particularly Coulomb’s law. This approach proposes the elliptical k-nearest neighbors search method, enabling fast convergence navigation and avoiding high solution cost or infeasible paths by exploring more problem-specific search-worthy areas. It demonstrates benefits in search efficiency and cost reduction, particularly in confined, high-dimensional environments. It can be viewed as an extension of nearest neighbors search techniques. Fusing invalid vertex data with physical dynamics facilitates force-direction-based search regions, resulting in an improved convergence rate to the optimum. FDIT* outperforms existing single-query, sampling-based planners on the tested problems in ℝ4to ℝ16and has been demonstrated on a real-world mobile manipulation task.

The rising demand for customized products necessitates the integration of multiple robotic systems, underscoring the need for advanced production planning and scheduling. This paper introduces an ontology-based, artificial intelligence-enhanced method for dynamic task planning and scheduling, aimed at improving the efficiency of production process, reducing machine downtime, and consequently increasing throughput in assembly operations. Designed to generate and execute feasible production plans dynamically, this method minimizes manual planning and scheduling efforts. We evaluate its effectiveness using two gear assembly use cases with various robot skills, highlighting its flexibility in planning and scheduling and its contributions to the evolution of smart manufacturing. The method’s adaptability suggests its applicability across diverse smart factory environments.

Osteoarthritis of the knee, a widespread cause of knee disability, is commonly treated in orthopedics due to its rising prevalence. Lower extremity misalignment, pivotal in knee injury etiology and management, necessitates comprehensive mechanical alignment evaluation via frequently-requested weight-bearing long leg radiographs (LLR). Despite LLR's routine use, current analysis techniques are error-prone and time-consuming. To address this, we conducted a multicentric study to develop and validate a deep learning (DL) model for fully automated leg alignment assessment on anterior-posterior LLR, targeting enhanced reliability and efficiency. The DL model, developed using 594 patients' LLR and a 60%/10%/30% data split for training, validation, and testing, executed alignment analyses via a multi-step process, employing a detection network and nine specialized networks. It was designed to assess all vital anatomical and mechanical parameters for standard clinical leg deformity analysis and preoperative planning. Accuracy, reliability, and assessment duration were compared with three specialized orthopedic surgeons across two distinct institutional datasets (136 and 143 radiographs). The algorithm exhibited equivalent performance to the surgeons in terms of alignment accuracy (DL: 0.21 ± 0.18°to 1.06 ± 1.3°vs. OS: 0.21 ± 0.16°to 1.72 ± 1.96°), interrater reliability (ICC DL: 0.90 ± 0.05 to 1.0 ± 0.0 vs. ICC OS: 0.90 ± 0.03 to 1.0 ± 0.0), and clinically acceptable accuracy (DL: 53.9%-100% vs OS 30.8%-100%). Further, automated analysis significantly reduced analysis time compared to manual annotation (DL: 22 ± 0.6 s vs. OS; 101.7 ± 7 s, p ≤ 0.01). By demonstrating that our algorithm not only matches the precision of expert surgeons but also significantly outpaces them in both speed and consistency of measurements, our research underscores a pivotal advancement in harnessing AI to enhance clinical efficiency and decision-making in orthopaedics.

Future robots operating in fast-changing anthropomorphic environments need to be reactive, safe, flexible, and intuitively use both arms (comparable to humans) to handle task-space constrained manipulation scenarios. Furthermore, dynamic environments pose additional challenges for motion planning due to a continual requirement for validation and refinement of plans. This work addresses the issues with vector-field-based motion generation strategies, which are often prone to local-minima problems. We aim to bridge the gap between reactive solutions, global planning, and constrained cooperative (two-arm) manipulation in partially known surroundings. To this end, we introduce novel planning and real-time control strategies leveraging the geometry of the task space that are inherently coupled for seamless operation in dynamic scenarios. Our integrated multiagent global planning and control scheme explores controllable sets in the previously introducedcooperative dual task spaceand flexibly controls them by exploiting the redundancy of the high degree-of-freedom (DOF) system. The planning and control framework is extensively validated in complex, cluttered, and nonstationary simulation scenarios where our framework is able to complete constrained tasks in a reliable manner, whereas existing solutions fail. We also perform additional real-world experiments with a two-armed 14 DOF torque-controlled KoBo robot. Our rigorous simulation studies and real-world experiments reinforce the claim that the framework is able to run robustly within the inner loop of modern collaborative robots with vision feedback.

Natural language is one of the most intuitive ways to express human intent. However, translating instructions and commands towards robotic motion generation and deployment in the real world is far from being an easy task. The challenge of combining a robot's inherent low-level geometric and kinodynamic constraints with a human's high-level semantic instructions traditionally is solved using task-specific solutions with little generalizability between hardware platforms, often with the use of static sets of target actions and commands. This work instead proposes a flexible language-based framework that allows a user to modify generic robotic trajectories. Our method leverages pre-trained language models (BERT and CLIP) to encode the user's intent and target objects directly from a free-form text input and scene images, fuses geometrical features generated by a transformer encoder network, and finally outputs trajectories using a transformer decoder, without the need of priors related to the task or robot information. We significantly extend our own previous work presented in [1] by expanding the trajectory parametrization space to 3D and velocity as opposed to just XY movements. In addition, we now train the model to use actual images of the objects in the scene for context (as opposed to textual descriptions), and we evaluate the system in a diverse set of scenarios beyond manipulation, such as aerial and legged robots. Our simulated and real-life experiments demonstrate that our transformer model can successfully follow human intent, modifying the shape and speed of trajectories within multiple environments. Codebase avail-able at: https://github.com/arthurfenderbucker/LaTTe-Language-Trajectory-TransformEr.git.

As robots become more and more intelligent, the complexity of the algorithms behind them is increasing. Since these algorithms require high computation power from the onboard robot controller, the weight of the robot and energy consumption increases. A promising solution to tackle this issue is to relocate the expensive computation to the cloud. In this pioneering work, the possibility of relocating a state-of-the-art nonlinear control is investigated. To this end, the Unified Force-Impedance Controller (UFIC) is relocated to a remote location and high frequency feedback loop is established by including the remote controller in the loop. Passivity analysis is used to ensure the stability of the whole system, comprising the robot in interaction with the environment, the communication channel, as well as the remote controller. The instability associated with the communication channel is resolved by Time Domain Passivity Approach (TDPA). The performance of the proposed framework is experimentally evaluated on a robot arm in interaction with the environment. The results illustrate the stability of the system to a time-varying delay of up to 50 ± 10ms.

Tactile sensors are becoming more prevalent in numerous research domains, including robotics, human-robot interaction, and grasping. As the development of customized soft tactile skin for various applications continues to gain momentum, there is an increasing demand for the automation of design and manufacturing processes based on user specifications. Our work presents a partially automated framework for designing and customizing silicone-based skin-like sensors for objects of arbitrary shapes. We assess the performance of stretch and contact sensors featuring custom patterns on complex surfaces, subjecting them to position control, grasping, and manipulation scenarios. Our study's findings demonstrate the feasibility of fabricating skin-like sensors effectively within a semi-automated framework, with potential applications in the aforementioned research domains.

As robots and humans increasingly share the same workspace, the development of safe motion plans becomes paramount. For real-world applications, nonetheless, it is critical that safety solutions are achieved without compromising performance. The computation of safe, time-efficient trajectories, however, usually requires rather complex often decoupled planning and optimization methods which degrades the nominal performance. In this work, instead, we cast the problem as a graph search-based scheme that enables us to solve the problem efficiently. The graph search is guided by an informed cost balance criterion. In this context we present the S* algorithm which minimizes the total planning time by equilibrising shortest time-efficient paths and paths with higher safe velocities. The approach is compatible with standards and validated both in rigorous simulation trials on a 6 DoF UR5 robot as well as real world experiments on a Franka Emika 7 DoF research robot.

With the increasing demand for new healthcare solutions and technologies, such as those resulting from the COVID-19 crisis, and the growing elderly population, exoskeletons for teleoperation are a promising solution for many future medical applications. In this context, we propose two force- sensitive upper-limb exoskeletons for teleoperation, that are characterized by: i) torque-controlled robotic actuators, ii) rigid-body model compensations, and iii) a lightweight design achieved through the use of Bowden cable transmissions and remotely placed actuators. Specifically, we present a semi-active upper-limb exoskeleton for which we demonstrate human- device interaction control and bilateral teleoperation with force- feedback, evaluated via simulation, in the lab and over the Internet. We also introduce a design for a future fully-active upper-limb exoskeleton with two contact force/torque sensors, for a dual-arm device, which features a novel 3-degrees-of- freedom exoskeleton shoulder design and a contact wrench mitigation controller, as demonstrated through simulation. With this work, we propose the essential technical steps towards a novel teleoperation system for elderly care.

Identifying the inertial parameters of real robotic manipulators is a fundamental step towards realistic modeling and better controller performances, which is crucial for safe human-robot interaction. Our work introduces a novel framework for identifying a generalized set of base inertial parameters of a serial link manipulator. This framework is designed to be adaptable to accommodate any new mounting configuration of the robot. Our theoretical analysis highlights the influence of the robot's mounting configuration on the emergence of new parameters that cannot be identified through the conventional vertical base-axis mounting approach studied previously. To validate our proposed framework, we carried out two main experiments: the first involved simulation to establish the feasibility of our concept, and in the second, our framework was employed on a Franka Emika Robot in a real-world scenario to demonstrate and validate our approach. Our simulation results confirmed the feasibility of our proposed framework, while our real-world experiment successfully identified the generalized base inertial parameter set and validated its applicability to a new robot mounting configuration.

Studying the manipulation of deformable linear objects has significant practical applications in industry, including car manufacturing, textile production, and electronics automation. However, deformable linear object manipulation poses a significant challenge in developing planning and control algorithms, due to the precise and continuous control required to effectively manipulate the deformable nature of these objects. In this paper, we propose a new framework to control and maintain the shape of deformable linear objects with two robot manipulators utilizing environmental contacts. The framework is composed of a shape planning algorithm which automatically generates appropriate positions to place fixtures, and an object-centered skill engine which includes task and motion planning to control the motion and force of both robots based on the object status. The status of the deformable linear object is estimated online utilizing visual as well as force information. The framework manages to handle a cable routing task in real-world experiments with two Panda robots and especially achieves contact-aware and flexible clip fixing with challenging fixtures.

One can estimate the velocity and acceleration of robot manipulators by utilizing nonlinear observers. This involves combining inertial measurement units (IMUs) with the motor encoders of the robot through a model-based sensor fusion technique. This approach is lightweight, versatile (suitable for a wide range of trajectories and applications), and straightforward to implement. In order to further improve the estimation accuracy while running the system, we propose to adapt the noise information in this paper. This would automatically reduce the system vulnerability to imperfect modelings and sensor changes. Moreover, viable strategies to maintain the system stability are introduced. Finally, we thoroughly evaluate the overall framework with a seven DoF robot manipulator whose links are equipped with IMUs.

One of the most important aspects of autonomous systems is safety. This includes ensuring safe human-robot and safe robot-environment interaction when autonomously performing complex tasks or in collaborative scenarios. Al-though several methods have been introduced to tackle this, most are unsuitable for real-time applications and require carefully handcrafted obstacle descriptions. In this work, we propose a method combining high-frequency and real-time self and environment collision avoidance of a robotic manipulator with low-frequency, multimodal, and high-resolution environmental perceptions accumulated in a digital twin system. Our method is based on geometric primitives, so-called primitive skeletons. These, in turn, are information-compressed and real-time compatible digital representations of the robot's body and environment, automatically generated from ultra-realistic virtual replicas of the real world provided by the digital twin. Our approach is a key enabler for closing the loop between environment perception and robot control by providing the millisecond real-time control stage with a current and accurate world description, empowering it to react to environmental changes. We evaluate our whole-body collision avoidance on a 9-DOFs robot system through five experiments, demonstrating the functionality and efficiency of our framework.

Compliance with global guidelines for sustainable and responsible production in modern industry requires a comparative analysis of consumer devices' energy consumption (EC). This also holds true for the newly established generation of lightweight industrial robots (LIRs). To identify potential strategies for energy optimization, standardized benchmarking procedures are required. However, to the best of the authors' knowledge, there is currently no standardized method for benchmarking the EC of manipulators. In response to this need, we have developed a comprehensive benchmarking framework to evaluate the EC of various LIR designs, delving into the theoretical power consumption under both static and dynamic conditions. Our analysis has led to the proposal of seven proposed metrics—three static and four dynamic. The static metrics—controller consumption, joint electronics consumption, and mechanical brakes' consumption—evaluate the maintenance EC of the robot. Meanwhile, we suggest three dynamic metrics that gauge the system's energy efficiency during motion, with or without payload. We extend this metrics selection by introducing the cost of transportation map for manipulators. For each of the metrics, we suggest a standardized measurement procedure based on state-of-the-art norms and literature. The metric set and experimental procedures are demonstrated using five manipulators (UR3e, UR5e, FR3, M0609, Gen3). Among the results, we can see interesting trends for future optimization of the electronic components and their architecture, e.g., reducing the robot's EC by decentralizing computation via low-consumption onboard controllers for basic tasks and external servers for complex ones.

Shared-autonomy control strategies in teleoperation combine human decision-making and robot precision to solve complex tasks. In other words, advanced autonomous control algorithms can compensate for imprecise human commands, reduce the mental workload of the user, and enable the execution of tasks that otherwise wouldn't be feasible. This paper addresses one of these previously challenging scenarios. Herein, we present a novel control framework and motion generator that allows for real-time non-prehensile slosh-free teleoperation of liquids. The proposed approach is able to generate robust trajectories on the follower side which ensures task-space, joint-space, and manipulability constraint satisfaction. Our findings were evaluated through user studies and real-world scenarios. Participants were even explicitly challenged to try to spill liquid through teleoperation, reaching speeds up to 0.6 m/s.

Sophisticated manipulation requires both compliance and accuracy. While tactile robots excel at being compliant, their accuracy is often inadequate for complex manipulation. Contact-rich assembly tasks, such as the insertion and manipulation of objects with small tolerances pose an enormous challenge. Complex, highly integrated assemblies, especially in high-tech areas such as robotics, sensors, or machines, still require human personnel, as they cannot be automated in a satisfactory way. To automate such tasks, especially in the context of labor shortage and Industry 4.0, these limitations must be overcome. Robots need to guarantee force limits for active environments in order to avoid harm or damage. Therefore, in this work, we adapt standard Cartesian impedance control by introducing an integration term for position accuracy and wrench limits for safe compliant interaction with unknown and active environments. We combine this with a virtual energy tank to guarantee the general passivity of the controller. Our controller is benchmarked against standard impedance control for absolute positioning accuracy across the robot workspace. Furthermore, we show its applicability to an industrial insertion task. We demonstrate absolute positioning accuracy (residual error| Ax| < 4e – 4) comparable to rigid robots while preserving compliant behavior.

Automation in the life science research laboratory is a paradigm that has gained increasing relevance in recent years. Current robotic solutions often have a limited scope, which reduces their acceptance and prevents the realization of complex workflows. The transport and manipulation of laboratory supplies with a robot is a particular case where this limitation manifests. In this paper, we deduce a taxonomy of biolaboratory liquid containers that clarifies the need for a flexible grasping solution. Using the taxonomy as a guideline, we design fingers for a parallel robotic gripper which are developed with a monolithic dual-extrusion 3D print that integrates rigid and soft materials to optimize gripping properties. We design fine-tuned fingertips that provide stable grasps of the containers in question. A simple actuation system and a low weight are maintained by adopting a passive compliant mechanism. The ability to resist chemicals and high temperatures and the integration with a tool exchange system render the fingers usable for daily laboratory use and complex workflows. We present the task suitability of the fingers in experiments that show the wide range of vessels that can be handled as well as their tolerance against displacements and their grasp stability.

As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are already in use in the field of robotic health care research. We further provide ground truth data within one room, for assessing SLAM algorithms running directly on a health care robot.

This paper assesses the usefulness of immersive technology to evaluate perceived discomfort and proxemic behavior towards a robot depending on its size. Therefore, we compared a real and an augmented reality (AR) robot presentation. In a within-subject design, a service humanoid approached participants (N = 32) in four trials in a counterbalanced order. One trial presented a real robot, another showed a same-sized AR version, and two trials showed down-scaled AR versions of the robot. The perceived discomfort and the comfort distance were measured. For the presentation mode comparison, the distance estimation error was measured additionally. The results show that the comfort distance was greater for the AR robot than for the real one. The comfort distance was also greater for the largest robot compared to the smaller sizes, while there was no difference when comparing the smaller ones. There was no difference in perceived discomfort between the presentation modes or the robot sizes. The distance estimation error was greater in AR. The study indicates that results obtained with the AR and real robot are comparable relative to each other. Therefore, utilizing AR could effectively evaluate various versions of robots in terms of the discomfort they induce—a critical prerequisite prior to the manufacturing process. Finally, AR might provide better validity for the evaluation of subjective measures.

Bimanual humanoid assistive robots can be a valuable tool to improve access to physical therapy and multi-dimensional physical status monitoring for older adults living at home. However, at present, there is no implementation of end-effector robot-assisted rehabilitation in assistive social robots. Therefore, this paper illustrates the first steps of a user-centered design approach to develop such a robot for upper limb treatments and rehabilitation. Based on observation of geriatric rehabilitation and expert interviews with physical therapists, an online survey was conducted with 87 physical therapists. The first part of the questionnaire aimed to better understand the context of use, current practices, and goals of geriatric rehabilitation. Our findings suggest that integrating exercises that combine physical and cognitive skills and aim to improve specific activities of daily living (ADLs) are critical. Secondly, a KANO analysis was conducted to prioritize 20 potential features for the robot. Among these features, assist-as-needed (AAN) control for physical exercises, voice control for general settings, and the capability to perform treatments while the patient is seated or lying down were identified as essential or “must-have” features. Third, the possibility of an autonomously conducted weekly assessment of patients’ active range of motion (ROM) and weekly to monthly muscle function testing could allow the best possible monitoring of their functional status. Generally, applying a user-centered design approach allows an interdisciplinary team tasked with developing assistive robots to establish a common objective, based on which an initial prototype can be designed in the next step.

Flexible and versatile mobile robotic coworkers are becoming an indispensable commodity for helping humans with repetitive or physically demanding work. A key challenge with these systems is respecting the strict safety requirements in shared and collaborative workspaces. This inevitably requires solving their whole-body dynamics to obtain the necessary inertial impact properties. In this article, we present an integrated impact dynamics and safe precollision control framework to address the discussed challenge. We propose a novel modular dynamics approach that provides efficient formulations for reusing the uncoupled subsystem dynamics when evaluating the coupled system. Our approach is generalized for deriving the whole-body impact dynamics of any articulated floating-base robot. Furthermore, it outperforms classical monolithic approaches for computing the dynamics, making it favorable for systems with more than two dynamic subsystems while allowing decentralized computations. Finally, based on the proposed modular and generalized impact dynamics and extending our previous work, we introduce the generalized safe motion unit as a unified safety scheme for floating-base robotic structures with branched manipulation extremities. The proposed concepts are evaluated on an exemplary wheeled mobile manipulator, considering realistic use cases in simulation and real-world experiments. The obtained results validated the efficacy of our framework and developed methods.

Upper limb prostheses are commonly mounted to the human residual limb by a passive socket. By this design, the sensitive residual limb is exposed to high reaction wrenches, which can be a source of medical complications. In this article, we introduce an active force-sensitive robotic socket, which carries the prosthesis, offloads the residual limb, and allows guidance via small interaction forces at the same time. We investigate the feasibility of this concept by a force-sensitive and wearable shoulder exoskeleton, calledexoprosthesiswhen being combined with a prosthesis. We provide a first mechatronics prototype, two floating base controllers, and an analysis of the loads acting on the user body. Simulations and experiments confirm the concept and reveal that the wrench at residual limb can be fully compensated for the static case and by$\approx \text{50}\%$for the investigated motions. Human-in-the-loop tests are successfully performed by three able-bodied users showing the later real-world use case in a complex grasping situation. Overall, we believe that a force-sensitive robotic socket has the potential to advance prosthetics to a new level as it provides an intuitive and seamless user control interface.

Strategies for safe human-robot interaction (HRI), such as the well-established Safe Motion Unit, provide a velocity scaling for biomechanically safe robot motion. In addition, psychologically-based safety approaches are required for trustworthy HRI. Such schemes can be very conservative and robot motion complying with such safety approaches should be time efficient within the robot motion planning. In this study, we improve the efficiency of a previously introduced approach for psychologically-based safety in HRI via a Model Predictive Control robot motion planner that simultaneously adjusts Cartesian path and speed to minimise the distance to the target pose as fast as possible. A subordinate real-time motion generator ensures human physical safety by integrating the Safe Motion Unit. Our motion planner is validated by two experiments. The simultaneous adjustment of path and velocity accomplishes highly time efficient robot motion, while considering the human physical and psychological safety. Compared to direct path velocity scaling approaches our planner enables 28 % faster motion execution.

Knowledge of the robot's load inertial parameters is indispensable for accurate and safe operation, especially in collaborative robotics. However, an intuitive method for online inertial payload identification, usable while the robot is executing another online generated task, is still lacking. In this work, we propose an online payload identification approach based on the momentum observer using proprioceptive sensors of tactile robots and a novel filter design of kinematic measure-ments. Furthermore, we introduce a novel calibration scheme, that allows circumventing constraints of current calibration methods for payload identification. Specifically, the requirement of performing exactly the same motion for calibration as well as for the identification process is released. This is achieved by introducing an average virtual calibration object that improves the robot model for the identification process. In experiments with a Franka Emika Panda robot, it is shown that the proposed methods surpass common methods in terms of identification error. Especially, the novel calibration approach shows high robustness against temporal and spatial misalignment of the motions.

As collaborative robots move closer to human environments, motion generation and reactive planning strategies that allow for elaborate task execution with minimal easy-to-implement guidance whilst coping with changes in the environment is of paramount importance. In this paper, we present a novel approach for generating real-time motion plans for point-to-point tasks using a single successful human demonstration. Our approach is based on screw linear interpolation, which allows us to respect the underlying geometric constraints that characterize the task and are implicitly present in the demonstration. We also integrate an original reactive collision avoidance approach with our planner. We present extensive experimental results to demonstrate that with our approach, by using a single demonstration of moving one block, we can generate motion plans for complex tasks like stacking multiple blocks (in a dynamic environment). Analogous generalization abilities are also shown for tasks like pouring and loading shelves. For the pouring task, we also show that a demonstration given for one-armed pouring can be used for planning pouring with a dual-armed manipulator of different kinematic structure.

In physical human-robot interaction (pHRI), safety is a key requirement. As collisions between humans and robots can generally not be avoided, it must be ensured that the human is not harmed. The robot reflected mass, the contact geometry, and the relative velocity between human and robot are the parameters that have the most significant influence on human injury severity during a collision. The reflected mass depends on the robot configuration and can be optimized especially in kinematically redundant robots. In this paper, we propose the Mean Reflected Mass (MRM) metric. The MRM is independent of the direction of contact/motion and enables assessing and optimizing the robot posture w.r.t. safety. In contrast to existing metrics, it is physically interpretable, meaning that it can be related to biomechanical injury data for realistic and model-independent safety analysis. For the Franka Emika Panda, we demonstrate in simulation that an optimization of the robot's MRM reduces the mean collision force. Finally, the relevance of the MRM for real pHRI applications is confirmed through a collision experiment.

Nowadays, wheeled mobile robots constitute a considerable portion of robots in industrial applications. Generally, regardless of their purpose, these systems are not designed to physically interact with humans, other robots, or the environment. In this study, we present a novel safe autonomous mobile - SAM - robot, which is a torque-controlled compliant robot that is conceived for safe human-robot interaction. This work provides an overview of the development philosophy of the system, its mechanical and mechatronics structure along with control and navigation architecture. Preliminary results show the advantages of the proposed mobile robot while interacting with its surroundings. We believe that this study will bring the wheeled mobile robots one step closer to the proactive interaction with their environment and humans surrounding them.

We discuss a novel method for estimating task Cartesian position and velocity in robot manipulators. This is done by model-based fusion of inertial measurement units with motor encoders. The model is developed to robustly handle the uncertainties in the trajectory. Thus, not only the approach benefits from high fidelity and bandwidth thanks to multiple-sensory fusion, but it also enforces stability despite poorly formulated motions. This empowers the method to be utilized in complex closed-loop applications, where both task position and velocity information is required.

Natural language is the most intuitive medium for us to interact with other people when expressing commands and instructions. However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands. In this work, we provide a flexible language-based interface for human-robot collaboration, which allows a user to reshape existing trajectories for an autonomous agent. We take advantage of recent advancements in the field of large language models (BERT and CLIP) to encode the user command, and then combine these features with trajectory information using multi-modal attention transformers. We train the model using imitation learning over a dataset containing robot trajectories modified by language commands, and treat the trajectory generation process as a sequence prediction problem, analogously to how language generation architectures operate. We evaluate the system in multiple simulated trajectory scenarios, and show a significant performance increase of our model over baseline approaches. In addition, our real-world experiments with a robot arm show that users significantly prefer our natural language interface over traditional methods such as kinesthetic teaching or cost-function programming. Our study shows how the field of robotics can take advantage of large pre-trained language models towards creating more intuitive interfaces between robots and machines. Project webpage: https://arthurfenderbucker.github.io/NL_trajectory_reshaper/

Teleoperated robots are believed to play an important role for future applications in industry, medicine and other domains. Examples for this are remote assembly and maintenance, surgery, diagnosis or deep-sea and space exploration. Such applications are made possible by state-of-the-art tactile manipulators, well-researched control schemes and novel communication technologies such as the fifth generation of mobile communication (5G). The achievable performance is highly dependent on the communication delay and thus on the distance between leader and follower station, as well as the potentially used wireless protocol. Specially in this regard, 5G is a promising technology compared to the other communication protocols for transferring tactile information. In this paper, we introduce our telepresence reference platform, which can be used for empirical evaluation of different algorithms and communications. Comparative analysis are conducted to capture the influence of wireless communication protocols on telepresence systems consisting of complex robotic arms. The experiment compares the influence of 5G, LTE and WiFi communication protocols with regard to the motion and force tracking performance of the system.

Reaching human-level performance in tactile manipulation is one of the grand challenges in nowadays robotics research. Over the past decade significant progress in both skill control and learning was made. However, the achievable execution speed still falls behind the human ability, without clearly understanding whether the specific shortcomings are mainly in the control, skill learning, or motion planning layer. For gaining a better understanding of this complex problem, we draw an experimental side-by-side comparative case study. First, given a task program for a challenging benchmarking task, the goal is to objectify the achievable task performance from a human expert programmer against autonomously learning these assembly behaviors with a state-of-the-art skill learning framework. Second, we compare the manually tuned and learned robot skills to the performance of an adult human solving the task manually. For the former, it could be shown that despite longer learning duration, the task execution speed of the machine learning-based solution is equivalent to the one programmed by the human expert. For the latter, the identified performance gap remained significantly larger, where only for some specific isolated skills the system was able to reach comparable or even faster than human execution speeds. The overall analysis gave also useful hints where in particular manipulation policies and arm-hand coordination still need significant improvements in the future.

Tactile robots shall be deployed for dynamic task execution in production lines with small batch sizes. Therefore, these robots should have the ability to respond to changing conditions and be easy to (re-)program. Operating under uncertain environments requires unifying subsystems such as robot motion and force policy into one framework, referred to as tactile skills. In this paper, we propose the enhancement of these skills for passivity-based skill motion learning in stiffness-adaptive unified force-impedance control. To achieve the increased level of adaptability, we represent all tactile skills by three basic primitives: contact initiation, manipulation, and contact termination. To ensure passivity and stability, we develop an energy-based approach for unified force-impedance control that allows humans to teach the robot motion through physical interaction during the execution of a tactile task. We incorporate our proposed framework into a tactile robot to experimentally validate the motion adaptation by interaction performance and stability of the control. While the polishing task is presented as our use case through the paper, the experiments can also be carried out with various tactile skills. Finally, the results show the novel controller's stability and passivity to contact-loss and stiffness adaptation, leading to successful programming by interaction.

Kinesthetic teaching of collaborative robots is applied for intuitive and flexible robot programming by demonstration. This enables non-experts to program such robots on the task-level. Multiple strategies exist to teach velocity- or torque-controlled robots and, thus, the maneuverability among commercial robots differs significantly. However, currently there exists no metric that quantifies how “well” the robot can be guided, e.g., how much effort is required to initiate a motion. In this paper, we propose standardized procedures to quantitatively assess robot manual maneuverability. First, we identify different motion phases during kinesthetic teaching. For each phase, we then propose metrics and experimental setups to evaluate them. The experimental protocols are applied to the proprietary teaching schemes of five commercial robots, namely the KUKA LWR iiwa 14, Yuanda Yu+, Franka Emika robot, and Universal Robot's UR5e and UR10e. The experimental comparison highlights distinct differences between the robots and shows that the proposed methods are a meaningful contribution to the performance and ergonomics assessment of collaborative robots.

Laboratory automation is a suitable solution to establish higher reproducibility with less manual work and thus higher quality standards in life sciences. To date, mobile robots are capable of performing autonomous pick-and-place tasks in the laboratory, and specialized pipetting machines can be used for sequenced liquid handling. However, the complex and creative process of developing new research protocols requires flexible robotic systems that can perform tasks such as pipetting in more versatile ways. In addition, the correct technique, according to ISO standards, has a great influence on precision and accuracy and therefore on reproducibility. This paper introduces our Intelligent Robotic Lab Assistants in the framework of our holistic, human-like, but standardized paradigm for collaborative lab automation, AI.Laboratory. Our system demonstrates mastery of pipetting following ISO 8655 as a force-sensitive robotic manipulation skill, which is a key component of our taxonomy of cell culture skills and the first steps toward true intelligent robotic laboratory assistants. This intelligent robotic pipetting skill is a versatile tool for general handling of µL-liquids, using only standard laboratory equipment that can be flexibly positioned in the robot's workspace. To demonstrate its pipetting performance, flexible handling of small volumes from 10 µL to 1000 µL was experimentally validated to the ISO 8655 standard, demonstrating superhuman performance that outperformed laymen, human experts, and other commercial and non-commercial robotic pipetting systems.

We present an accessible robot interface for telemanipulation (A-RIFT), which preserves the haptic channel partially in a zero-additional-cost interface by visual substitution of force feedback (VSFF). This work explores a gap in the literature, resulting from the focus on performance improvements in telerobotics at increasing interface costs. Unlike most telemanipulation interfaces for high-degree-of-freedom robotic systems, this one requires minimal training and can be run in a web browser under high latency conditions, using an Internet connected computer with the user's own mouse and keyboard. To evaluate the performance of the system, we ran a controlled user study (N=12) to test how different distances (local vs. remote) and VSFF (on vs. off) affect the system's usability. As expected, participants in remote conditions performed worse than those in closer proximity. Despite several participants claiming that the visual display of force feedback did not help them, our analysis of their task performance showed that operators in remote condition actually performed statistically significantly better with the visual force feedback display than without it. These results indicate a promising new interface design direction for low-cost telemanipulation.

This paper is about fast slosh-free fluid transportation. Existing approaches are either computationally heavy or only suitable for specific robots and container shapes. We model the end effector as a point mass suspended by a spherical pendulum and study the requirements for slosh-free motion and the validity of the point mass model. In this approach, slosh-free trajectories are generated by controlling the pendulum's pivot and simulating the motion of the point mass. We cast the trajectory optimization problem as a quadratic program-this strategy can be used to obtain valid control inputs. Through simulations and experiments on a 7 DoF Franka Emika Panda robot we validate the effectiveness of the proposed approach.

Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators, this problem has been partially addressed. With a suitable optimal control strategy, the approximate decoupling of the motor from the link can be achieved to maximize the energy transfer into the distal link prior to launch. However, such continuous stiffness variation is complex and typically leads to oscillatory swing-up motions instead of clear launch sequences. To circumvent this issue, we investigate decoupling for speed maximization with a dedicated novel actuator concept denoted Bi-Stiffness Actuation. With this, it is possible to fully decouple the link from the joint mechanism by a switch-and-hold clutch and simultaneously keep the elastic energy stored. We show that with this novel paradigm, it is not only possible to reach the same optimal performance as with power-equivalent variable stiffness actuation, but even directly control the energy transfer timing. This is a major step forward compared to previous optimal control approaches, which rely on optimizing the full time-series control input.

Manipulability ellipsoids efficiently capture the human pose and reveal information about the task at hand. Their use in task-dependent robot teaching - particularly their transfer from a teacher to a learner - can advance emulation of human-like motion. Although in recent literature focus is shifted towards manipulability transfer between two robots, the adaptation to the capabilities of the other kinematic system is to date not addressed and research in transfer from human to robot is still in its infancy. This work presents a novel manipulability domain adaptation method for the transfer of manipulability information to the domain of another kinematic system. As manipulability matrices/ellipsoids are symmetric positive-definite (SPD) they can be viewed as points on the Riemannian manifold of SPD matrices. We are the first to address the problem of manipulability transfer from the perspective of point cloud registration. We propose a manifold-aware Iterative Closest Point algorithm (ICP) with parallel transport initialization. Furthermore, we introduce a correspondence matching heuristic for manipulability ellipsoids based on inherent geometric features. We confirm our method in simulation experiments with 2-DoF manipulators as well as 7-DoF models representing the human-arm kinematics.

In modern highly dynamic robot manipulation, collisions between a robot and objects may be intentionally executed to improve performance. To distinguish between these deliberate contacts and accidental collisions beyond the limit of state-of-the-art human-robot interactions, new sensing approaches are required. This work seeks an easy-to-implement and real-time capable solution to detect the identity of the impacted material. We developed an inertial measurement unit (IMU) based setup that records vibration signals occurring after collisions. Furthermore, a data-set was generated in an unsupervised learning manner using the measurements of collision experiments with several materials commonly used in realistic applications. The data-set was used to train an artificial neural network to classify the type of material involved. Our results show that the neural net detects collisions and a detailed distinction between materials is achieved, even with estimating different human body parts. The unsupervised data-set generation allows for a simple integration of new classes, which provides broader applicability of our approach. As the calculations are running faster than the control cycle of the robot, the output of our classifier can be used in real-time to decide about the robots reaction behavior.

In order to transform a robot into an intelligent machine it needs to be enabled to react to unforeseen events (most importantly collisions) during task execution and have a plan on how to continue the task afterwards. This requires a flexible operational framework that allows to define adaptive reactions and interactions with the motion generation and task planning stage. Within this work we first reason about the choices the robot has for reactions to unforeseen events such as collisions with respect to safety of humans in the workspace, the robot itself and the environment as well as the successful task execution. We further present a flexible reflex engine together with a concept of integration into the motion generation and control work flow. The reflex engine and it's reflex maneuvers are a combination of state machines and decision trees that take into account the state of the robot and the world. It is capable of choosing safe reactions and can differentiate between different levels of contact severity and according reaction sets. Several reflex maneuvers are evaluated towards safety performance criteria in real robot experiments using an ISO/TS 15066 conform measurement device. Some of the tested reflexes are furthermore integrated into an implementation of the proposed approach for a simple real world example task where the robot needs to pickup a container and dispose it's content into a bin.

In physical human-robot interaction (pHRI), robots need to detect and react to intended and unintended contacts in a safe manner. Proprioceptive sensing capabilities and collision detection and identification techniques differ among commercially available robots, which means that also their sensitivity to detect dynamic collisions with the environment or the human co-worker differ. Up to now, there exists no standardized procedure for assessing the contact sensitivity of a robotic system. In this paper, we propose the concept of contact sensitivity maps (CSM), a relationship between the robot's dynamic impact properties and the reliability of its collision handling. The CSM allows the robot user to determine for which robot workspace areas and dynamic collision parameters (mass, velocity) reliable contact detection and reaction can be expected. We propose a standardized benchmarking procedure and test setup for deriving CSMs. Finally, we analyze and compare the experimental results of the Universal Robots UR10e, UR5e, and Franka Emika Panda, where we observe significant differences in contact sensitivity.

This paper addresses the problem of real-time planning in constrained dual-arm manipulation scenarios. Our proposed coupling leverages manipulability information of the cooperative bimanual task-space to a vector-field based planner by means of a repulsive circulatory field, while geometric primitives in Spin(3)⋉ℝ3are explored for flexible task relaxation. Furthermore, the circular field informs the cooperative framework about the safety boundaries which are in turn used to further relax motion constraints within a collision-free ball in Cartesian space. This builds a funnel along the trajectory which can be directly tracked through the proposed switching of task-primitive-priorities. The switching strategy follows an approach that ensures robustness to chattering and continuity in the joint-space. Experiments verify that our framework can run within the inner control loop of Franka Emika Panda robots.

We transform classical robot inertial parameter identification into an online learning problem by integrating state-of-the-art gradient descent techniques and first-order principles from mechanics and differential geometry. Through this, incremental learning of fully physically feasible inertial properties without requiring any prior information is made possible. This is achieved using a version of Riemannian gradient descent equipped with experience replay that guarantees feasible parameter updates at all times during learning. Analysis of the method's performance are done on a virtual manipulator focusing on the influence that different measurement setups have on the estimation as well as on parameter feasibility and re-learning. Finally, we present experimental results on a real 7 DoF manipulator and evaluate the quality of the generated inverse dynamics torques and the corresponding model error.

While many advancements have been made in the development of template models for describing upright-trunk locomotion, the majority of the effort has been focused on the stance phase. In this paper, we develop a new compact dynamic model as a first step toward a fully unified locomotion template model (ULT-model) of an upright-trunk forward hopping system, which will also require a unified control law in the next step. We demonstrate that all locomotion subfunctions are enabled by adding just a point foot mass and a parallel leg actuator to the well-known trunk SLIP model and that a stable limit cycle can be achieved. This brings us closer toward the ultimate goal of enabling closed-loop dynamics for anchor matching and thus achieving simple, efficient, robust and stable upright-trunk gait control, as observed in biological systems.

Template models are frequently used to simplify the control dynamics for robot hopping or running. Passive limit cycles can emerge for such systems and be exploited for energy-efficient control. A grand challenge in locomotion is trunk stabilization when the hip is offset from the center of mass (CoM). The swing phase plays a major role in this process due to the moment of inertia of the leg; however, many template models ignore the leg mass. In this work, the authors consider a robot hopper model (RHM) with a rigid trunk and leg plus a hip that is displaced from the CoM. It has been previously shown that no passive limit cycle exists for such a model given a linear hip spring. In this work, we show that passive limit cycles can be found when a nonlinear hip spring is used instead. To the authors’ knowledge, this is the first time that a passive limit cycle has been found for this type of system.

The fast growth of communication technologies such as 5G provides high bandwidth and low latency wireless internet access. This enables both high definition video stream and real-time robot commands transmitted between robots and operators in the context of telepresence and teleoperation. Although there has been substantial research to establish algorithms that convert images to robot motions and telerobotic systems, little effort was made in establishing a clear scheme that enable artists to draw portraits using telerobotic systems. In this paper, we provide an easy-to-follow structure and implementation of a robot avatar for portrait drawing by artists through remote manipulation. The proposed telerobotic system uses a digital tablet and motion capture suit as input devices, which provides accurate drawing and continuous motion data stream respectively. With sensor fusion of the input data on the robot side, the drawing process presented in this work uses a unified force and impedance controller to ensure smooth and uniform pen-strokes. The proposed scheme was used to synthesise a system that was used by an artist to successfully finish the portrait drawing of Elon Musk. Finally, we show the effectiveness of the introduced control framework through an experiment. In particular, we validate the benefit of combining unified force and impedance control with sensor fusion of the digital tablet and motion capture suit data.

Improving robot systems via newly-developed sensing devices, control algorithms, or state estimators in order to obtain safe and efficient human-robot interaction as well as tactile manipulation skills requires standardized performance measurement protocols for objective comparison. Common protocols to evaluate robot motion performance are currently defined in EN ISO 9283:1998. For tactile and safety performance, however, no common metrics were agreed on nor standardized yet. In this paper, we propose a set of quantifiable performance criteria for robot performance analysis, objectifying robot force sensing, force control, and collision detection/reaction performance. We introduce the corresponding measurement setups and protocols, demonstrate and experimentally validate each with a Universal Robot UR10e and UR5e as well as a Franka Emika Panda robot arm. The proposed performance criteria, metrics, and experimental setups constitute the basis of a fully tactile performance and safety benchmarking framework that allows to objectively evaluate tactile robot performance via reproducible reference tests.

Similar to human work, robotic tasks sometimes require two hands to be accomplished. This requires coordinated motion planning and control. While fulfilling the task in a coordinated manner is already a big challenge, the task at hand becomes even harder when obstacles are introduced in the environment that need to be avoided. Furthermore in the case of dynamic environments, contacts cannot be avoided all the time, even with robust planning. In addition to geometric constraints, bimanual systems need to be able to detect and react to contacts during task execution. To this aim, we integrate a vector-field based planning scheme, that is able to avoid obstacles, with contact detection and reactive control methods based on contact wrench estimation such as admittance control. We also fuse the real contact forces into the planner directly together with the circular repulsive fields. The resulting planner-controller combination is capable of obstacle avoidance planning as well as reaction control in the case of unforeseen contacts that can also be used in situations where the manipulation needs to be guided by the environment such as landing control in only roughly known environments. We evaluate our approach on the torque-controlled Kobo bimanual set-up and also perform rigorous simulation studies.

Fast and robust gate perception is of great importance in autonomous drone racing. We propose a convolutional neural network-based gate detector (GateNet1) that concurrently detects gate’s center, distance, and orientation with respect to the drone using only images from a single fish-eye RGB camera. GateNet achieves a high inference rate (up to 60 Hz) on an onboard processor (Jetson TX2). Moreover, GateNet is robust to gate pose changes and background disturbances. The proposed perception pipeline leverages a fish-eye lens with a wide field-of-view and thus can detect multiple gates in close range, allowing a longer planning horizon even in tight environments. For benchmarking, we propose a comprehensive dataset (AU-DR) that focuses on gate perception. Throughout the experiments, GateNet shows its superiority when compared to similar methods while being efficient for onboard computers in autonomous drone racing. The effectiveness of the proposed framework is tested on a fully-autonomous drone that flies on previously-unknown track with tight turns and varying gate positions and orientations in each lap.

The need for comprehensive telemedicine solutions is becoming increasingly relevant due to challenges associated with the ageing population, the increasing shortage of health-care providers, and, more recently, the global pandemic. Existing solutions primarily focus on, e.g., electronic medical records, audiovisual connections, and, in some cases, robotic systems with very basic capabilities. Here we present a fundamentally new, holistic approach to a remote doctor visit, which enables transparent remote examination, anomaly detection, diagnosis, and rehabilitation. Our dual doctor-patient twin paradigm involves two robotic systems: one representing the doctor to the patient ("GARMI") and one representing the patient to the doctor ("MUCKI"). Through bidirectional telepresence control, this system enables transparent, natural, remote haptic interaction between doctor and patient. The control, interaction, and knowledge transfer to the doctor is enhanced by AI-based visual motion and facial expression analysis as well as a digital twin of the patient. Thus, each stage of a doctor visit can be replicated in the context of telemedicine and shared autonomy: from first assessment to observation-based and remote physical examination, to a better-informed doctor diagnosis and robot-assisted telerehabilitation.

Humanoid robots in unknown environments need to be able to quickly react to contacts in order to ensure safety of humans and their own hardware. For showing useful reactions to contacts, the robot needs information about possibly multiple contacts such as their respective contact locations and wrenches. In this paper, we introduce our algorithm rm-Code, a real-time multi-contact detection, isolation and identification algorithm for tree-structured floating base robots based on generalized external forces and (optional) external wrenches measured by force/torque sensors within the kinematic chain. Those entities have been deduced in the literature using proprioceptive sensing only. Furthermore, the algorithm is fast enough for online computation. To the best of our knowledge, this is the first algorithm combining all of these properties. Rm-Code is quantitatively evaluated in simulation. The results show that our solution is able to accurately solve the problem when fed with perfect input data. In a second step, possible sources of error in the presence of noisy input data are analyzed. It is concluded that purely proprioception based contact isolation and identification in the multi-contact case has certain limitations under realistic conditions. However, these limitations could be overcome easily by integrating simple link contact detection, e.g. bumpers or other similarly simple means.

Haptic feedback in teleoperation of flying robots can enable safe flight in unknown and densely cluttered environments. It is typically part of the robot's control scheme and used to aid navigation and collision avoidance via artificial force fields displayed to the operator. However, to achieve fully immersive embodiment in this context, high fidelity force feedback is needed. In this paper we present a telepresence scheme that provides haptic feedback of the external forces or wind acting on the robot, leveraging the ability of a state-of-the-art flying robot to estimate these values online. As a result, we achieve true force feedback telepresence in flying robots by rendering the actual forces acting on the system. To the authors' knowledge, this is the first telepresence scheme for flying robots that is able to feedback real contact forces and does not depend on their representations. The proposed event-based teleoperation scheme is stable under varying latency conditions. Secondly, we present a haptic interface design such that any haptic interface with at least as many force-sensitive and active degrees of freedom as the flying robot can implement this telepresence architecture. The approach is validated experimentally using a Skydio R1 autonomous flying robot in combination with a ForceDimension sigma.7 and a Franka Emika Panda as haptic devices.

In this paper we introduce dentronics as a new emerging application domain for collaborative lightweight robots in the dental context backed up by a user survey supporting the clear need. Specifically, we developed a multi-modal interaction framework, applied this framework to a specific dental use-case, and conducted a preliminary user-study for evaluation. Our results demonstrate usability and feasibility beyond a controlled experimental setup. We conclude that dentronics is indeed within reach given today's technology and deserves further investigation towards clinical use.

In this paper we introduce a novel framework for expressing and learning force-sensitive robot manipulation skills. It is based on a formalism that extends our previous work on adaptive impedance control with meta parameter learning and compatible skill specifications. This way the system is also able to make use of abstract expert knowledge by incorporating process descriptions and quality evaluation metrics. We evaluate various state-of-the-art schemes for meta parameter learning and experimentally compare selected ones. Our results clearly indicate that the combination of our adaptive impedance controller with a carefully defined skill formalism significantly reduces the complexity of manipulation tasks even for learning peg-in-hole with submillimeter industrial tolerances. Overall, the considered system is able to learn variations of this skill in under 20 minutes. In fact, experimentally the system was able to perform the learned tasks without visual feedback faster than humans, leading to the first learning-based solution of complex assembly at such real-world performance.

In this paper the first robot-based prototype of a semi-autonomous upper-limb exoprosthesis is introduced, unifying exoskeletons and prostheses [1]. A central goal of this work is to minimize unnecessary interaction forces on the residual limb by compensating gravity effects via a upper body grounded exoskeleton. Furthermore, the exoskeleton provides the residual limb's kinematic data that allows to design more intelligent coordinated control concepts. The soft-robotics design of a prototype consisting of a transhumeral prosthesis and a robot-based exoskeleton substitute is outlined. For this class of hybrid systems a human embodied dynamics model and semi-autonomous coordinated motion strategies are derived. Here, in contrast to established standard sequential strategies all joints are moved simultaneously according to a desired task. In combination with an app-based programming framework the strategy goals are set either user-based via kinesthetic teaching or autonomously via 3D visual perception. This enables the user to execute tasks faster and more intuitive. First experimental evaluations show promising performance with a healthy subject.

Kinematic redundancy enhances the dexterity and flexibility of robot manipulators. By exploiting the redundant degrees of freedom, auxiliary null space tasks can be carried out in addition to the primary task. Such auxiliary tasks are often formulated in terms of a performance or safety criterion that shall be minimized. If the optimization criterion, however, is defined in global terms, then it is directly affected by the primary task. As a consequence, the task achievement of the auxiliary task may be unnecessarily detrimented by the main task. In addition to modifying the primary task via constraint relaxation, a possible solution for improving the performance of the auxiliary task is to relax the primary task temporarily via time scaling. This gives the null space task more time for achieving its objective. In this paper, we propose several such time scaling schemes and verify their performance for a DLR/KUKA Lightweight Robot with one redundant degree of freedom. Finally, we extend the concept to multiple prioritized tasks and provide a simulation example.

This paper deals with the problem of accurately computing and estimating joint velocity and acceleration in robotic manipulators. Generally, it is well known that numerical differentiation of noisy position signals even with significant filtering is no viable solution. This is especially true for computing joint acceleration. Specifically, our solution to this problem fuses joint position measurement with link accelerometers, which are affordable and easy to install. Since the sensor readings are affected by noise, drift and bias, suitable data fusion and filtering methods are proposed for improving the estimation for practical use. Simulation results based on a realistic dynamics model of a 7-DoF robot including various parasitic effects and experimental results with a 7-DoF robot demonstrate the effectiveness of our approach. This method would have multiple use, e.g., in monitoring external joint torques and handle possibly unforeseen collisions. Furthermore, other applications such as load identification and compensation as well as state feedback linearization for flexible joint robots could finally become possible also practical.

In practical robotic applications various types of tools are attached for manipulating objects. Besides adding gravitational load to the robot, which results in larger joint torques, such payloads influence the collision safety characteristics through changing surface curvature properties, reflected mass and effective robot speed along a desired motion direction. In this paper, we evaluate the effect a known, unactuated pay-load that is attached to the end-effector has on the intantaneous reflected inertial parameters and maximum task velocity of a robot. The proposed mass update approach relies on the analysis of the kinetic energy matrices, while the velocity maximization is tackled by formulating static optimization problems with different constraints on angular motion of the end-effector. Finally, for analyzing the validity of the introduced approach in the framework of Safety Maps, we discuss simulation results of a PUMA 560 robot that has an exemplary payload attached to its end-effector.

In this paper we propose a novel energy-based control scheme for an assist-as-needed rehabilitation strategy, which both adapts the level of support based on patient participation and allows the patient to deviate from the prescribed motion in favor of his/her safety. We build an energy network model, with which we can monitor the energy flow through the system and prescribe a threshold on stored energy. We also develop an adaptive motion control law that shapes the desired trajectory in order to respect the stored energy threshold. Next, we show how adapting the stored energy threshold can be used to change the level of responsiveness to the patient as well as to prevent excessive energy transfer to the human by the system. A criterion is defined for setting this energy threshold, which can be further used for monitoring the patient active participation and for adapting and learning the appropriate assistance level during rehabilitation. Experimental results based on implementation in MATLAB Simscape® and on the VEMO robotic system demonstrate the feasibility of the suggested approach. The presented control scheme can be applied to any system, including position- and torque-controlled robots, and does not require the use of EMG sensors or precise force measurements.

In this paper, we discuss and speculate about the concept of the Tactile Robot connected with human operators via smart wearables as an essential multimodal embodiment of the coming Tactile Internet. The Tactile Robot, succeeding the recently introduced kinesthetic soft robot, is the upcoming next step in the evolution of rapidly developing robotic platforms that are capable of sensitive physical interaction with their environment. From the combination of rich tactile feedback with state-of-the-art robotics, technology, and algorithms emerge the potential of a meaningful and immersive connection to human operators via the vastly progressing smart wearables and virtual reality/augmented reality devices, effectively creating real-world avatars. Moreover, the Tactile Internet is believed to make it possible to create avatar collectives spanning different application domains and, therefore, cover heterogeneous robotic platforms. We hypothesize that this development will enable us to seamlessly interact with heterogeneous systems such as industrial assembly lines, service robots, automated medical units, or even deep sea and space exploration units. This new paradigm of an immersive coexistence between humans and robots builds on numerous technological advances in robotics, multimodal teleoperation, wearable technology, distributed computing, or network technology, for example. However, such a vision obviously poses major challenges in multiple areas that are still to be overcome. In this paper, we discuss the potentials and enabling technologies together with foreseeable application domains in the framework of the Tactile Internet. Furthermore, we address major challenges and hypothesize about potential solutions.

In this paper we introduce a novel point-to-point trajectory planner for serial robotic structures that combines the ability to avoid self-collisions and to respect motion constraints, while complying with the requirement of being C4continuous. The latter property makes our approach also suited for 4th order dynamics flexible joint robots, which gained significant practical relevance recently. In particular, we address the problem of generating a smooth, kinematically nearly time-optimal SE(3) trajectory while simultaneously avoiding potential collisions of the robot end-effector with its base as well as respecting the Cartesian unreachable states induced by the joint limits of the proximal kinematic structure.

Performing skillfull manipulation is a very challenging task for robots. So far, even experts could barely program them to e.g. perform the well known peg-in-hole problem in the real world. Autonomously acquiring such skills, let alone generalizing them to new tasks, is still a major challenge. Typically, manipulation learning is approached with the help of large computation power, very long learning times, or possibly both. However, the performance achieved up to now is still far from human performance. We show the results of our new paradigm to robot manipulation. It bridges and unifies basic motor control, simple and complex manipulation strategies and high-level manipulation planning. The robots show autonomous skill learning, intra-class and inter-class generalization of insertion skills at human-level performance.

Encoding desired motions into dynamic movement primitives (DMPs) is a common way for generating compact task representations that are able to handle sensor-based goal adaptations. At the same time, a robot should not only express adaptive motion capabilities at planning level, but use also contact wrench feedback in the adaptation and learning process of the DMP. Despite first approaches exist in this direction, no fully integrated approach has been proposed so far. In this paper, we introduce a new class of admittance-coupled DMPs that addresses environmental changes by including contact wrench feedback dynamics into the DMP formalism. Moreover, a novel iterative learning approach is devised that is based on monitoring the overall system passivity analysis in terms of reference power tracking. Simulations and experimental results with the Kuka LWR robot maintaining a non-rigid contact with the environment (wiping a surface) are shown for supporting the validity of our approach.

For creating robots that are capable of human-like performance in terms of speed, energetic properties, and robustness, intrinsic compliance is a promising design element. In this paper, we investigate the principle effects of elastic energy storage and release for basketball dribbling in terms of open-loop cycle stability. We base the analysis, which is performed for the 1-degree-of-freedom (DoF) case, on error propagation, peak power performance during hand contact, and robustness with respect to varying hand stiffness. As the ball can only be controlled during contact, an intrinsically elastic hand extends the contact time and improves the energetic characteristics of the process. To back up our basic insights, we extend the 1-DoF controller to 6-DoFs and show how passive compliance can be exploited for a 6-DoF cyclic ball dribbling task with a 7-DoF articulated Cartesian impedance controlled robot. As a human is able to dribble blindly, we decided to focus on the case of contact force sensing only, i.e., no visual information is necessary in our approach. We show via simulation and experiment that it is possible to achieve a stable dynamic cycle based on the 1-DoF analysis for the primary vertical axis together with control strategies for the secondary translations and rotations of the task. The scheme allows also the continuous tracking of a desired dribbling height and horizontal position. The approach is also used to hypothesize about human dribbling and is validated with captured data.

Humans can skilfully use tools and interact with the environment by adapting their movement trajectory, contact force, and impedance. Motivated by the human versatility, we develop here a robot controller that concurrently adapts feedforward force, impedance, and reference trajectory when interacting with an unknown environment. In particular, the robot's reference trajectory is adapted to limit the interaction force and maintain it at a desired level, while feedforward force and impedance adaptation compensates for the interaction with the environment. An analysis of the interaction dynamics using Lyapunov theory yields the conditions for convergence of the closed-loop interaction mediated by this controller. Simulations exhibit adaptive properties similar to human motor adaptation. The implementation of this controller for typical interaction tasks including drilling, cutting, and haptic exploration shows that this controller can outperform conventional controllers in contact tooling.

In this paper the concept of a unified torque/impedance controller is applied to a pneumatically actuated, antagonistic robot joint. The investigated control algorithm consists of a cascaded structure in which the outer torque/impedance controller commands a desired torque to two cylinder-based force controllers. The torque-/impedance controller is equipped with a virtual tank that ensures passivity of the control loops above force level. Additionally, a shaping function provides a continuous transition to impedance control in case of sudden contact loss during a torque control operation. External torques used in the feedback loop for contact force regulation are estimated by a momentum observer. Experimental and simulation results show a maximum deflection of 9.7° over 130 ms in case of contact loss until a resting position is reached. Additionally, step and sinusoidal torque tracking up to 5 Hz was successfully tested at good performance.

High-performance collision handling, which is divided into the five phases detection, isolation, estimation, classification and reaction, is a fundamental robot capability for safe and sensitive operation/interaction in unknown environments. For complex humanoid robots collision handling is obviously significantly more complex than for classical static manipulators. In particular, the robot stability during the collision reaction phase has to be carefully designed and relies on high fidelity contact information that is generated during the first three phases. In this paper, a unified realtime algorithm is presented for determining unknown contact forces and contact locations for humanoid robots based on proprioceptive sensing only, i.e. joint position, velocity and torque, as well as force/torque sensing along the structure. The proposed scheme is based on nonlinear model-based momentum observers that are able to recover the unknown contact forces and the respective locations. The dynamic loads acting on internal force/torque sensors are also corrected based on a novel nonlinear compensator. The theoretical capabilities of the presented methods are evaluated in simulation with the Atlas robot. In summary, we propose a full solution to the problem of collision detection, collision isolation and collision identification for the general class of humanoid robots.

Kinematic redundancy is a characteristic and beneficial property in collaborative robots nowadays as it enhances the flexibility and dexterity of the system. While the robot is manipulating an object, it is often necessary to kinematically reconfigure the robot, for example, when it obstructs the human. For this, internal or so-called null space motions can be carried out which do not affect the main task. In general, it is desirable that the human coworker can anticipate how the robot will move at any time. However, for null space motions this is typically not the case as they are non-intuitive and not suitable for interaction. In this work, we develop intuitive null space interaction behaviors for redundant manipulators, where the human can easily guide the robot. We want to provide users with a tool, that is straightforward to implement and solves real-world problems effectively. Two practical applications for an eight- and ten-DOF robot demonstrate the performance of the proposed method.

Ensuring safety is a primary goal in physical human-robot interaction. In various collision experiments it was found that the robot's effective mass, velocity, and geometry are the key parameters which influence the human injury severity during an impact. Recently, a velocity controller was proposed that limits the robot speed to a biomechanically safe value, taking into account the mass and the curvature in the direction of movement for a given point of interest. The mass and the geometry depend on the mechanical design, however, the effective mass also depends on the robot configuration. In this paper, we exploit the redundant degree(s) of freedom of a joint torque controlled seven- and eight-DOF robot to minimize the effective mass without affecting the desired Cartesian end-effector trajectory and with the goal to improve the performance of the safe velocity controller at the same time. Given recent results in robotics injury analysis, we analyze when such a redundancy resolution scheme actually improves safety. For the considered robots, we find reflected mass extrema that can be obtained by null space motions, and propose a real-time, torque-based redundancy resolution scheme, which is finally verified in experiments.

Robot assistants and professional coworkers are becoming a commodity in domestic and industrial settings. In order to enable robots to share their workspace with humans and physically interact with them, fast and reliable handling of possible collisions on the entire robot structure is needed, along with control strategies for safe robot reaction. The primary motivation is the prevention or limitation of possible human injury due to physical contacts. In this survey paper, based on our early work on the subject, we review, extend, compare, and evaluate experimentally model-based algorithms for real-time collision detection, isolation, and identification that use only proprioceptive sensors. This covers the context-independent phases of the collision event pipeline for robots interacting with the environment, as in physical human–robot interaction or manipulation tasks. The problem is addressed for rigid robots first and then extended to the presence of joint/transmission flexibility. The basic physically motivated solution has already been applied to numerous robotic systems worldwide, ranging from manipulators and humanoids to flying robots, and even to commercial products.

Flying in unknown environments may lead to unforeseen collisions, which may cause serious damage to the robot and/or its environment. In this context, fast and robust collision detection combined with safe reaction is, therefore, essential and may be achieved using external wrench information. Also, deliberate physical interaction requires a control loop designed for such a purpose and may require knowledge of the contact wrench. In principle, the external wrench may be measured or estimated. Whereas measurement poses large demands on sensor equipment, additional weight, and overall system robustness, in this paper we present a novel model-based method for external wrench estimation in flying robots. The algorithm is based on the onboard inertial measurement unit and the robot's dynamics model only. We design admittance and impedance controllers that use this estimate for sensitive and robust physical interaction. Furthermore, the performance of several collision detection and reaction schemes is investigated in order to ensure collision safety. The identified collision location and associated normal vector located on the robot's convex hull may then be used for sensorless tactile sensing. Finally, a low-level collision reflex layer is provided for flying robots when obstacle avoidance fails, also under wind influence. Our experimental and simulation results show evidence that the methodologies are easily implemented and effective in practice.

We consider the problem of estimating the wind velocity perceived by a flying multicopter, from data acquired by onboard sensors and knowledge of its aerodynamics model only. We employ two complementary methods. The first is based on the estimation of the external wrench (force and torque) due to aerodynamics acting on the robot in flight. Wind velocity is obtained by inverting an identified model of the aerodynamic forces. The second method is based on the estimation of the propeller aerodynamic power, and provides an estimate independent of other sensors. We show how to calculate components of the wind velocity using multiple aerodynamic power measurements, when the poses between them are known. The method uses the motor current and angular velocity as measured by the electronic speed controllers, essentially using the propellers as wind sensors. Verification of the methods and model identification were done using measurements acquired during autonomous flights in a 3D wind tunnel.

Soft robotics methods such as impedance control and reflexive collision handling have proven to be a valuable tool to robots acting in partially unknown and potentially unstructured environments. Mainly, the schemes were developed with focus on classical electromechanically driven, torque controlled robots. There, joint friction, mostly coming from high gearing, is typically decoupled from link-side control via suitable rigid or elastic joint torque feedback. Extending and applying these algorithms to stiff hydraulically actuated robots poses problems regarding the strong influence of friction on joint torque estimation from pressure sensing, i.e. link-side friction is typically significantly higher than in electromechanical soft robots. In order to improve the performance of such systems, we apply state-of-the-art fault detection and estimation methods together with observer-based disturbance compensation control to the humanoid robot Atlas. With this it is possible to achieve higher tracking accuracy despite facing significant modeling errors. Compliant end-effector behavior can also be ensured by including an additional force/torque sensor into the generalized momentum-based disturbance observer algorithm from [1].

Detecting and interpreting contacts is a crucial aspect of physical Human-Robot Interaction. In order to discriminate between intended and unintended contact types, we derive a set of linear and non-linear features based on physical contact model insights and from observing real impact data that may even rely on proprioceptive sensation only. We implement a classification system with a standard non-linear Support Vector Machine and show empirically both in simulations and on a real robot the high accuracy in off- as well as on-line settings of the system. We argue that these successful results are based on our feature design derived from first principles.

It has recently been shown that intrinsically elastic robots are capable of outperforming rigid robots in terms of peak velocity by making systematic use of energy storage and release. Certainly, high link side velocities are beneficial for performance, however, they also increase the probability of self damage or human injury in case of a collision. To ensure the physical integrity of both human and robot, it is therefore crucial to avoid potentially dangerous collisions and react in a compliant manner if unwanted contact has occurred or may occur unforeseeable. In this paper, we consider the most intuitive collision anticipation and pre-reaction scheme, namely stopping an elastic robot, if possible in minimum time. For 1-DOF elastic joints with limited elastic deflection we extend existing model-based and model-free controllers and compare their performance. Furthermore, we analyze the braking trajectory that is achieved with the different strategies. The 1-DOF solution is extended to the double pendulum case, where we show that feasible estimates for maximum and final position can be obtained at the very first instant of braking.

In this paper we propose a novel hybrid Cartesian force/impedance controller that is equipped with energy tanks to preserve passivity. Our approach overcomes the problems of (hybrid) force control, impedance control, and set-point based indirect force control. It allows accurate force tracking, full compliant impedance behavior, and safe contact resemblance simultaneously by introducing a controller shaping function that robustly handles unexpected contact loss and avoids chattering behavior that switching based approaches suffer from. Furthermore, we propose a constructive way of initiating the energy tanks via the concept of task energy. This represents an estimate of the energy consumption of a given force control task prior to execution. The controller can be applied to both rigid body and flexible joint dynamics. To show the validity of our approach, several simulations and experiments with the KUKA/DLR LWR-III are carried out.

In this paper, we extend our previous external wrench estimation scheme for flying robots with an aerodynamic model such that we are able to simultaneously estimate aerodynamic and contact forces online. This information can be used to identify the metric wind velocity vector via model inversion. Noticeably, we are still able to accurately sense collision forces at the same time. Discrimination between the two is achieved by identifying the natural contact frequency characteristics for both “interaction cases”. This information is then used to design suitable filters that are able to separate the aerodynamic from the collision forces for subsequent use. Now, the flying system is able to correctly respond to typical contact forces and does not accidentally “hallucinate” contacts due to a misinterpretation of wind disturbances. Overall, this paper generalizes our previous results towards significantly more complex environments.

Many future application scenarios of robotics envision robotic agents to be in close physical interaction with humans: On the factory floor, robotic agents shall support their human co-workers with the dull and health threatening parts of their jobs. In their homes, robotic agents shall enable people to stay independent, even if they have disabilities that require physical help in their daily life - a pressing need for our aging societies. A key requirement for such robotic agents is that they are safety-aware, that is, that they know when actions may hurt or threaten humans and actively refrain from performing them. Safe robot control systems are a current research focus in control theory. The control system designs, however, are a bit paranoid: programmers build “software fences” around people, effectively preventing physical interactions. To physically interact in a competent manner robotic agents have to reason about the task context, the human, and her intentions. In this paper, we propose to extend cognition-enabled robot control by introducing humans, physical interaction events, and safe movements as first class objects into the plan language. We show the power of the safety-aware control approach in a real-world scenario with a leading-edge autonomous manipulation platform. Finally, we share our experimental recordings through an online knowledge processing system, and invite the reader to explore the data with queries based on the concepts discussed in this paper.

In this paper we present an experimental study on real-time collision avoidance with potential fields that are based on 3D point cloud data and processed on the Graphics Processing Unit (GPU). The virtual forces from the potential fields serve two purposes. First, they are used for changing the reference trajectory. Second they are projected to and applied on torque control level for generating according nullspace behavior together with a Cartesian impedance main control loop. The GPU algorithm creates a map representation that is quickly accessible. In addition, outliers and the robot structure are efficiently removed from the data, and the resolution of the representation can be easily adjusted. Based on the 3D robot representation and the remaining 3D environment data, the virtual forces that are fed to the trajectory planning and torque controller are calculated. The algorithm is experimentally verified with a 7-Degree of Freedom (DoF) torque controlled KUKA/DLR Lightweight Robot for static and dynamic environmental conditions. To the authors knowledge, this is the first time that collision avoidance is demonstrated in real-time on a real robot using parallel GPU processing.

In this paper, we present a method for learning and online generalization of maneuvers for quadrotor-type vehicles. The maneuvers are formulated as optimal control problems, which are solved using a general purpose optimal control solver. The solutions are then encoded and generalized with Dynamic Movement Primitives (DMPs). This allows for real-time generalization to new goals and in-flight modifications. An effective method for joining the generalized trajectories is implemented. We present the necessary theoretical background and error analysis of the generalization. The effectiveness of the proposed method is showcased using planar point-to-point and perching maneuvers in simulation and experiment.

Recently, intrinsically elastic joints became increasingly popular due to several reasons. Most importantly, elasticity improves impact robustness and, if used wisely, energy efficiency. Potential energy storage and release capabilities in the joints allow to outperform rigid manipulators by means of achievable peak link velocity. It has therefore been of great interest to find explosive or cyclic motions, similar to those of humans or animals, that make systematic use of joint elasticity. In this context, we address two important control problems in the present paper. First, we find all potential system states that a visco-elastic joint with constrained deflection may reach from its equilibrium state and analyze the influence of system parameters on the according reachable set. While high link velocities are certainly desirable in terms of performance, they may also increase the robot's level of dangerousness and/or the risk of self damage during potentially unforeseen collisions. Thus, we tackle the problem of how to brake a visco-elastic joint in minimum time. Furthermore, the results are extended to a near-optimal real-time control law for elastic n-DOF manipulators. The proposed braking controller is experimentally verified on a KUKA/DLR LWR4 in joint impedance control.

Flying in unknown environments can lead to unwanted collisions with the environment. If not being accounted for, these may cause serious damage to the robot and/or its environment. Fast and robust collision detection combined with safe reaction is therefore essential in this context. Deliberate physical interaction may also be required in some applications. The robot can then switch into an interaction mode when contact occurs. The control loop must also be designed with interaction in mind. To implement these mechanisms, knowledge of environmental interaction forces is required. In principle, they may be measured or estimated. In this paper, we present a novel model-based method for external wrench estimation in flying robots. The estimation is based on proprioceptive sensors and the robot's dynamics model only. Using this estimate, we also design admittance and impedance controllers for sensitive and robust physical interaction. We also investigate the performance of our collision detection and reaction schemes in order to guarantee collision safety. Upon collision, we determine the collision location and normal located on the robot's geometric model. The method relies on the complete wrench information provided by our scheme. This allows applications such as tactile environment mapping.

In this paper we present an approach to extending the capabilities of telemanipulation systems by intelligently augmenting a human operator's motion commands based on quantitative three-dimensional scene perception at the remote telemanipulation site. This framework is the first prototype of the Augmented Shared-Control for Efficient, Natural Telemanipulation (ASCENT) System. ASCENT aims to enable new robotic applications in environments where task complexity precludes autonomous execution or where low-bandwidth and/or high-latency communication channels exist between the nearest human operator and the application site. These constraints can constrain the domain of telemanipulation to simple or static environments, reduce the effectiveness of telemanipulation, and even preclude remote intervention entirely. ASCENT is a semi-autonomous framework that increases the speed and accuracy of a human operator's actions via seamless transitions between one-to-one teleoperation and autonomous interventions. We report the promising results of a pilot study validating ASCENT in a transatlantic telemanipulation experiment between The Johns Hopkins University in Baltimore, MD, USA and the German Aerospace Center (DLR) in Oberpfaffenhofen, Germany. In these experiments, we observed average telemetry delays of 200ms, and average video delays of 2s with peaks of up to 6s for all data. We also observed 75% frame loss for video streams due to bandwidth limits, giving 4fps video.

The DLR Robot Motion Simulator is a serial kinematics based platform that employs an industrial robot (as opposed to the conventional `Hexapod') to impart motion cues to the attached simulator cell. This simulation platform is the culmination of ongoing research on motion simulation at the Robotics and Mechatronics Center, German Aerospace Center (DLR). Safety tests were undertaken to ascertain the effects of critical motions and subsequent emergency stop procedures on the prospective human passengers of the simulator cell. To this end, an Anthropomorphic Test Device (ATD) aka `crash test dummy' was used as a human surrogate for these tests. Several severity indices were evaluated for the head-neck region, which was found to be more susceptible to injuries compared to the rest of the body. The results of this study are discussed in this paper.

In this paper we describe a system for aerial manipulation composed of a helicopter platform and a fully actuated seven Degree of Freedom (DoF) redundant industrial robotic arm. We present the first analysis of such kind of systems and show that the dynamic coupling between helicopter and arm can generate diverging oscillations with very slow frequency which we called phase circles. Based on the presented analysis, we propose a control approach for the whole system. The partial decoupling between helicopter and arm - which eliminates the phase circles - is achieved by means of special movement of robotic arm utilizing its redundant DoF. For the underlying arm control a specially designed impedance controller was proposed. In different flight experiments we showcase that the proposed kind of system type might be used in the future for practically relevant tasks. In an integrated experiment we demonstrate a basic manipulation task - impedance based grasping of an object from the environment underlaying a visual object tracking control loop.

Designing intrinsically elastic robots recently attracted significant attention. Inspired by the elasticity in biological muscles, these designs aim at enabling robots to imitate human or animal motions during various tasks such as hopping, running, etc. In particular, reaching peak velocities, using the stored energy in the according elasticities, is of great interest. Applying optimal control theory, we investigate the problem of maximizing link velocity for visco-elastic joints. The main contribution of the paper is thus isolating the effects of mechanical joint damping on the optimal control policy.

Elastic robots have a distinct feature that makes them especially interesting to optimal control: their ability to mechanically store and release potential energy. However, solving any kind of optimal control problem for such highly nonlinear dynamics is feasible only numerically, i.e. offline. In turn, optimal solutions would only contribute a clear benefit for dynamic environments/tasks (apart from rather general insights), if they would be accessible/generalizable in real-time. In this paper, we propose a framework for executing near-optimal motions for elastic arms in real-time. We approach the problem as follows. First, we define a set of prototypical optimal control problems. These represent a reasonable set of motions that an intrinsically elastic robot arm is sought to execute. Exemplary, we solve the optimal control problem for some of these prototypes in a roughly covered task space. Then, we encode the resulting optimal trajectories in a dynamical system via Dynamic Movement Primitives (DMPs). Finally, a distance and cost function based metric forms the basis to generalize from the learned parameterizations to a new unsolved optimal control problem in real-time. In short, we intend to overcome the well known problems of optimal control and learning with associated generalization: being offline and being suboptimal, respectively.

Anthropomorphic robots that aim to approach human performance agility and efficiency are typically highly redundant not only in their kinematics but also in actuation. Variable-impedance actuators, used to drive many of these devices, are capable of modulating torque and impedance (stiffness and/or damping) simultaneously, continuously, and independently. These actuators are, however, nonlinear and assert numerous constraints, e.g., range, rate, and effort limits on the dynamics. Finding a control strategy that makes use of the intrinsic dynamics and capacity of compliant actuators for such redundant, nonlinear, and constrained systems is nontrivial. In this study, we propose a framework for optimization of torque and impedance profiles in order to maximize task performance, which is tuned to the complex hardware and incorporating real-world actuation constraints. Simulation study and hardware experiments 1) demonstrate the effects of actuation constraints during impedance control, 2) show applicability of the present framework to simultaneous torque and temporal stiffness optimization under constraints that are imposed by real-world actuators, and 3) validate the benefits of the proposed approach under experimental conditions.

This article presents a versatile controller that enables various contact tooling tasks with minimal prior knowledge of the tooled surface. The controller is derived from results of neuroscience studies that investigated the neural mechanisms utilized by humans to control and learn complex interactions with the environment. We demonstrate here the versatility of this controller in simulations of cutting, drilling and surface exploration tasks, which would normally require different control paradigms. We also present results on the exploration of an unknown surface with a 7-DOF manipulator, where the robot builds a 3D surface map of the surface profile and texture while applying constant force during motion. Our controller provides a unified control framework encompassing behaviors expected from the different specialized control paradigms like position control, force control and impedance control.

In contrast to common rigid or actively compliant systems, Variable Stiffness Arms are capable of storing potential energy in their joint and convert it into kinetic energy, respectively speed. This capability is well known from humans and is a good example for the outstanding performance of biological systems. However, only since some years intrinsic compliance is considered as a key feature and not a drawback in robot design. Therefore, only very little work has been carried out on exploiting the natural dynamics of elastic arms for such explosive motion sequences. In this paper, we treat the problem of how to optimally achieve maximum link velocity at a given final time for Variable Stiffness Arms. We show that solutions to this problem lead to excitation motions, which enable the robot to move on the link side at much higher speed than on the motor side. In particular, the robot uses the dynamic transfer of elastic joint energy into link side kinetic energy for further acceleration. In our work we consider the practically relevant input and state constraints, and give experimental verification of the developed methods on the new DLR Hand-Arm system.

Anthropomorphic robots that aim to approach human performance agility and efficiency are typically highly redundant not only in their kinematics but also in actuation. Variable-impedance actuators, used to drive many of these devices, are capable of modulating torque and passive impedance (stiffness and/or damping) simultaneously and independently. Here, we propose a framework for simultaneous optimisation of torque and impedance (stiffness) profiles in order to optimise task performance, tuned to the complex hardware and incorporating real-world constraints. Simulation and hardware experiments validate the viability of this approach to complex, state dependent constraints and demonstrate task performance benefits of optimal temporal impedance modulation.

Enabling robots to safely interact with humans is an essential goal of robotics research. The developments achieved over the last years in mechanical design and control made it possible to have active cooperation between humans and robots in rather complex situations. In these terms, safe behavior of the robot even under worst-case situations is crucial and forms also a basis for higher level decisional aspects. In order to quantify what safe behavior really means, the definition of injury, as well as understanding its general dynamics are essential. This insight can then be applied to design and control robots such that injury due to robot-human impacts is explicitly taken into account. In this paper we approach the problem from a medical injury analysis point of view in order to formulate the relation between robot mass, velocity, impact geometry, and resulting injury qualified in medical terms. We transform these insights into processable representations and propose a motion supervisor that utilizes injury knowledge for generating safe robot motions. The algorithm takes into account the reflected inertia, velocity, and geometry at possible impact locations. The proposed framework forms a basis for generating truly safe velocity bounds that explicitely consider the dynamic properties of the manipulator and human injury.

Intrinsically elastic robots, which technically implement some key characteristics of the human muskoskeletal system, have become a major research topic in nowadays robotics. These novel devices open up entirely new control approaches. They base on temporary storage of potential energy and its timed transformation into kinetic energy. In legged locomotion, such considerations have been a common tool for unveiling the respective fundamental physical processes. However, in arm control, elasticities were typically considered parasitic. In this video we outline our efforts in exploiting the inherent capabilities of intrinsically elastic robots in order to bring them closer to human performance. Instead of applying purely kinematic learing-by-demonstration approaches, which are certainly suboptimal, we argue for using model based techniques in order to optimally exploit the system dynamics such that highly dynamic motion and manipulation capabilities can be achieved. In particular, the explicit use of elasticities as temporary energy tanks can be fully exploited, if they are modeled adequately as an integral part of the mechanism. We also believe that such approaches can substantially contribute to the understanding of human motion biomechanics.

Designing intrinsically elastic robot systems, making systematic use of their properties in terms of impact decoupling, and exploiting temporary energy storage and release during excitative motions is becoming an important topic in nowadays robot design and control. In this paper we treat two distinct questions that are of primary interest in this context. First, we elaborate an accurate estimation of the maximum contact force during simplified human/obstacle-robot collisions and how the relation between reflected joint stiffness, link inertia, human/obstacle stiffness, and human/obstacle inertia affect it. Overall, our analysis provides a safety oriented methodology for designing intrinsically elastic joints and clearly defines how its basic mechanical properties influence the overall collision behavior. This can be used for designing safer and more robust robots. Secondly, we provide a closed form solution of reaching maximum link side velocity in minimum time with an intrinsically elastic joint, while keeping the maximum deflection constraint. This gives an analytical tool for determining suitable stiffness and maximum deflection values in order to be able to execute desired optimal excitation trajectories for explosive motions.

Intrinsically elastic joints have become increasingly popular over the last years. Commonly, they are considered to outperform rigid actuation in terms of peak dynamics, robustness, and energy efficiency. In particular, the possible increase of link speed by adequate motor excitation trajectories, such that the elastic transmission temporarily stores elastic energy and then timely converts it into kinetic link energy, is a new control problem in robotics. However, despite being a popular argument in favor of elastic actuation, it was not shown yet that this potential speed gain is truly inherent to the physical properties of the mechanism. In order to argue that “elasticity is superior to input torque”, i.e. size and weight, it still needs to be derived that this new feature does not come at the cost of increasing weight for a given actuation technology. Therefore, we analyze, under which circumstances “extracting” a certain amount of mass from a rigid joint and “investing” this into an elastic mechanism in the drive train leads to such a performance increase. For this, we derive the general scaling behavior of rigid joints and compare their capabilities in terms of maximum velocity to the performance behavior of an elastic joint, while taking into consideration the most important real-world constraints.

Most of today's robots have rigid structures and actuators requiring complex software control algorithms and sophisticated sensor systems in order to behave in a compliant and safe way adapted to contact with unknown environments and humans. By studying and constructing variable impedance actuators and their control, we contribute to the development of actuation units which can match the intrinsic safety, motion performance and energy efficiency of biological systems and in particular the human. As such, this may lead to a new generation of robots that can co-exist and co-operate with people and get closer to the human manipulation and locomotion performance than is possible with current robots.

An anthropomorphic hand arm system using variable stiffness actuation has been developed at DLR. It is aimed to reach its human archetype regarding size, weight and performance. The main focus of our development is put on robustness, dynamic performance and dexterity. Therefore, a paradigm change from impedance controlled, but mechanically stiff joints to robots using intrinsic variable compliance joints is carried out. Collisions of the rigid joint robot at high speeds with stiff objects induce the energy too fast for an active controller to prevent damages. In contrast, passively compliant robots are able to temporarily store energy. In this case the resulting internal forces applied to the robot structure and the drive trains are reduced. Furthermore, the energy storage allows to outperform the dynamics of stiff robots. The hand drives and the electronics are completely integrated within the forearm. Extremely miniaturized electronics have been developed to drive the 52 motors of the system and interface their sensors. Several variable stiffness actuation principles used in the arm joints and the hand are presented. The paper highlights the different requirements that they have to fulfill. A first test of the systems robustness and dynamics has been performed by driving nails with a grasped hammer and is demonstrated in the attached video.

Modeling of low severity soft-tissue injury due to unwanted collisions of a robot in collaborative settings is an important aspect to be treated in safe physical Human-Robot Interaction (pHRI). Up to now, safety evaluations for pHRI were mainly conducted by using safety criteria related with impact forces and head accelerations. These indicate severe injury in the robotics context and leave out low severity injury such as contusions and lacerations. However, for the design of an intrinsically safer robot arm, a reliable evaluation of the collision between a human and a robot that is based on skin injury criteria is essential. In this paper, we propose a novel human-robot collision model with and without covering, which is based on the impact stress distribution. The reliability of the proposed collision model is verified by a comparison with various cadaver experiments taken from existing biomechanical literature. Since the stress characteristics acting on the human head can be analyzed with this new collision model, the occurrence of certain soft-tissue injury can be estimated. Furthermore, the method serves for selecting the appropriate covering parameters, as e.g. elastic modulus and thickness, by evaluating the chosen skin injury indices.

In this paper we present a novel control architecture for realizing human-friendly behaviors and intuitive state based programming. The design implements strategies that take advantage of sophisticated soft-robotics features for providing reactive, robust, and safe robot actions in dynamic environments. Quick access to the various functionality of the robot enables the user to develop flexible hybrid state automata for programming complex robot behaviors. The real-time robot control takes care of all safety critical aspects and provides reactive reflexes that directly respond to external stimuli.

The increasing ability of industrial robots to perform complex tasks in collaboration with humans requires more capable ways of communication and interaction. Traditional systems use separate interfaces such as touchscreens or control panels in order to operate the robot, or to communicate its state and prospective actions to the user. Transferring human communication, such as gestures to technical non-humanoid robots, creates various opportunities for more intuitive human-robot-interaction. Interaction shall no longer require a separate interface such as a control panel. Instead, it should take place directly between human and robot. To explore intuitive interaction, we identified gestures that are relevant for co-working tasks from human observations. Based on a decomposition approach we transferred them to robotic systems of increasing abstraction and experimentally evaluated how well these gestures are recognized by humans. We created a human-robot interaction use-case in order to perform the task of handling dangerous liquid. Results indicate that several gestures are well perceived when displayed with context information regarding the task.

For achieving dynamic manipulation capabilities that are comparable to human performance in terms of speed, energetic properties, and robustness, intrinsic elasticity is widely proposed as a necessary robot design element. In this paper we show how passive compliance can be exploited for a 6-degree-of-freedom (DoF) cyclic ball dribbling task with a 7-DoF articulated Cartesian impedance controlled DLR Lightweight Robot III. For this, the robot is equipped with an elastic hand, which extends the contact time and therefore, also enlarges both, observability and controllability of the ball. We show via simulation and experiment that it is possible to achieve a stable dynamic cycle based on a 1 DoF analysis from [1] for the main axis together with control strategies for the secondary translations and rotations of the task. The scheme allows also the continuous tracking of a desired dribbling height and horizontal position. As a human is able to dribble blindly, we decided to solve the task by force sensing only, i.e. no vision is used for our approach, however, it could be easily incorporated.

This paper presents a novel human-like learning controller to interact with unknown environments. Strictly derived from the minimization of instability, motion error, and effort, the controller compensates for the disturbance in the environment in interaction tasks by adapting feedforward force and impedance. In contrast with conventional learning controllers, the new controller can deal with unstable situations that are typical of tool use and gradually acquire a desired stability margin. Simulations show that this controller is a good model of human motor adaptation. Robotic implementations further demonstrate its capabilities to optimally adapt interaction with dynamic environments and humans in joint torque controlled robots and variable impedance actuators, without requiring interaction force sensing.

After briefly summarizing the mechanical design of the two joint prototypes for the new DLR variable compliance arm, the paper exemplifies the dynamic modelling of one of the prototypes and proposes a generic variable stiffness joint model for nonlinear control design. Based on this model, the design of a simple, gain scheduled state feedback controller for active vibration damping of the mechanically very weakly damped joint is presented. Moreover, the computation of the motor reference values out of the desired stiffness and position is addressed. Finally, simulation and experimental results validate the proposed methods.

In this paper we introduce a classification of intrinsically compliant joint mechanisms. Furthermore, we outline design considerations for realizing such devices in order to match the requirements for robust and performant actuation. Based on this elaboration, a new design concept is presented, the DLR QA-Joint. Its performance is investigated by various experiments, covering velocity increase using the elastic energy, joint protection capabilities, and control performance.

Up to now, mostly blunt human-robot impacts were investigated in the robotics literature. In this context, the influence of robot mass and velocity during rigid impacts with and without the possibility of the human being clamped was quantified. In this paper an analysis of soft-tissue injuries caused by sharp tools, which are mounted on/grasped by a robot is carried out as the next step down the road to a full safety analysis of robots for HRI. We conducted an analysis of soft-tissue injuries based on available biomechanical and forensic data and to our knowledge for the first time in robotics present various experimental results with biological tissue for validation. Furthermore, possible countermeasures are evaluated quantitatively based biomechanically relevant quantities.

In this paper we present the synchronization and driver architecture of the DLR LWR-III, which supplies an easy to use interface for applications. For our purpose we abstracted the robot hardware entirely from the control algorithms using the common device driver concept of modern operating systems. The software architecture is split into two modular parts. On the one side, there are device drivers that communicate with the hardware components. On the other side, there are realtime applications realized as Simulink Models, which provide advanced control algorithms. This ensures a clean separation between the two modules and provides a communication over a common and approved interface. Furthermore we investigated how we can ensure synchronization to the hardware over the device driver interfaces and how we can ensure that it meets hard realtime requirements. The main result of this paper is to realize a synchronization between LWR-III hardware and Simulink control applications while targeting small latencies with respect to hard realtime requirements. The design is implemented and verified on WindRiver™VxWorks™.

Because bin-picking effectively mirrors great challenges in robotics, it has been a relevant robotic showpiece application for several decades. In this paper we describe the computer vision algorithms in combination with the sophisticated control schemes of the robot and demonstrate a reliable and robust solution to the chosen problem. This paper approaches the bin-picking issue by applying the latest state-of-the-art hardware components, namely an impedance controlled lightweight robot and a Time-of-Flight camera. Lightweight robots have gained new capabilities in both sensing and actuation without suffering a decrease in speed and payload. Time-of- Flight cameras are superior to common proximity sensors in the sense that they provide depth and intensity images in video frame rate independent of textures. The bin-picking solution presented in this paper aims at extending the classical bin-picking problem by incorporating an environment model and allowing for the physical human-robot interaction during the entire process. Existing imprecisions in Time-of-Flight camera measurements and environment uncertainties are compensated by the compliant behavior of the robot. The overall process is implemented in a generic state machine that also monitors the entire bin-picking process.

In this paper we present various new insights on the effect intrinsic joint elasticity has on safety in pHRI. We address the fact that the intrinsic safety of elastic mechanisms has been discussed rather one sided in favor of this new designs and intend to give a more differentiated view on the problem. An important result is that intrinsic joint elasticity does not reduce the Head Injury Criterion or impact forces compared to conventional actuation with some considerable elastic behavior in the joint, if considering full scale robots. We also elaborate conditions under which intrinsically compliant actuation is potentially more dangerous than rigid one. Furthermore, we present collision detection and reaction schemes for such mechanisms and verify their effectiveness experimentally.

In this overview paper we present current work on safety analysis for physical Human-Robot Interaction (pHRI) and motion control methods for robotic co-workers. In particular, we introduce the analysis tools for investigating the potential injury a human would suffer during robot-human impacts. Furthermore, we outline our concept for establishing a procedure towards standardized crash testing in robotics with automobile crash-test dummies. Since it is only possible to investigate blunt impacts with these devices, we developed a drop testing setup for analyzing soft-tissue injury in robotics from a biomechanics perspective. In the second part of the paper, some of our methods for task preserving and task relaxing motion schemes are described, which enable collision avoidance in real-time. The algorithms are well suited to work in an integrated fashion with the soft robotics control developed for the DLR Lightweight Robot III (LWR-III). In addition, it is shown how the torque sensing capabilities of the robot can be used to support reactive motion schemes. Finally, an overview of our human-friendly control architecture for the LWR-III is given, which unifies the rich bundle of developed methods for this manipulator.

This paper describes a novel method for motion generation and reactive collision avoidance. The algorithm performs arbitrary desired velocity profiles in absence of external disturbances and reacts if virtual or physical contact is made in a unified fashion with a clear physically interpretable behavior. The method uses physical analogies for defining attractor dynamics in order to generate smooth paths even in presence of virtual and physical objects. The proposed algorithm can, due to its low complexity, run in the inner most control loop of the robot, which is absolutely crucial for safe Human Robot Interaction. The method is thought as the locally reactive real-time motion generator connecting control, collision detection and reaction, and global path planning.

After giving a rich data basis of our impact tests with standardized crash-test dummies in Part I of this work we address in Part II various aspects related to these tests in a case based discussion. The presented facts, the knowledge gained from our previous work, and the data from Part I lead us to recommendations for standardized crash-testing procedures in robotics. The proposed impact procedures will help to compare blunt robot-human impacts on a common basis. We will discuss additional requirements which will enhance the completeness of testing procedures.

After analyzing fundamental impact characteristics of robot-human collisions in our previous work, the intention in the present paper is to augment existing knowledge in this field, verify previously given statements with standardized equipment of the German Automobile Club (ADAC), and provide a crash-test report for robots in general. Various new insights are achieved and a systematic and extensive set of data is provided. The presented work is divided into two papers. The main purpose of Part I is to give, similarly to reports known from the automobile world1, a fact based and result oriented view on our newest robot crash-test experiments. In Part II detailed discussions of the results listed in the present paper and recommendations towards a standard crash-test protocol for robot safety are carried out.

Accidents occurring with classical industrial robots often lead to fatal injuries. Presumably, this is to a great extent caused by the possibility of clamping the human in the confined workspace of the robot. Before generally allowing physical cooperation of humans and robots in future applications it is therefore absolutely crucial to analyze this extremely dangerous situation. In this paper we will investigate many aspects relevant to this sort of injury mechanisms and discuss the importance to domestic environments or production assistants. Since clamped impacts are intrinsically more dangerous than free ones it is fundamental to discuss and evaluate metrics to ensure safe interaction if clamping is possible. We compare various robots with respect to their injury potential leading to a main safety requirement of robot design: Reduce the intrinsic injury potential of a robot by reducing its weight.

The desired coexistence of robotic systems and humans in the same physical domain, by sharing their workspace and actually cooperating in a physical manner, poses the very fundamental problem of ensuring safety to the user. In this paper we will show the influence of robot mass and velocity during blunt unconstrained impacts with humans. Several robots with weights ranging from 15-2500 kg are impacted at different velocities with a mechanical human head mockup. This is used to measure the so-called head injury criterion, mainly a measure for brain injury. Apart from injuries indicated by this criterion and a detailed analysis of chest impacts we point out that e.g. fractures of facial bones can occur during collisions at typical robot velocities. Therefore, this injury mechanism which is more probable in robotics is evaluated in detail.

Currently, large efforts are undertaken to bring robotic applications to domestic environments. Especially physical human-robot cooperation is a major concern and various design and control methodologies were developed on the way to achieve this task. In particular, this necessitates the evaluation of injury risks a human is exposed to in case he is hit by a robot. In this video several blunt impact tests are shown, leading to an assessment of which factors dominate injury severity. We will illustrate the effect robot speed, robot mass, and constraints in the environment have on safety in human- robot impacts. It will be shown that the intuition of high impact loads being transmitted by heavy robots is wrong. Furthermore, the conclusion is induced that free impacts are by far less dangerous than being crushed.

In the framework of physical Human-Robot Interaction (pHRI), methodologies and experimental tests are presented for the problem of detecting and reacting to collisions between a robot manipulator and a human being. Using a lightweight robot that was especially designed for interactive and cooperative tasks, we show how reactive control strategies can significantly contribute to ensuring safety to the human during physical interaction. Several collision tests were carried out, illustrating the feasibility and effectiveness of the proposed approach. While a subjective “safety” feeling is experienced by users when being able to naturally stop the robot in autonomous motion, a quantitative analysis of different reaction strategies was lacking. In order to compare these strategies on an objective basis, a mechanical verification platform has been built. The proposed collision detection and reactions methods prove to work very reliably and are effective in reducing contact forces far below any level which is dangerous to humans. Evaluations of impacts between robot and human arm or chest up to a maximum robot velocity of 2.7 m/s are presented.

This video presents a humanoid two-arm system developed as a research platform for studying dexterous two-handed manipulation. The system is based on the modular DLR-Lightweight-Robot-III and the DLR-Hand-II. Two arms and hands are combined with a three degrees-of-freedom movable torso and a visual system to form a complete humanoid upper body. The diversity of the system is demonstrated by showing the mechanical design, several control concepts, the application of rapid prototyping and hardware-in-the-loop (HIL) development as well as two-handed manipulation experiments and the integration of path planning capabilities.

A robot manipulator sharing its workspace with humans should be able to quickly detect collisions and safely react for limiting injuries due to physical contacts. In the absence of external sensing, relative motions between robot and human are not predictable and unexpected collisions may occur at any location along the robot arm. Based on physical quantities such as total energy and generalized momentum of the robot manipulator, we present an efficient collision detection method that uses only proprioceptive robot sensors and provides also directional information for a safe robot reaction after collision. The approach is first developed for rigid robot arms and then extended to the case of robots with elastic joints, proposing different reaction strategies. Experimental results on collisions with the DLR-III lightweight manipulator are reported