Robotic assembly tasks remain an open challenge due to their long horizon nature and complex part relations. Behavior trees (BTs) are increasingly used in robot task planning for their modularity and flexibility, but creating them manually can be effort-intensive. Large language models (LLMs) have recently been applied to robotic task planning for generating action sequences, yet their ability to generate BTs has not been fully investigated. To this end, we propose LLM-as-BT-Planner, a novel framework that leverages LLMs for BT generation in robotic assembly task planning. Four in-context learning methods are introduced to utilize the natural language processing and inference capabilities of LLMs for producing task plans in BT format, reducing manual effort while ensuring robustness and comprehensibility. Additionally, we evaluate the performance of fine-tuned smaller LLMs on the same tasks. Experiments in both simulated and real-world settings demonstrate that our framework enhances LLMs' ability to generate BTs, improving success rate through in-context learning and supervised fine-tuning.

The increasing integration of robots in close human environments necessitates robust safety measures that can adapt to evolving tasks and conditions. Current standards rely on task-specific safety evaluations that are often inflexible, requiring repeated assessments whenever task parameters change. This work proposes MonLog, a data-driven, probabilistic method to automatically derive safety curves (SCs) from recent injury protection data sets. By leveraging non-linear modeling techniques, our approach addresses the limitations of conventional linear SCs, which often result in overly conservative speed restrictions. We present a comprehensive test routine to validate our method, highlighting improvements in both compliance with safety constraints and operational efficiency. Our findings demonstrate that the proposed approach not only enhances safety but also optimizes robotic performance, making it suitable for a wide range of applications.

Robot manipulation researchers reference human performance as a goal for their work, however, human data is seldom present in robotics benchmarks. We introduce a real-world benchmark targeting manipulation skills for performing electrical circuit inspection with a multimeter using an Internet-connected electronic task board. We present timing study results and an exemplary robot solution across six different tasks from the Robothon Grand Challenge at the automatica conference in 2023. Contributions from 16 robot teams were collected using task boards we manufactured and distributed as part of the 30-day international competition as an initial performance database. Our work systematically highlights the skill gap between the winning robot solution and the best human performance from a group of 30 subjects. Our goal is to chronicle progress over time in robot manipulation skills and provide a standardized, physical benchmark across the global community. Videos of the team submissions, the exemplary robot solution, as well as the project reproduction code are provided in the included repository.

Assembly is a crucial skill for robots in both modern manufacturing and service robotics. However, mastering transferable insertion skills that can handle a variety of high-precision assembly tasks remains a significant challenge. This paper presents a novel framework that utilizes diffusion models to generate 6D wrench for high-precision tactile robotic insertion tasks. It learns from demonstrations performed on a single task and achieves a zero-shot transfer success rate of 95.7% across various novel high-precision tasks. Our method effectively inherits the self-adaptability demonstrated by our previous work. In this framework, we address the frequency misalignment between the diffusion policy and the real-time control loop with a dynamic system-based filter, significantly improving the task success rate by 9.15%. Furthermore, we provide a practical guideline regarding the trade-off between diffusion models' inference ability and speed.

Learning from Demonstration (LfD) techniques enable robots to learn and generalize tasks from user demonstrations, eliminating the need for coding expertise among end-users. One established technique to implement LfD in robots is to encode demonstrations in a stable Dynamical System (DS). However, finding a stable dynamical system entails solving an optimization problem with bilinear matrix inequality (BMI) constraints, a non-convex problem which, depending on the number of scalar constraints and variables, demands significant computational resources and is susceptible to numerical issues such as floating-point errors. To address these challenges, we propose a novel compositional approach that enhances the applicability and scalability of learning stable DSs with BMIs.

This paper presents a novel approach for robot navigation in environments containing deformable obstacles. By integrating Learning from Demonstration (LfD) with Dynamical Systems (DS), we enable adaptive and efficient navigation in complex environments where obstacles consist of both soft and hard regions. We introduce a dynamic modulation matrix within the DS framework, allowing the system to distinguish between traversable soft regions and impassable hard areas in real-time, ensuring safe and flexible trajectory planning. We validate our method through extensive simulations and robot experiments, demonstrating its ability to navigate deformable environments. Additionally, the approach provides control over both trajectory and velocity when interacting with deformable objects, including at intersections, while maintaining adherence to the original DS trajectory and dynamically adapting to obstacles for smooth and reliable navigation.

A drawing robot avatar is a robotic system that allows for telepresence-based drawing, enabling users to remotely control a robotic arm and create drawings in real-time from a remote location. The proposed control framework aims to improve bimanual robot telepresence quality by reducing the user workload and required prior knowledge through the automation of secondary or auxiliary tasks. The introduced novel method calculates the near-optimal Cartesian end-effector pose in terms of visual feedback quality for the attached eye-to-hand camera with motion constraints in consideration. The effectiveness is demonstrated by conducting user studies of drawing reference shapes using the implemented robot avatar compared to stationary and teleoperated camera pose conditions. Our results demonstrate that the proposed control framework offers improved visual feedback quality and drawing performance.

Intrinsically elastic robots surpass their rigid counterparts in a range of different characteristics. By temporarily storing potential energy and subsequently converting it to kinetic energy, elastic robots are capable of highly dynamic motions even with limited motor power. However, the time-dependency of this energy storage and release mechanism remains one of the major challenges in controlling elastic robots. A possible remedy is the introduction of locking elements (i.e. clutches and brakes) in the drive train. This gives rise to a new class of robots, so-called clutched-elastic robots (CER), with which it is possible to precisely control the energy-transfer timing. A prevalent challenge in the realm of CERs is the automatic discovery of clutch sequences. Due to complexity, many methods still rely on pre-defined modes. In this paper, we introduce a novel contact-implicit scheme designed to optimize both control input and clutch sequence simultaneously. A penalty in the objective function ensures the prevention of unnecessary clutch transitions. We empirically demonstrate the effectiveness of our proposed method on a double pendulum equipped with two of our newly proposed clutch-based Bi-Stiffness Actuators (BSA).

Established techniques that enable robots to learn from demonstrations are based on learning a stable dynamical system (DS). To increase the robots’ resilience to perturbations during tasks that involve static obstacle avoidance, we propose incorporating barrier certificates into an optimization problem to learn a stable and barrier-certified DS. Such optimization problem can be very complex or extremely conservative when the traditional linear parameter-varying formulation is used. Thus, different from previous approaches in the literature, we propose to use polynomial representations for DSs, which yields an optimization problem that can be tackled by sum-of-squares techniques. Finally, our approach can handle obstacle shapes that fall outside the scope of assumptions typically found in the literature concerning obstacle avoidance within the DS learning framework. Supplementary material can be found at the project webpage: https://martinschonger.github.io/abc-ds

The basis for robotics skill learning is an adequate representation of manipulation tasks based on their physical properties. As manipulation tasks are inherently invariant to the choice of reference frame, an ideal task representation would also exhibit this property. Nevertheless, most robotic learning approaches use unprocessed, coordinate-dependent robot state data for learning new skills, thus inducing challenges regarding the interpretability and transferability of the learned models.In this paper, we propose a transformation from spatial measurements to a coordinate-invariant feature space, based on the pairwise inner product of the input measurements. We describe and mathematically deduce the concept, establish the task fingerprints as an intuitive image-based representation, experimentally collect task fingerprints, and demonstrate the usage of the representation for task classification. This representation motivates further research on data-efficient and transferable learning methods for online manipulation task classification and task-level perception.

Non-prehensile transportation of unstable objects presents a challenging task in robotics. To ensure the success of the transportation, it is necessary to consider both the object’s stability via contact dynamics and the motion constraints of the robot. We propose two novel trajectory planning methods derived from sampling and dynamic programming algorithms, tested on a 7-DoF Franka Emika robot against common strategies like Model Predictive Control (MPC) and S-curve planning, particularly under the constraint of a non-rotating tray. The results demonstrate the effectiveness of our methodologies in improving transportation speed. This research contributes to advancements in robotic manipulation techniques by tackling non-prehensile manipulation of dynamically unstable objects.

Recent advancements in artificial hand development have primarily concentrated on enhancing adaptive grasping, dexterity, as well as the integration of biomimetic skin. However, few designs have successfully combined lightweight, cost-effective solutions, and tactile sensing along with adaptive grasping in a human-sized prototype. We propose, an open-source, highly integrated artificial hand. It leverages a compliant linkage mechanism for versatile grasping capabilities, featuring six degrees of actuation and MEMS-based tactile sensors on every fingertip.

Mobile manipulators (MM) have proven valuable in assisting humans in industrial settings. However, their strict separation from humans in controlled environments limits their effectiveness. Efforts have been made to bridge this gap for physical human-robot interaction (pHRI), leading to the development of collaborative mobile manipulators (CMM). Nonetheless, unpredictable environments continue to present challenges. This paper introduces an innovative suspension design for mobile bases (MBs) to enhance the safety and autonomy of CMMs. We propose an electromechanical approach leveraging variable stiffness and combining passive springs with adaptive transmission mechanisms. Through simulation, physical prototype development, and experimental validation, we demonstrate the effectiveness of our approach in stabilizing the MB against external disturbances. Our findings provide valuable insights for the development of CMMs in dynamic environments.

One of the most important aspects of autonomous systems is safety. This includes ensuring safe human-robot and safe robot-environment interaction when autonomously performing complex tasks or in collaborative scenarios. Al-though several methods have been introduced to tackle this, most are unsuitable for real-time applications and require carefully handcrafted obstacle descriptions. In this work, we propose a method combining high-frequency and real-time self and environment collision avoidance of a robotic manipulator with low-frequency, multimodal, and high-resolution environmental perceptions accumulated in a digital twin system. Our method is based on geometric primitives, so-called primitive skeletons. These, in turn, are information-compressed and real-time compatible digital representations of the robot's body and environment, automatically generated from ultra-realistic virtual replicas of the real world provided by the digital twin. Our approach is a key enabler for closing the loop between environment perception and robot control by providing the millisecond real-time control stage with a current and accurate world description, empowering it to react to environmental changes. We evaluate our whole-body collision avoidance on a 9-DOFs robot system through five experiments, demonstrating the functionality and efficiency of our framework.

Nowadays, wheeled mobile robots constitute a considerable portion of robots in industrial applications. Generally, regardless of their purpose, these systems are not designed to physically interact with humans, other robots, or the environment. In this study, we present a novel safe autonomous mobile - SAM - robot, which is a torque-controlled compliant robot that is conceived for safe human-robot interaction. This work provides an overview of the development philosophy of the system, its mechanical and mechatronics structure along with control and navigation architecture. Preliminary results show the advantages of the proposed mobile robot while interacting with its surroundings. We believe that this study will bring the wheeled mobile robots one step closer to the proactive interaction with their environment and humans surrounding them.

Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators, this problem has been partially addressed. With a suitable optimal control strategy, the approximate decoupling of the motor from the link can be achieved to maximize the energy transfer into the distal link prior to launch. However, such continuous stiffness variation is complex and typically leads to oscillatory swing-up motions instead of clear launch sequences. To circumvent this issue, we investigate decoupling for speed maximization with a dedicated novel actuator concept denoted Bi-Stiffness Actuation. With this, it is possible to fully decouple the link from the joint mechanism by a switch-and-hold clutch and simultaneously keep the elastic energy stored. We show that with this novel paradigm, it is not only possible to reach the same optimal performance as with power-equivalent variable stiffness actuation, but even directly control the energy transfer timing. This is a major step forward compared to previous optimal control approaches, which rely on optimizing the full time-series control input.

This paper addresses the problem of real-time planning in constrained dual-arm manipulation scenarios. Our proposed coupling leverages manipulability information of the cooperative bimanual task-space to a vector-field based planner by means of a repulsive circulatory field, while geometric primitives in Spin(3)⋉ℝ3are explored for flexible task relaxation. Furthermore, the circular field informs the cooperative framework about the safety boundaries which are in turn used to further relax motion constraints within a collision-free ball in Cartesian space. This builds a funnel along the trajectory which can be directly tracked through the proposed switching of task-primitive-priorities. The switching strategy follows an approach that ensures robustness to chattering and continuity in the joint-space. Experiments verify that our framework can run within the inner control loop of Franka Emika Panda robots.

The fast growth of communication technologies such as 5G provides high bandwidth and low latency wireless internet access. This enables both high definition video stream and real-time robot commands transmitted between robots and operators in the context of telepresence and teleoperation. Although there has been substantial research to establish algorithms that convert images to robot motions and telerobotic systems, little effort was made in establishing a clear scheme that enable artists to draw portraits using telerobotic systems. In this paper, we provide an easy-to-follow structure and implementation of a robot avatar for portrait drawing by artists through remote manipulation. The proposed telerobotic system uses a digital tablet and motion capture suit as input devices, which provides accurate drawing and continuous motion data stream respectively. With sensor fusion of the input data on the robot side, the drawing process presented in this work uses a unified force and impedance controller to ensure smooth and uniform pen-strokes. The proposed scheme was used to synthesise a system that was used by an artist to successfully finish the portrait drawing of Elon Musk. Finally, we show the effectiveness of the introduced control framework through an experiment. In particular, we validate the benefit of combining unified force and impedance control with sensor fusion of the digital tablet and motion capture suit data.

Similar to human work, robotic tasks sometimes require two hands to be accomplished. This requires coordinated motion planning and control. While fulfilling the task in a coordinated manner is already a big challenge, the task at hand becomes even harder when obstacles are introduced in the environment that need to be avoided. Furthermore in the case of dynamic environments, contacts cannot be avoided all the time, even with robust planning. In addition to geometric constraints, bimanual systems need to be able to detect and react to contacts during task execution. To this aim, we integrate a vector-field based planning scheme, that is able to avoid obstacles, with contact detection and reactive control methods based on contact wrench estimation such as admittance control. We also fuse the real contact forces into the planner directly together with the circular repulsive fields. The resulting planner-controller combination is capable of obstacle avoidance planning as well as reaction control in the case of unforeseen contacts that can also be used in situations where the manipulation needs to be guided by the environment such as landing control in only roughly known environments. We evaluate our approach on the torque-controlled Kobo bimanual set-up and also perform rigorous simulation studies.

In this paper, we provide a compositional framework for synthesizing hybrid controllers for interconnected discrete-time control systems enforcing specifications expressed by co-Büchi automata. In particular, we first decompose the given specification to simpler reachability tasks based on automata representing the complements of original co-Büchi automata. Then, we provide a systematic approach to solve those simpler reachability tasks by computing cor-responding control barrier functions. We show that such control barrier functions can be constructed compositionally by assuming some small-gain type conditions and composing so-called local control barrier functions computed for subsystems. We provide two systematic techniques to search for local control barrier functions for subsystems based on the sum-of-squares optimization program and counter-example guided inductive synthesis approach. Finally, we illustrate the effectiveness of our results through two large-scale case studies.

No abstract available.