In continuous hot-dip galvanizing, there is a strong economic incentive to reduce zinc consumption by improving coating weight control. In this context, the long transport delay between the jet-wiping process and coating weight measurements is a major challenge and motivates the use of model-based control strategies, e.g., feedforward control or internal model control. These strategies rely heavily on the accuracy of the underlying model. Thus far, most control applications have used a simple power law model of the coating weight. However, the power law typically cannot cover the entire operating range of a hot-dip galvanizing plant with a single set of parameters. The power law is a simplified version of a physics-based model and usually represents a good local approximation of the jet-wiping process. Therefore, this work explores integrating the power law into control-oriented machine learning models. It is shown that the physics-based knowledge of the process can be systematically incorporated into both a neural network and a hierarchical local model tree (HILOMOT). The proposed physics-informed models are then compared to existing models from the literature, using measurement data from an industrial hot-dip galvanizing plant. This comparison demonstrates how incorporating the power law can enhance a machine learning model both in prediction accuracy and the number of parameters required, with the physics-informed HILOMOT model outperforming all other considered models.

Forestry machines operated in forest production environments face challenges when performing manipulation tasks, especially regarding the complicated dynamics of underactuated crane systems and the heavy weight of logs to be grasped. This study investigates the feasibility of using reinforcement learning for forestry crane manipulators in grasping and lifting heavy wood logs autonomously. We first build a simulator using Mujoco physics engine to create realistic scenarios, including modeling a forestry crane with 8 degrees of freedom from CAD data and wood logs of different sizes. We further implement a velocity controller for autonomous log grasping with deep reinforcement learning using a curriculum strategy. Utilizing our new simulator, the proposed control strategy exhibits a success rate of 96% when grasping logs of different diameters and under random initial configurations of the forestry crane. In addition, reward functions and reinforcement learning baselines are implemented to provide an open-source benchmark for the community in large-scale manipulation tasks. A video with several demonstrations can be seen at https://www.acin.tuwien.ac.at/en/d18a/.

Human-robot interaction requires robots to process language incrementally, adapting their actions in real-time based on evolving speech input. Existing approaches to language-guided robot motion planning typically assume fully specified instructions, resulting in inefficient stop-and-replan behavior when corrections or clarifications occur. In this paper, we introduce a novel reasoning-based incremental parser which integrates an online motion planning algorithm within the cognitive architecture. Our approach enables continuous adaptation to dynamic linguistic input, allowing robots to update motion plans without restarting execution. The incremental parser maintains multiple candidate parses, leveraging reasoning mechanisms to resolve ambiguities and revise interpretations when needed. By combining symbolic reasoning with online motion planning, our system achieves greater flexibility in handling speech corrections and dynamically changing constraints. We evaluate our framework in real-world human-robot interaction scenarios, demonstrating online adaptions of goal poses, constraints, or task objectives. Our results highlight the advantages of integrating incremental language understanding with real-time motion planning for natural and fluid human-robot collaboration. The experiments are demonstrated in the accompanying video at www.acin.tuwien.ac.at/42d5.

Automated visual quality inspection is a core topic of robotics and computer vision. In industrial applications, the CAD model of the object to be inspected is often known and can be used to generate appropriate sensor poses (viewpoints) from which to inspect the object’s surface and assure the quality of its geometry. Current approaches in this field generate optimal viewpoints by evaluating the geometric coverage but the photometric appearance of the object is usually not considered. This lack of photometric information results in a loss of crucial cues to establish actual visibility, especially when the object to inspect presents specular highlights (e.g., polished metal parts) and a complex geometry. In this paper, we propose integrating photometric information into the viewpoint evaluation to consider the object’s appearance. To achieve this, we embed a bidirectional reflectance distribution function (BRDF) within the evaluation of the viewpoint candidates. We benchmark different BRDFs with increasingly realistic rendering to prove the concept of our approach. Specifically, we consider the Blinn-Phong and Cook-Torrance reflectance models. Our simulation results demonstrate the suitability and importance of using a photometric approach that considers material properties and selects optimal viewpoints for specific materials.

Foundation models such as ChatGPT have made significant strides in robotic tasks due to their universal representation of real-world domains. In this paper, we leverage foundation models to tackle grasp detection, a persistent challenge in robotics with broad industrial applications. Despite numerous grasp datasets, their object diversity remains limited compared to real-world figures. Fortunately, foundation models possess an extensive repository of real-world knowledge, including objects we encounter in our daily lives. As a consequence, a promising solution to the limited representation in previous grasp datasets is to harness the universal knowledge embedded in these foundation models. We present Grasp-Anything, a new large-scale grasp dataset synthesized from foundation models to implement this solution. Grasp-Anything excels in diversity and magnitude, boasting 1M samples with text descriptions and more than 3M objects, surpassing prior datasets. Empirically, we show that Grasp-Anything successfully facilitates zero-shot grasp detection on vision-based tasks and real-world robotic experiments. Our dataset and code are available at https://airvlab.github.io/grasp-anything/.

In this paper, a numerically efficient flexible control scheme for the absolute accuracy of industrial robots is presented and experimentally validated. A model-based controller that leverages all typically available parameters is combined with an online path iterative learning controller (ILC). The ILC law is employed to compensate for the unknown residual error dynamics caused by elastic and transmission effects. The proposed approach combines several benefits, including the possibility of a continuous execution of trials, a straightforward generalization of the learned data to different execution speeds, and learning from partial trials. The experimental validations on a 6-axis industrial robot with a laser tracker absolute measurement system show a 95% improvement in absolute accuracy after two trials. When the laser tracker is removed, the learned feedforward controller can sustain the accuracy achieved even without trial-by-trial learning.

Permanent magnet linear synchronous motors (PMLSM) are gaining attention in high-precision production systems. Smooth and accurate motion is vital, specifically for applications that involve on-the-move processing. This necessitates highly precise position measurement of the shuttles moving along the curvilinear motor track. In this work, we address an anisotropic magnetoresistive sensor (AMR) based position sensor system, where tolerances in the mounting, the magnetization of the permanent magnets (PM), and the sensor electronics can yield intolerably high position errors if no calibration is performed. This paper proposes a novel, user-friendly, cost-effective online calibration method utilizing iterative learning. The method leverages an acceleration sensor mounted directly on the shuttles, making it readily deployable in final PMLSM setups on production sites. Experimental results from a test stand demonstrate a significant reduction in measurement errors, paving the way for smooth shuttle motion along the entire curvilinear track within the PMLSM.

Automatic visual quality inspection is pivotal in both computer vision and robotics. It plays a crucial role in manufacturing, where robotic systems are increasingly employed to enhance the speed and efficiency of visual quality assessments. Several inspection planning methodologies have been developed; however, they often address the inspection challenge from a singular perspective of robotics or computer vision. This work introduces a comprehensive approach that synergistically integrates principles from both domains. We present an innovative algorithm designed to generate optimal inspection poses by considering the interplay between the inspected object’s geometry and the kinematics of the robotic setup used for inspection. This is accomplished by taking advantage of the concept of visibility. The effectiveness of our algorithm is demonstrated through simulations and experiments, revealing complete coverage for diverse geometries and materials with a small number of inspection poses. Moreover, we benchmark our framework against box constraints and workspace sampling techniques to generate feasible inspection poses. The results indicate superior performance in achieving extensive coverage and reducing the number of required optimal inspection poses, enhancing the overall inspection process.

Learning from demonstration (LfD) has emerged as a promising approach enabling robots to acquire complex tasks directly from human demonstrations. However, tasks involving surface interactions on freeform 3D surfaces present unique challenges in modeling and execution, especially when geometric variations exist between demonstrations and robot execution. This paper proposes a novel framework called probabilistic surface interaction primitives (ProSIP), which systematically incorporates the surface path and the local surface features into the learning procedure. An instrumented tool allows seamless recording and execution of human demonstrations. By design, ProSIPs are independent of time, invariant to rigid-body displacements, and apply to any robotic platform with a Cartesian controller. The framework is employed for an edge-cleaning task of bathroom sinks. The generalization capability to various object geometries and significantly distorted objects is demonstrated. Simulations and an experimental setup with a 9-degrees-of-freedom robotic platform confirm the performance.

This work proposes a highly parallelized optimization scheme to simultaneously optimize the robot base and tool center point (TCP) placement within a robotic work cell for a sequence of pick-and-place tasks. The placement is optimized for minimum cycle time by considering the scenario holistically, including point-to-point trajectory planning while respecting the kinodynamic constraints of the robot, collision avoidance in highly constrained environments, redundancy in grasp configurations and inverse kinematic solutions, and the cyclic constraint of the process. The proposed algorithm is applied to optimize the robot base and TCP placements in a spatially constrained packaging scenario, demonstrating a cycle time reduction of 41% compared to state-of-the-art approaches. The results are validated experimentally using a KUKA LBR iiwa with 7 degrees of freedom, where the TCP placement is realized using topology optimization and 3D printing.

Real-time applications for autonomous operations depend largely on fast and robust vision-based localization systems. Since image processing tasks require processing large amounts of data, the computational resources often limit the performance of other processes. To overcome this limitation, traditional marker-based localization systems are widely used since they are easy to integrate and achieve reliable accuracy. However, classical marker-based localization systems significantly depend on standard cameras with low frame rates, which often lack accuracy due to motion blur. In contrast, event-based cameras provide high temporal resolution and a high dynamic range, which can be utilized for fast localization tasks, even under challenging visual conditions. This paper proposes a simple but effective event-based pose estimation system using active LED markers (ALM) for fast and accurate pose estimation. The proposed algorithm is able to operate in real time with a latency below 0.5 ms while maintaining output rates of 3 kHz. Experimental results in static and dynamic scenarios are presented to demonstrate the performance of the proposed approach in terms of computational speed and absolute accuracy, using the OptiTrack system as the basis for measurement. Moreover, we demonstrate the feasibility of the proposed approach by deploying the hardware, i.e., the event-based camera and ALM, and the software in a real quadcopter application. Our project page is available at: almpose.github.io

We present an inflatable finger pad that allows regular parallel-jaw grippers to vary their grasp stiffness while maintaining a contact force and contact to non-planar surfaces. An eversible radial bellows structure made of silicone rubber allows the pad to extend to four times its original height and to retract into a rigid pod when not needed. The bellows act as passive universal joints when everted, enabling aerial contact with surfaces inclined by up to 45 degree. The bellow geometry is intentionally nonlinear but avoids bistable configurations to facilitate control. We find that the nonlinear stiffness behavior allows a pair of opposing pads to increase the compliance of a grasp twenty fold, resulting in a total of two orders of magnitude difference between the most stiff and most compliant configuration. Crucially, high compliance can be achieved while exerting a contact force between 0.7 N and 5 N, allowing for compliant but firm grasps. Pads are manufactured using printed molds and sacrificial-mold casting.

Robots need to be as simple to use as tools in a workshop and allow non-experts to program, modify and execute tasks. In particular for repetitive tasks in high-mix/low-volume production, robotic support and physical human-robot interaction (pHRI) help to significantly increase productivity. In path-following control (PFC), the geometric description of the path is decoupled from the time evolution of the robot's end-effector along the path. PFC is inherently suitable for pHRI since path progress can be derived from the interaction with the human. In this work, an extension to multi-path PFC is proposed, which allows smooth transitions between the paths initiated by the human. Additionally, two pHRI modes called path snap-in and orientation snap-in are proposed, which use attractive forces to snap the robot end-effector onto a path or a predefined orientation. Moreover, the stability properties of PFC are inherited and the method is applicable to linear, nonlinear and self-intersecting paths. The proposed pHRI modes are validated on an experimental drilling task for teach-in (using orientation snap-in) and execution (using path snap-in) with the kinematically redundant collaborative robot Kuka Lbr iiwa 14 R820.

At first glance, a floating-base robotic system is a kinematic chain, and its equations of motion are described by the inertia-coupled dynamics of its shape and movable base. However, the dynamics embody an additional structure due to the momentum evolution, which acts as a velocity constraint. In prior works of robot dynamics, matrix transformations of the dynamics revealed a block-diagonal inertia. However, the structure of the transformed matrix of Coriolis/Centrifugal (CC) terms was not examined, and is the primary contribution of this article. To this end, we simplify the CC terms from robot dynamics and derive the analogous terms from geometric mechanics. Using this interdisciplinary link, we derive a two-part structure of the CC matrix, in which each partition is iteratively computed using a self-evident velocity dependency. Through this CC matrix, we reveal a commutative property, the velocity dependencies of the skew-symmetry property, the invariance of the shape dynamics to the basis of momentum, and the curvature as a matrix operator. Finally, we show the application of the proposed CC matrix structure through controller design and locomotion analysis.

In this paper, the experimental swing-up of a spherical pendulum mounted on a collaborative robot is presented. The complete mechanical system consists of nine degrees of freedom (DoFs). The primary focus of this work is the design of a fast trajectory planning for the swing-up by systematically incorporating the kinematic and dynamics constraints. The proposed algorithm consists of two steps: First, an offline trajectory optimization is used to build a database of swing-up trajectories, with an average computing time of 10 s for one trajectory. Second, a fast trajectory replanner based on a constrained quadratic program is described, which computes the swing-up trajectory for an arbitrary initial configuration of the system with an average computing time of 0.2 s. Simulations and experimental results demonstrate the swing-up of the spherical pendulum using a discrete time-variant linear quadratic regulator as a feedback controller.

The robot base placement of an industrial robot in flexible production lines is crucial due to the limited workspace of robots, in particular for complex continuous paths that change frequently. Costly and time-consuming repositioning of the robot can be avoided by merely adapting the tool center point (TCP) of the robot, which is the focus of this work. To this end, an algorithm for the optimal TCP placement for a set of tool paths is proposed. This algorithm is based on a fast joint-space path planner which is capable of moving through kinematic singularities and takes into account wide turning ranges of individual robot axes. Furthermore, the proposed concept also applies to the optimal robot base placement. The feasibility of the approach is demonstrated for a trim application in shoe production for a set of 44 complex continuous tool paths.

In this article, a novel approach for the versatile wrinkle-free application of (curved) precut adhesive tapes on freeform 3-D surfaces is presented. Straight and curved tape application paths are mapped onto the 3-D object as geodesics and as lines with imposed geodesic curvature, respectively. The proposed surface-based path following control concept extends the classical path following control by a novel parallel contact frame and a parallel projection operator. Using a static state feedback, the robotic system is transformed into a system with linear input-output behavior in the path coordinates. This allows to traverse a path on a 3-D object with a draping roll without turning around the surface normal vector. The latter prevents distortions and wrinkles of the applied tape. Experimental results with a Kuka LBR iiwa 14 R820 demonstrate the feasibility of the proposed approach.

Multi-phase electric machines are frequently used in applications where a high system reliability is required. An accurate but computationally simple mathematical model of these electric machines is essential for the development of model-based control and fault detection strategies. Typically, fundamental wave models (dq0-models) are utilized for this task. They, however, exhibit a low accuracy for motor designs or in operating ranges where magnetic saturation or nonfundamental wave characteristics is relevant. In this article, magnetic equivalent circuit (MEC) modeling is utilized to derive a highly accurate model for a dual three-phase permanent magnet synchronous motor (PMSM). Thereby, graph theory is applied to both the electric and magnetic system to systematically derive a mathematical model of minimum order. The proposed approach allows for an accurate prediction of the system behavior in the entire operating range, including the case of a winding short circuit. The feasibility and high accuracy of the proposed model is proven by a comparison of the model with measurement results.

The workspace of a robot can be restricted by virtual fixtures to assist an operator in physical human-robot interaction tasks. This paper introduces a combination of surface following control (SFC) with compliance control and presents a path/SFC approach to systematically generate virtual fixtures. This approach allows implementation of numerous types of constraints like guidance and forbidden region virtual fixtures, hard and soft constraints, as well as static and dynamic virtual fixtures, and their combinations. Additionally, closed-loop stability proofs of the proposed control concepts are given. The flexibility of the presented approach is demonstrated by a series of measurement results from an industrial robot.

This paper presents a path planning concept for trucks with trailers with kingpin hitching. This system is nonholonomic, has no flat output and is not stable in backwards driving direction. These properties are major challenges for path planning. The presented approach concentrates on the loading bay scenario. The considered task is to plan a path for the truck-trailer system from a start to a specified target configuration corresponding to the loading bay. Thereby, close distances to obstacles and multiple driving direction changes have to be handled. Furthermore, a so-called jackknife position has to be avoided. In a first step, an initial path is planned from the target to the start configuration using a tree-based path planner. Afterwards this path is refined locally by solving an optimal control problem. Due to the local nature of the planner, heuristic rules for direction changes are formulated. The performance of the proposed path planner is evaluated in simulation studies.

This paper presents a fast optimization based algorithm for car parking. The challenge arises from the non-holonomic characteristics of the car and the close distance to the obstacles. The presented approach utilizes the Minkowski sum to account for obstacle avoidance. The geometric path planning problem is decoupled from the kinematic problem and discretized with respect to the path parameter by means of a Runge-Kutta discretization. For the discrete path segments, an optimization problem is formulated to calculate the path independent of the parking scenario. This static optimization problem can be solved numerically in a very efficient way. The performance of the algorithm is evaluated in several simulation scenarios.

In this paper a compliance control law for kinematically redundant manipulators is proposed. The controller contains a Cartesian compliance part and a nullspace compliance part which are complemented by a power-conserving decoupling term. The approach deliberately avoids inertia shaping in order to obtain a control law which does not require the measurement of external forces and becomes less sensitive with respect to model uncertainties. While the controller formulation explicitly uses nullspace velocity coordinates, no integration of these velocities is required. Except for the kinematic singularities of the manipulator’s Jacobian matrix, no further algorithmic singularities are introduced. Asymptotic stability of the closed-loop system is shown by utilizing semi-definite Lyapunov functions. Finally, a short planar simulation study is presented which validates the effectiveness of the approach.

The current discussion on physical human robot interaction and the related safety aspects, but also the interest of neuro-scientists to validate their hypotheses on human motor skills with bio-mimetic robots, led to a recent revival of tendon-driven robots. In this paper, the modeling of tendon-driven elastic systems with nonlinear couplings is recapitulated. A control law is developed that takes the desired joint position and stiffness as input. Therefore, desired motor positions are determined that are commanded to an impedance controller. We give a physical interpretation of the controller. More importantly, a static decoupling of the joint motion and the stiffness variation is given. The combination of active (controller) and passive (mechanical) stiffness is investigated. The controller stiffness is designed according to the desired overall stiffness. A damping design of the impedance controller is included in these considerations. The controller performance is evaluated in simulation.

In this paper, a novel type of impedance controllers for flexible joint robots is proposed. As a target impedance, a desired stiffness and damping are considered without inertia shaping. For this problem, two controllers of different complexity are proposed. Both have a cascaded structure with an inner torque feedback loop and an outer impedance controller. For the torque feedback, a physical interpretation as a scaling of the motor inertia is given, which allows to incorporate the torque feedback into a passivity-based analysis. The outer impedance control law is then designed differently for the two controllers. In the first approach, the stiffness and damping terms and the gravity compensation term are designed separately. This outer control loop uses only the motor position and velocity, but no noncollocated feedback of the joint torques or link side positions. In combination with the physical interpretation of torque feedback, this allows us to give a proof of the asymptotic stability of the closed-loop system based on the passivity properties of the system. The second control law is a refinement of this approach, in which the gravity compensation and the stiffness implementation are designed in a combined way. Thereby, a desired static stiffness relationship is obtained exactly. Additionally, some extensions of the controller to viscoelastic joints and to Cartesian impedance control are given. Finally, some experiments with the German Aerospace Center (DLR) lightweight robots verify the developed controllers and show the efficiency of the proposed control approach.

In this paper a novel approach to the Cartesian impedance control problem for robots with flexible joints is presented. The proposed controller structure is based on simple physical considerations, which are motivating the extension of classical position feedback by an additional feedback of the joint torques. The torque feedback action can be interpreted as a scaling of the apparent motor inertia. Furthermore the problem of gravity compensation is addressed. Finally, it is shown that the closed loop system can be seen as a feedback interconnection of passive systems. Based on this passivity property a proof of asymptotic stability is presented.

This paper addresses the impedance control problem for flexible joint manipulators. An impedance controller structure is proposed, which is based on an exact decoupling of the torque dynamics from the link dynamics. A formal stability analysis of the proposed controller is presented for the general tracking case. Preliminary experimental results are given for a single flexible joint.