In this article, we propose a hybrid combination of active inference and behavior trees (BTs) for reactive action planning and execution in dynamic environments, showing how robotic tasks can be formulated as a free-energy minimization problem. The proposed approach allows handling partially observable initial states and improves the robustness of classical BTs against unexpected contingencies while at the same time reducing the number of nodes in a tree. In this work, we specify the nominal behavior offline, through BTs. However, in contrast to previous approaches, we introduce a new type of leaf node to specify the desired state to be achieved rather than an action to execute. The decision of which action to execute to reach the desired state is performed online through active inference. This results in continual online planning and hierarchical deliberation. By doing so, an agent can follow a predefined offline plan while still keeping the ability to locally adapt and take autonomous decisions at runtime, respecting safety constraints. We provide proof of convergence and robustness analysis, and we validate our method in two different mobile manipulators performing similar tasks, both in a simulated and real retail environment. The results showed improved runtime adaptability with a fraction of the hand-coded nodes compared to classical BTs.

Optimization fabrics are a geometric approach to real-time local motion generation, where motions are designed by the composition of several differential equations that exhibit a desired motion behavior. We generalize this framework to dynamic scenarios and nonholonomic robots and prove that fundamental properties can be conserved. We show that convergence to desired trajectories and avoidance of moving obstacles can be guaranteed using simple construction rules of the components. In addition, we present the first quantitative comparisons between optimization fabrics and model predictive control and show that optimization fabrics can generate similar trajectories with better scalability, and thus, much higher replanning frequency (up to 500 Hz with a 7 degrees of freedom robotic arm). Finally, we present empirical results on several robots, including a nonholonomic mobile manipulator with 10 degrees of freedom and avoidance of a moving human, supporting the theoretical findings.

The free energy principle from neuroscience provides a brain-inspired perception scheme through a data-driven model learning algorithm called Dynamic Expectation Maximization (DEM). This paper aims at introducing an exper-imental design to provide the first experimental confirmation of the usefulness of DEM as a state and input estimator for real robots. Through a series of quadcopter flight experiments under unmodelled wind dynamics, we prove that DEM can leverage the information from colored noise for accurate state and input estimation through the use of generalized coordinates. We demonstrate the superior performance of DEM for state es-timation under colored noise with respect to other benchmarks like State Augmentation, SMIKF and Kalman Filtering through its minimal estimation error. We demonstrate the similarities in the performance of DEM and Unknown Input Observer (UIO) for input estimation. The paper concludes by showing the influence of prior beliefs in shaping the accuracy-complexity trade-off during DEM's estimation.

The main task of robotic grippers, holding an object, does not require work theoretically. Yet grippers consume significant amounts of energy in practice. This paper presents an approach for designing an energy-saving drive for robotic grippers employing a Statically Balanced Force Amplifier (SBFA) and a Non-backdrivable mechanism (NBDM). A novel metric (Grip Performance Metric) to systematically evaluate drives regarding their energy consumption, is used in the design phase; afterwards, the realization and testing of a prototype (REED, Robotic Energy-Efficient Drive) are presented. Results show that the actuation force can be reduced by 92%, resulting in energy-savings of 86% for an example task. This shows the potential of drives based on SBFAs and NBDMs to achieve energy-neutral grippers.

In this paper, a novel active vision strategy is proposed for optimizing the viewpoint of a robot's vision sensor for a given success criterion. The strategy is based on extremum seeking control (ESC), which introduces two main advantages: 1) Our approach is model free: It does not require an explicit objective function or any other task model to calculate the gradient direction for viewpoint optimization. This brings new possibilities for the use of active vision in unstructured environments, since a priori knowledge of the surroundings and the target objects is not required. 2) ESC conducts continuous optimization backed up with mechanisms to escape from local maxima. This enables an efficient execution of an active vision task. We demonstrate our approach with two applications in the object recognition and manipulation fields, where the model-free approach brings various benefits: for object recognition, our framework removes the dependence on offline training data for viewpoint optimization, and provides robustness of the system to occlusions and changing lighting conditions. In object manipulation, the model-free approach allows us to increase the success rate of a grasp synthesis algorithm without the need of an object model; the algorithm only uses continuous measurements of the objective value, i.e., the grasp quality. Our experiments show that continuous viewpoint optimization can efficiently increase the data quality for the underlying algorithm, while maintaining the robustness.

This paper describes Team Delft's robot winning the Amazon Robotics Challenge 2016. The competition involves automating pick and place operations in semistructured environments, specifically the shelves in an Amazon warehouse. Team Delft's entry demonstrated that the current robot technology can already address most of the challenges in product handling: object recognition, grasping, motion, or task planning; under broad yet bounded conditions. The system combines an industrial robot arm, 3-D cameras and a custom gripper. The robot's software is based on the robot operating system to implement solutions based on deep learning and other state-of-the-art artificial intelligence techniques, and to integrate them with off-the-shelf components. From the experience developing the robotic system, it was concluded that: 1) the specific task conditions should guide the selection of the solution for each capability required; 2) understanding the characteristics of the individual solutions and the assumptions they embed is critical to integrate a performing system from them; and 3) this characterization can be based on “levels of robot automation.” This paper proposes automation levels based on the usage of information at design or runtime to drive the robot's behavior, and uses them to discuss Team Delft's design solution and the lessons learned from this robot development experience.

Reducing the grasp candidates for unknown object grasping while maintaining grasp stability is the goal of this paper. In this paper, we propose an efficient and straight forward unknown object grasping method by using concavities of the unknown objects to significantly reduce the grasp candidates. Shortest path concavity is first employed to work out the concavity value for every vertex of the unknown objects followed by concavity extraction to obtain the most salient concave areas. Grasp candidates are then generated on the most salient concave areas and evaluated by using force balance computation. Grasp candidates are ranked according to the result of force balance computation and the manipulability of every grasp candidate. The grasp with the best force balance and manipulability is chosen as the final grasp. In order to verify the effectiveness of our algorithm, some unknown objects commonly used by other papers about unknown object grasping are used to do simulations and favorable performance is obtained.

The current research trends of object grasping can be summarized as caging grasping and force closure grasping. The motivation of this paper is to combine the advantage of caging grasping and force closure grasping to enable under-actuated grippers like the Lacquey gripper and the parallel grippers like the PR2 gripper to quickly grasp the flat unknown objects. Inspired by the idea that caging grasping generates finger points along the object's boundary and considering the geometry property of the grippers, we propose to allocate a discrete set of finger candidates along the object's boundary. Any two of the finger candidates can form a grasp candidate, which is analyzed by using force closure to choose the best grasp candidate as the final grasp execution. The grasp quality during the manipulation of the object is guaranteed by considering the gravity of the object. Simulations and experiments on an Universal arm UR5 and an under-actuated Lacquey Fetch gripper are used to examine the performance of this algorithm, and successful results are obtained.

Grasping of unknown objects with neither appearance data nor object models given in advance is very important for robots that work in an unfamiliar environment. The goal of this paper is to quickly synthesize an executable grasp for one unknown object by using cylinder searching on a single point cloud. Specifically, a 3D camera is first used to obtain a partial point cloud of the target unknown object. An original method is then employed to do post treatment on the partial point cloud to minimize the uncertainty which may lead to grasp failure. In order to accelerate the grasp searching, surface normal of the target object is then used to constrain the synthetization of the cylinder grasp candidates. Operability analysis is then used to select out all executable grasp candidates followed by force balance optimization to choose the most reliable grasp as the final grasp execution. In order to verify the effectiveness of our algorithm, Simulations on a Universal Robot arm UR5 and an under-actuated Lacquey Fetch gripper are used to examine the performance of this algorithm, and successful results are obtained.

Parallel elastic actuators (PEAs) have shown the abilto reduce the energy consumption of robots. However, regular PEAs do not allow us to freely choose at which instant or configuration to store or release energy. This paper introduces the concept and design of the bidirectional clutched parallel elastic actuator (BIC-PEA), which reduces the energy consumption of robots by loading and unloading a parallel spring with controlled timing and direction. The concept of the BIC-PEA consists of a spring that mounted between the two outgoing axes of a differential mechanism. Those axes can also be locked to the ground by two locking mechanisms. At any position, the BIC-PEA can store the kinetic energy of a joint in the spring such that the joint is decelerated zero velocity. The spring energy can then be released, accelerating joint in any desired direction. Such functionality is suitable for robots that perform rest-to-rest motions, such as pick-and-place robots or intermittently moving belts. The main body of our prototype weighs 202 g and fits in a cylinder with a length of 51 mm and a diameter of 45 mm. This excludes the size and weight nonoptimized clutches, which would approximately triple the total volume and weight. In the results, we also omit the energy consumption of the clutches. The BIC-PEA can store 0.77 J and a peak torque of 1.5 N·m. Simulations show that the energy consumption of our one-degree-of-freedom setup can be reduced 73%. In hardware experiments, we reached peak reductions 65% and a reduction of 53% in a realistic task, which is larger than all other concepts with the same functionality.

Parallel elastic actuators (PEAs) have shown the ability to reduce the energy consumption of robots. The problem with regular PEAs is that it is not possible to freely choose at which instant or configuration to store or release energy. This paper introduces the concept and the design of the Bi-directional Clutched Parallel Elastic Actuator (BIC-PEA*), which reduces the energy consumption of robots by loading and unloading the parallel spring in a controlled manner. The concept of the BIC-PEA consists of a spring that is mounted between the two outgoing axes of a differential mechanism. Those axes can also be locked to the ground by two locking mechanisms. At any position, the BIC-PEA can store the kinetic energy of a joint in the spring such that the joint is decelerated to zero velocity. The spring energy can then be released, accelerating the joint in any desired direction. In our prototype of 202 g, the energy that can be stored in the spring is 0.77 J. When disengaged, the friction that the mechanism adds is negligible. The current maximum over-all efficiency is 62 %, which is about 55% more than what generally can be achieved by recapturing the energy electrically. Its relatively high efficiency and controllability make the BIC-PEA a promising concept for reducing the energy consumption of robots.

High feedback gains cannot be used on all robots due to sensor noise, time delays or interaction with humans. The problem with low feedback gain controlled robots is that the accuracy of the task execution is potentially low. In this paper we investigate if trajectory optimization of feedback-feedforward controlled robots improves their accuracy. For rest-to-rest motions, we find the optimal trajectory indirectly by numerically optimizing the corresponding feedforward controller for accuracy. A new performance measure called the Manipulation Sensitivity Norm (MSN) is introduced that determines the accuracy under most disturbances and modeling errors. We tested this method on a two DOF robotic arm in the horizontal plane. The results show that for all feedback gains we tested, the choice for the trajectory has a significant influence on the accuracy of the arm (viz. position errors being reduced from 2.5 cm to 0.3 cm). Moreover, to study which features of feedforward controllers cause high or low accuracy, four more feedforward controllers were tested. Results from those experiments indicate that a trajectory that is smooth or quickly approaches the goal position will be accurate.

Sampling-based methods, such as Probabilistic Roadmap Method(PRM)[1], Rapidly-Exploring Random Tree (RRT)[2], have been proposed as promising solutions for kinodynamic problems. Nevertheless, it's still a challenge for practical application especially for complex systems. In particular, most of the forward propagation is fruitless, which lead to heavy computation and be time-consuming. This paper presents a greedy kinodynamic motion planner: Guided RRT. The main characteristics are that the environments are explored by Geometric trees previously and nodes near the geometric feasible path enjoy more preference. Instead of exploring the environments uniformly, the new approach expands towards the goal greedily along a series of waypoints, with probabilistically completeness. And to guarantee the effective of the greed, a new distance metric based on Euclidean metric are proposed by considering both the current position and the following position with zero-input. We compare our technique with standard RRT and show that it achieves favorable performance when planning under kinodynamic constraints.

Most conventional robotic arms depend on sensory feedback to perform their tasks. When feedback is inaccurate, slow or otherwise unreliable, robots should behave more like humans: rely on feedforward instead. This paper presents an approach to perform repetitive tasks with robotic arms, without the need for feedback (i.e. the control is open loop). The cyclic motions of the repetitive tasks are analyzed using an approach similar to limit cycle theory. We optimize open loop control signals that result in open loop stable motions. This approach to manipulator control was implemented on a two DOF arm in the horizontal plane with a spring on the first DOF, of which we show simulation and hardware results. The results show that both in simulation and in hardware experiments, it is possible to create open loop stable cycles. However, the two resulting cycles are different due to model inaccuracies. We also show simulation and hardware results for an inverted pendulum, of which we have a more accurate model. These results show stable cycles that are the same in simulation and hardware experiments.

The dynamic feasibility of solutions to motion planning problems using Rapidly Exploring Random Trees depends strongly on the choice of the distance metric used while planning. The ideal distance metric is the optimal cost of traversal between two states in the state space. However, it is computationally intensive to find the optimal cost while planning. We propose a novel approach to overcome this barrier by using a supervised learning algorithm that learns a nonlinear function which is an estimate of the optimal cost, via offline training. We use the Iterative Linear Quadratic Regulator approach for estimating an approximation to the optimal cost and learn this cost using Locally Weighted Projection Regression. We show that the learnt function approximates the original cost with a reasonable tolerance and more importantly, gives a tremendous speed up of a factor of 1000 over the actual computation time. We also use the learnt metric for solving the pendulum swing up planning problem and show that our metric performs better than the popularly used Linear Quadratic Regulator based metric.

Grasping of unknown objects (neither appearance data nor object models are given in advance) is very important for robots that work in an unfamiliar environment. In this paper, in order to make grasping of unknown objects more reliable and faster, we propose a novel grasping algorithm which does not require to build a 3D model of the object. For most objects, one point cloud is enough. For other objects, at most two point clouds are enough to synthesize reliable grasp. Taking grasping range and width of robot hand into consideration, the most suitable grasping region can be calculated on the contour of the point cloud of unknown object by maximizing the coefficient of force balance. Further analysis of the point cloud in the best grasping region can obtain the grasping position and orientation of robot hand. The point cloud information is processed on line, the grasping algorithm can quickly get the grasping position and orientation and then drive robot to the grasping point to execute grasping action. Simulations and experiments on an Universal arm UR5 and an underactuated Lacquey Fetch gripper are used to examine the performance of the algorithm, and successful results are obtained.

In this paper we simultaneously optimize the parameters describing the morphology of a robot arm and the parameters of its nonlinear controller. A novel concept of a pick-and-place robot arm is considered, which is called the resonating arm (RA). It uses a nonlinear spring mechanism to generate pick-and-place motions without the need for powerful actuators. This improves energy efficiency, cost and weight of the robot arm. Because of the complex interactions of the spring mechanism and the controller, we use evolutionary co-optimization to optimize the RA system as a whole. The results reveal that evolutionary co-optimization yields near optimal solutions for a 1 degree of freedom (1-DOF) RA, which require 43% less torque than the solution found through a separate optimization of the system and the control parameters. In case of a 2-DOF RA, evolutionary co-optimization resulted in credible solutions as well, but with less consistency.

The common view on feedforward control is that it needs an accurate model in order to accurately predict a future state of the system. However, in this paper we show that there are model inaccuracies that do not affect the final position of a motion, when using the right feedforward controller. Having an accurate final position is the main requirement in the task we consider: a pick-and-place task. We optimized the feedforward controllers such that the effect of model inaccuracies on the final position was minimized. The system we studied is a one DOF robotic arm in the horizontal plane, of which we show simulation and hardware results. The results show that the errors in the final position can be reduced to approximately zero for an inaccurate Coulomb, viscous or torque dependent friction. Furthermore, errors in the final position can be reduced, but not to zero, for an inaccurate inertia or motor constant. In conclusion, we show that for certain model inaccuracies, no feedback is required to eliminate the effect of an inaccurate model on the final position of a motion.

The purpose of this paper is to help engineers and researches to choose among the extremum seeking control (ESC) techniques for robotic applications such as object grasping, active object recognition and viewpoint optimization. These techniques are categorized into five main groups: Sliding mode ESC, neural network ESC, approximation based ESC, perturbation based ESC and adaptive ESC. These groups are explained briefly by stressing their working principles and the effect of the parameters. Then, the techniques are compared with respect to their robustness to noise and system dynamics by simulations. In conclusion, we propose the usage of the approximation based methods when the noise level is negligible. When noise is present, the neural network based optimizers are a better choice thanks to their hysteresis functions. However, if the system has both high noise and dynamic effects, then the perturbation based method is preferable since large motions provide robustness to noise and smooth references generated by the algorithm are less likely to cause instability. An application example is also given on texture density maximization.

Most conventional robotic arms use motors to accelerate the manipulator. This leads to an unnecessary high energy consumption when performing repetitive tasks. This paper presents an approach to reduce energy consumption in robotic arms by performing its repetitive tasks with the help of a parallel spring mechanism. A special non-linear spring characteristic has been achieved by attaching a spring to two connected pulleys. This parallel spring mechanism provides for the accelerations of the manipulator without compromising its ability to vary the task parameters (the time per stroke, the displacement per stroke the grasping time and the payload). The energy consumption of the arm with the spring mechanism is compared to that of the same arm without the spring mechanism. Optimal control studies show that the robotic arm uses 22% less energy due to the spring mechanism. On the 2 DOF prototype, we achieved an energy reduction of 20%. The difference was due to model simplifications. With a spring mechanism, there is an extra energetic cost, because potential energy has to be stored into the spring during startup. This cost is equal to the total energy savings of the 2 DOF arm during 8 strokes. Next, there could have been an energetic cost to position the manipulator outside the equilibrium position. We have designed the spring mechanism in such a way that this holding cost is negligible for a range of start- and end positions. The performed experiments showed that the implementation of the proposed spring mechanism results in a reduction of the energy consumption while the arm is still able to handle varying task parameters.

Swing-leg retraction was introduced as a way to improve the stability and disturbance rejection of running robots. It was also suggested that the reduced foot speed due to swing-leg retraction can help reduce impact energy losses, decrease peak forces, and minimize foot slipping. However, the extent to which swing-leg retraction rate influences all these benefits was unknown. In this paper, we present a study on the effect of swing-leg retraction rate on these benefits. The results of this study show that swing-leg retraction can indeed improve the performance of running robots in all of the suggested areas. However, the results also show that, for moderate and high running speeds, the optimal retraction rate for maximal disturbance rejection and stability is different from the optimal retraction rate for minimal impact losses, peak forces, and foot slipping. This discrepancy indicates an inherent tradeoff to consider when selecting the retraction rate for a robot control system: in general, retraction rate can be optimized for better stability and disturbance rejection or for more favorable efficiency, impact forces, and footing stability, but not all simultaneously. Furthermore, this tradeoff becomes more severe as running speed increases.

Grasping unknown objects is a crucial necessity for robots that operate in an unstructured environment. In this paper, we propose a novel grasping algorithm that uses active vision as basis. The algorithm uses the curvature information obtained from the silhouette of the object. By maximizing the curvature value, the pose of the robot is updated and a suitable grasping configuration is achieved. The algorithm has certain advantages over the existing methods: It does not require a 3D model of the object to be extracted, and it does not rely on any knowledge base obtained offline. This leads to a faster and still reliable grasping of the target object in 3D. The performance of the algorithm is examined by simulations and experiments, and successful results are obtained.

Swing leg retraction reduces touchdown energy losses of running by decreasing foot speed at the moment of ground contact, but does the additional acceleration of the swing leg cost more energy than is saved? To determine whether swing leg retraction can increase the overall energy efficiency of running robots, we find the optimally efficient gaits of a McGeer-like runner over a range of retraction rates. Results show that overall energy usage, including that used to swing the legs, scales with energy loss at touchdown, which is minimized at the retraction rate that zeros foot tangential speed at ground contact. Other benefits of swing leg retraction, such as reduced foot slippage and damaging touchdown forces, are realized simultaneously with optimal energy efficiency.

Real robots demonstrating online Reinforcement Learning (RL) to learn new tasks are hard to find. The specific properties and limitations of real robots have a large impact on their suitability for RL experiments. In this work, we derive the main hardware and software requirements that a RL robot should fulfill, and present our biped robot LEO that was specifically designed to meet these requirements. We verify its aptitude in autonomous walking experiments using a pre-programmed controller. Although there is room for improvement in the design, the robot was able to walk, fall and stand up without human intervention for 8 hours, during which it made over 43; 000 footsteps.

The concept of dasiaLimit Cycle Walkingpsila in bipedal robots removes the constraint of dynamic balance at every instance during gait. We hypothesize that this is crucial for the development of increasingly versatile and energy-effective humanoid robots. It allows the application of a wide range of gaits and it allows a robot to utilize its natural dynamics in order to reduce energy use. This paper presents the design and experimental results of our latest walking robot dasiaFlamepsila and the design of our next robot in line dasiaTUlippsila. The focus is on the mechanical implementation of series elastic actuation, which is ideal for Limit Cycle Walkers since it offers high controllability without having the actuator dominating the system dynamics. Walking experiments show the potential of our robots, showing good walking performance, though using simple control.

Limit cycle walkers are bipeds that exhibit a stable cyclic gait without requiring local controllability at all times during gait. A well-known example of limit cycle walking is McGeer's ldquopassive dynamic walking,rdquo but the concept expands to actuated bipeds as involved in this study. One of the stabilizing effects in limit cycle walkers is the dissipation of energy that occurs when the swing foot hits the ground. We hypothesize that this effect can be enhanced with a negative relation between the step length and step time. This relation is implemented through an open-loop strategy called swing-leg retraction; a predefined time trajectory for the swing leg makes the swing leg move backwards just prior to foot impact. In this paper, we study the effect of swing-leg retraction through three bipeds; a simple point mass simulation model, a realistic simulation model, and a physical prototype. Their stability is analyzed using Floquet multipliers, followed by an evaluation of how well disturbances are handled using the Gait Sensitivity Norm. We find that mild swing-leg retraction is optimal for the disturbance rejection of a limit cycle walker, as it results in a system response that is close to critically damped, rejecting the disturbance in the fewest steps. Slower retraction results in an overdamped response, characterized by a positive dominant Floquet multiplier. Likewise, faster retraction results in an underdamped response, characterized by a negative Floquet multiplier.

The construction of more capable bipedal robots highly depends on the ability to measure their performance. This performance is often measured in terms of speed or energy efficiency, but these properties are secondary to the robot's ability to prevent falling given the inevitable presence of disturbances, i.e., its disturbance rejection. Existing disturbance rejection measures (zero moment point, basin of attraction, Floquet multipliers) are unsatisfactory due to conservative assumptions, long computation times, or bad correlation to actual disturbance rejection. This paper introduces a new measure called the Gait Sensitivity Norm that combines a short calculation time with good correlation to actual disturbance rejection. It is especially suitable for implementation on limit cycle walkers, a class of bipeds that currently excels in terms of energy efficiency, but still has limited disturbance rejection capabilities. The paper contains an explanation of the Gait Sensitivity Norm and a validation of its value on a simple walking model as well as on a real bipedal robot. The disturbance rejection of the simple model is studied for variations of floor slope, foot radius, and hip spring stiffness. We show that the calculation speed is as fast as the standard Floquet multiplier analysis, while the actual disturbance rejection is correctly predicted with 93% correlation on average.

Passive dynamic walking is a promising idea for the development of simple and efficient two-legged walking robots. One of the difficulties with this concept is the addition of a stable upper body; on the one hand, a passive swing leg motion must be possible, whereas on the other hand, the upper body (an inverted pendulum) must be stabilized via the stance leg. This paper presents a solution to the problem in the form of a bisecting hip mechanism. The mechanism is studied with a simulation model and a prototype based on the concept of passive dynamic walking. The successful walking results of the prototype show that the bisecting hip mechanism forms a powerful ingredient for stable, simple, and efficient bipeds

Stability control for walking bipeds has been considered a complex task. Even two-dimensional fore-aft stability in dynamic walking appears to be difficult to achieve. In this paper we prove the contrary, starting from the basic belief that in nature stability control must be the sum of a number of very simple rules. We study the global stability of the simplest walking model by determining the basin of attraction of the Poincare/spl acute/ map of this model. This shows that the walker, although stable, can only handle very small disturbances. It mostly falls, either forward or backward. We show that it is impossible for any form of swing leg control to solve backward falling. For the problem of forward falling, we devise a simple but very effective rule for swing leg action: "You will never fall forward if you put your swing leg fast enough in front of your stance leg. In order to prevent falling backward the next step, the swing leg shouldn't be too far in front." The effectiveness of this rule is demonstrated with our prototype "Mike.".