Loader cranes with multiple actuated joints are complex systems to be operated by humans. Development of advanced assistance functions, such as end-effector velocity control in Cartesian space allows for utilizing the machine to its full speed and potential, wherein actuator limits, load balance, and singularities, as well as other complicated effects are handled by the automated function. To this end, this article provides a reinforcement learning-based policy optimization workflow for training and evaluating controllers using large-scale, parallelized invocations of forward kinematics. Monte Carlo evaluations of the closed-loop model are performed to inspect the stability and performance in the whole operational envelope of the loader crane for safe deployment on real machines. Our approach does not require any explicit inverse-kinematics model and is free from complex or hard-coded actuator limits or objectives. Results of simulations and experiments on a real loader crane are provided to showcase the performance of our approach in comparison to Jacobian inverse-based methods.

We present a method for estimating ego-velocity in autonomous navigation by integrating high-resolution imaging radar with an inertial measurement unit. The proposed approach addresses the limitations of traditional radar-based ego-motion estimation techniques by employing a neural network to process complex-valued raw radar data and estimate instantaneous linear ego-velocity along with its associated uncertainty. This uncertainty-aware velocity estimate is then integrated with inertial measurement unit data using an Extended Kalman Filter. The filter leverages the network-predicted uncertainty to refine the inertial sensor's noise and bias parameters, improving the overall robustness and accuracy of the ego-motion estimation. We evaluated the proposed method on the publicly available Coloradar dataset. Our approach achieves significantly lower error compared to the closest publicly available method, and also outperforms both instantaneous and scan matching-based techniques.

This paper proposes a novel control method for an autonomous wheel loader, enabling time-efficient navigation to an arbitrary goal pose. Unlike prior works which combine high-level trajectory planners with Model Predictive Control (MPC), we directly enhance the planning capabilities of MPC by incorporating a cost function derived from Actor-Critic Reinforcement Learning (RL). Specifically, we first train an RL agent to solve the pose reaching task in simulation, then transfer the learned planning knowledge to an MPC by incorporating the trained neural network critic as both the stage and terminal cost. We show through comprehensive simulations that the resulting MPC inherits the time-efficient behavior of the RL agent, generating trajectories that compare favorably against those found using trajectory optimization. We also deploy our method on a real-world wheel loader, where we demonstrate successful navigation in various scenarios.

Loader cranes with multiple actuated joints are complex systems to be operated by humans. Development of advanced assistance functions, such as end-effector velocity control in Cartesian space allows for utilizing the machine to its full speed and potential, wherein actuator limits, load balance, and singularities, as well as other complicated effects are handled by the automated function. To this end, this article provides a reinforcement learning-based policy optimization workflow for training and evaluating controllers using large-scale, parallelized invocations of forward kinematics. Monte Carlo evaluations of the closed-loop model are performed to inspect the stability and performance in the whole operational envelope of the loader crane for safe deployment on real machines. Our approach does not require any explicit inverse-kinematics model and is free from complex or hard-coded actuator limits or objectives. Results of simulations and experiments on a real loader crane are provided to showcase the performance of our approach in comparison to Jacobian inverse-based methods.

Loading multiple different materials with wheel loaders is a challenging task because various materials require different loading techniques. It’s, therefore, difficult to find a single controller capable of handling them all. One solution is to use a base controller and fine-tune it for different materials. Reinforcement Learning (RL) automates this process without the need for collecting additional human-annotated data. We investigated the feasibility of this approach using a full-size 24-tonnes wheel loader in the real world and demonstrated that it’s possible to fine-tune a neural network controller that was originally trained with imitation learning on blasted rock for use with an unknown gravel material, requiring 20 bucket fillings. Additionally, we showcased the adaptability of a controller pre-trained on woodchips for an unknown gravel material, requiring 40 bucket fillings. We also proposed a novel reward function for the material loading task. Finally, we examined how the sampling time of the reinforcement learning algorithm affects convergence speed and adaptability. Our results demonstrate that it’s optimal to match the sampling time of the RL algorithm to the delays of the wheel loader’s hydraulic actuators.

Safe motion control in unknown environments is one of the challenging tasks in robotics, such as autonomous navigation. Control Barrier Function (CBF), as a strong math-ematical tool, has been widely used in many safety-critical systems to satisfy safety requirements. However, there are only a handful of recent studies on safety controllers with perception inputs. Common assumptions in most of the works are that the CBF is already known and obstacles have predefined shapes. In this work, we introduce a novel Vision-based Control Barrier Function (V-CBF), which enables generalization to new environments and obstacles of arbitrary shapes. We then derive CBF safety conditions over RGB-D space and relate those to actual robot control inputs. To train the CBF function, we introduce a method to generate ground truth with desired properties complying with CBF and a method to generate part of the CBF as an image-to-image translation problem. We finally demonstrate the efficacy of V-CBF on the safe control of an autonomous car in CARLA simulator.

The ability of Gaussian processes (GPs) to predict the behavior of dynamical systems as a more sample-efficient alternative to parametric models seems promising for real-world robotics research. However, the computational complexity of GPs has made policy search a highly time and memory consuming process that has not been able to scale to larger problems. In this work, we develop a policy optimization method by leveraging fast predictive sampling methods to process batches of trajectories in every forward pass, and compute gradient updates over policy parameters by automatic differentiation of Monte Carlo evaluations, all on GPU. We demonstrate the effectiveness of our approach in training policies on a set of reference-tracking control experiments with a heavy-duty machine. Benchmark results show a significant speedup over exact methods and showcase the scalability of our method to larger policy networks, longer horizons, and up to thousands of trajectories with a sublinear drop in speed.

We have recently proposed two pile loading controllers that learn from human demonstrations: a neural network (NNet) [1] and a random forest (RF) controller [2]. In the field experiments the RF controller obtained clearly better success rates. In this work, the previous findings are drastically revised by experimenting summer time trained controllers in winter conditions. The winter experiments revealed a need for additional sensors, more training data, and a controller that can take advantage of these. Therefore, we propose a revised neural controller (NNetV2) which has a more expressive structure and uses a neural attention mechanism to focus on important parts of the sensor and control signals. Using the same data and sensors to train and test the three controllers, NNetV2 achieves better robustness against drastically changing conditions and superior success rate. To the best of our knowledge, this is the first work testing a learning-based controller for a heavy-duty machine in drastically varying outdoor conditions and delivering high success rate in winter, being trained in summer.

This work introduces a learning-based pile loading controller for autonomous robotic wheel loaders. Controller parameters are learnt from a small number of demonstrations for which low level sensor (boom angle, bucket angle and hydrostatic driving pressure), egocentric video frames and control signals are recorded. Application specific deep visual features are learnt from demonstrations using a Siamese network architecture and a combination of cross-entropy and contrastive loss. The controller is based on a Random Forest (RF) regressor that provides robustness against changes in field conditions (loading distance, soil type, weather and illumination). The controller is deployed to a real autonomous robotic wheel loader and it outperforms prior art with a clear margin.

This paper presents the development and testing of end-to-end Neural Network (NN) controllers for automated pile loading with a robotic wheel loader. NNs were trained using the Learning from Demonstration approach, i.e. by first recording sensor and control signals during manually-driven pile loading actions. Training made use of three input signals: boom angle, bucket angle and hydrostatic driving pressure; and three output signals: boom control, bucket control and the gas command. Most testing was conducted using NNs with 5 neurons in a single hidden layer, which were able to fill the bucket reasonably well. Qualitative comparisons were made to ascertain how the amount of training data and number of hidden neurons affects bucket filling performance, for NNs trained using both the Levenberg-Marquardt and Bayesian Regularization backpropagation algorithms. Different NNs trained with the same data were also compared. An additional pile transfer experiment compared the performance of an NN controller with a heuristic automated controller and manual human control. By estimating the total volume of material transferred using 3D laser scans, human control was found to have the highest performance, though the NN outperformed the heuristic controller. This indicated that end-to-end NN control trained by demonstration could offer improvement over current heuristic methods for automated pile loading.

Motion planning for manipulators under task space constraints is difficult as it constrains the joint configurations to always lie on an implicitly defined manifold. It is possible to view task constrained motion planning as an optimization problem with non-linear equality constraints, which can be solved by general non-linear optimization techniques. In this paper, we present a novel custom optimizer which exploits the underlying structure present in many task constraints. At the core of our approach are some simple reformulations, which when coupled with the method of alternating projection, leads to an efficient convex optimization based routine for computing a feasible solution to the task constraints. We subsequently build on this result and use the concept of Augmented Lagrangian to guide the feasible solutions towards those that also minimize the user defined cost function. We show that the proposed optimizer is fully distributive and thus, can be easily parallelized. We validate our formulation on some common robotic benchmark problems. In particular, we show that the proposed optimizer achieves cyclic motion in the joint space corresponding to a similar nature trajectory in the task space. Furthermore, as a baseline, we compare the proposed optimizer with an off-the-shelf non-linear solver provide in open source package SciPy. We show that for similar task constraint residuals and smoothness cost, it can be upto more than three times faster than the SciPy alternative.

This paper addresses the problem of pallet picking by an Articulated-Frame-Steering (AFS) hydraulic machine. We propose a macro-micro visual mobile manipulation architecture, where a smooth switching logic navigates the robot to pick an object. The state space is divided into several regions depending on the accuracy of the vision and robot's degrees of freedom. The control architecture benefits from the following phenomena: at distance, when the location of the object of interest is detected, its orientation may not be reliably estimated; at some closer distances, orientations also become available; and because pallets are wide with small height, yaw angle estimation are more accurate than pitch is. The switching logic is devised to control the corresponding degree of freedom of the mobile manipulator in each region. Moreover, in different regions, we employ different coordinate frames, namely an earth-fixed frame or an object-local frame, which is more natural for the problem in that region. We show that the architecture accomplishes the following: 1) it eliminates the need for replanning as the accuracy of pose estimation improves; and 2) it provides the mobile base with a longer corridor to steer toward the pallet and align its heading. We also incorporate a robust, accurate solution based on fiducial markers for object manipulation in unstructured outdoor environments and unfavorable weather conditions, which relies solely on a monocular camera for pallet detection. The presented experimental results demonstrate the superiority of the method, as the model starts following the target even when the pallet is still 6m away from the vehicle.

This paper, as a generalization of our previous works, presents a unified time-optimal path-following controller for Wheeled Mobile Robots (WMRs). Unlike other path-following controllers, we solve the path-following problem for all common categories of WMRs such as car-like, differential, omnidirectional, all wheels steerable and others. We show that the insertion of our path-following controller into the kinematic and non-holonomic constraints of the wheels, simplifies the otherwise impenetrable constraints, resulting in explicit monotonic functions between the velocity of the base and that of the wheels. Based on this foundation, we present a closed-form solution that keeps all the wheels' steering and driving velocities within their corresponding pre-specified bounds. Simulation data and experimental results from executing the controller in a real-time environment demonstrate the efficacy of the method.

This paper addresses the macro-micro configuration of a mobile manipulation problem for a forklift; specifically, it investigates pallet picking with visual feedback. A manipulator with limited degrees of freedom and differential constraint mobility, together with the large dimensions of the machine, requires reliable visual feedback (pallet pose) and navigation from relatively large distances. It has been shown that the problem can be divided into two parts in order to solve the related issues based on path following theories and visual servoing. Moreover, visual pallet detection is non-real-time and unreliable, especially due to large distances, unfavorable vibrations, and fast steering. To address these issues, we introduce a simple and efficient method that integrates the vision output with odometry and realizes a smooth and nonstop transition from global navigation to visual servoing. Real-world implementation on a small-sized forklift demonstrates the efficacy of the proposed macro-micro architecture.

Mobile robots with independently steerable wheels provide better robustness and efficiency compared to the other types of omnidirectional mobile robots. However, the non-holonomic constraints and singular configurations give rise to several challenging issues in exploiting the high maneuverability features of the robot. Many proposed motion controllers for such robots force the robot to stay outside of bulky regions around its singular points, which in turn limits the robot's dexterity. In this paper, which extends our previous works, we present an online trajectory generation along with a globally stable path following controller that enables the robot to follow any given smooth path and heading function. We show that the control signals extensively simplify the kinematic constraints and are utilized to develop an efficient online “Phase Plane” switching algorithm that bounds the velocities and accelerations of the actuators. Moreover, we show that the algorithm efficiently regulates the velocity of the robot around the singular configurations which allows the robot to realize wide ranges of complex maneuvers. The proposed control algorithm has been tested on iMoro(our four-wheeled independently steerable mobile manipulator), and the presented results show the efficacy of our method.

Mobile robots with independently steerable wheels possess many high maneuverability features of omnidirectional robots while benefiting from better performance and capability of moving on rough terrains. However, motion control of such robots is a challenging task due to presence of singular configurations and unboundedly large steering velocities in the neighborhood of those singularities. Many proposed approaches rely on numerical solutions that keep the robot out of bulky regions around the singular points and hence lose some of the robot maneuverability. Based on a class of traditional path followers we design a new globally stable path following controller that exploits the high maneuverability of the platform. This design allows us to derive a set of closed-form analytical functions that describe the robot base velocity as a function of the wheels driving and steering velocities while abide to the robot non-holonomic constraints. Those functions are then utilized to find the maximum instantaneous velocity of the body that keeps the wheels velocities under the pre-specified bounds no matter how much the robot gets close or far from its singular configurations. The control algorithms developed in this paper have been evaluated on iMoro, a four wheel independently steered mobile manipulator designed and developed at IHA/TUT. Experimental data is also shown that show efficacy of the method.