In this paper, a data-driven Model Predictive Controller (MPC) is presented, in which an off-policy Reinforcement Learning (RL) method called Deep Double Expected Sarsa is employed to update the weights of its cost function. While the parameterized MPC cost function is used as the current action-value function estimator, a Neural Network is used as the subsequent action-value function approximator. The target Neural Network is trained based on inputs and outputs of the primary MPC obtained at previous sampling times, whereby the training is performed either within each sampling time by sharing the time slots with the main algorithm or in parallel to the main algorithm as a whole. The latter reduces the required real-time computations per time slot. To compute the action of the target policy, two strategies are employed: Once a greedy policy using a minimization of the Neural Network model with respect to the action, and once the second element of the MPC vector related to the previous sampling time. Results show that there is no significant difference between the final control performance and training speed of both methods, whereas the real-time computational cost can be significantly reduced for the latter approach since the optimization related to the Neural Network can be omitted.

This paper presents a constructive heuristic approach to solve the live sawing optimization problem, which involves cutting rectangular boards from cylindrical logs with circular cross sections. The problem is defined as a two-dimensional strip bin-packing problem, where the objective is to orthogonally pack a given set of rectangular items into a set of strips inside a circle, while considering feasibility and technical constraints. The proposed approach uses a list of ordered rectangles and a combination of sorting, placement, and searching policies to build a feasible cutting, and presents a live sawing pattern generation that makes guillotine cuts possible for sawing machines. Our approach surpasses traditional methods, including a mathematical programming model and a state-of-the-art heuristic approach, in terms of computational effort, memory usage, and search requirements. Furthermore, it maintains superior cutting yield efficiency, as evidenced by the presented simulations and comparisons.

Rescue missions in mountain environments are hardly achievable by standard legged robots—because of the high slopes—or by flying robots—because of limited payload capacity. We present a concept for a rope-aided climbing robot which can negotiate up-to-vertical slopes and carry heavy payloads. The robot is attached to the mountain through a rope, and it is equipped with a leg to push against the mountain and initiate jumping maneuvers. Between jumps, a hoist is used to wind/unwind the rope to move vertically and affect the lateral motion. This simple (yet effective) two-fold actuation allows the system to achieve high safety and energy efficiency. Indeed, the rope prevents the robot from falling while compensating for most of its weight, drastically reducing the effort required by the leg actuator. We also present an optimal control strategy to generate point-to-point trajectories overcoming an obstacle. We achieve fast computation time ($16\ m$long jump, showing the effectiveness of the proposed approach, and confirming the interest of our concept. Finally, we performed a reachability analysis showing that the region of achievable targets is strongly affected by the friction properties of the foot-wall contact.

In this paper, the applicability of machine learning models and techniques to the Markov–Dubins path planning problem have been explored. Machine learning techniques are already applied to several fields, which range from computer vision, to physics simulation, to item recommendation, to user profiling. This pervasiveness has led to marked improvements in the implementation and support for applying machine learning models, in particular for specialised use cases such as low-power devices, embedded hardware, and real-time applications. On the other hand, the Markov–Dubins path planning problem, which is central in robotic nonholonomic trajectory design, is already covered by established numerical and optimisation techniques. However, the benefits of applying machine learning approaches to this problem remain to be investigated. In particular, there is the need to research potential speed-ups or application domains that would be better solved by a machine learning approach compared to the traditional algorithmic approaches. In this study, we train a state-of-the-art machine learning model in a supervised setting on Markov–Dubins and use it in two different ways: to directly predict the solution, and to filter candidate solutions. Also, a comparison of the quality of these predictions with a state-of-the-art Markov–Dubins solver is made. The results obtained indicate that machine learning approaches are comparable to state-of-the-art solutions: our bare model, directly predicting the solution, appears to be 8.3 times faster than the current standard, sacrificing the accuracy, which amounts to a value close to 92%; the hybrid model that filters the solutions prior to finding the best candidate runs in times that are comparable to the classical solver (58 ms) and has over 98% accuracy. A further comparison with alternative solvers and techniques, such as Optimal Control, NonLinear Programming and Mixed Integer NonLinear Programming has been made, confirming the benefits of the machine learning approach over these, for which the computational times are in the range of seconds. This opens new avenues for interdisciplinary applications of machine learning to more general planning problems (e.g., the same problem in 3D), where the number of possible manoeuvres is large and the computation of each of them requires a considerable computational effort, which makes the brute force trial-and-error infeasible.

This paper presents a parallel computing implementation of the Iterative Dynamic Programming (IDP) solution to the multipoint Markov-Dubins problem using GPUs. The multi-point Markov-Dubins problem requires the computation of the shortest path with bounded curvature that connects a sequence of planar points (waypoints). As well as being interesting in its own right, an efficient solution to this problem is key to finding optimal or suboptimal solutions of other problems such as the Dubins Travelling Salesman and the Dubins Orienteering problem. The constraint on the curvature makes the problem highly non-linear and complicates its solution. Classic methods are optimisation-based and cast the problem into the Nonlinear Programming (NLP) or Mixed Integer Nonlinear Programming (MINLP) frameworks, for which existing solutions cannot be significantly parallelised. On the contrary, the IDP solution proposed here is well suited for parallel execution. In the paper, we show that the parallel implementation of the IDP outperforms both the NLP/MINLP methods and the iterative version of the IDP methods in terms of accuracy, computation time and power consumption. Computation time and power consumption will be the main focus of the paper, because they are closely related to the implementation on an embedded platform.

In this article, we present a framework for planning an activity to be executed with the support of a robotic navigation assistant. The two main components are the activity and the motion planner. The activity planner composes a sequence of abstract activities, chosen from a given set, to synthesize a plan. Each activity is associated with a point of interest in the environment and with probabilistic parameters that depend on the plan, which are characterized by simulations in realistic scenarios. The low-level action to pass from an activity to the next is handled by the motion planner, which secures the physical feasibility of the chosen actions and their compatibility with the constraints posed by the user and the environment. Indeed, the final plan must respect the user constraints and optimise his/her satisfaction from the activity. We show a possible model for the problem as a chance constrained optimization along with an efficient technique to find high-quality solutions.