Maximizing the endurance of unmanned aerial vehicles (UAVs) in large-scale monitoring missions spanning over large areas requires addressing their limited battery capacity. Deploying unmanned ground vehicles (UGVs) as mobile recharging stations offers a practical solution, extending UAVs' operational range. This introduces the challenge of optimizing UAV-UGV routes for efficient mission point coverage and seamless recharging coordination. In this paper, we present a risk-aware deep reinforcement learning (Ra-DRL) framework with a multi-head attention mechanism within an encoder-decoder transformer architecture to solve this cooperative routing problem for a UAV-UGV team. Our model minimizes mission time while accounting for the stochastic fuel consumption of the UAV, influenced by environmental factors like wind velocity, ensuring adherence to a risk threshold to avoid mid-mission energy depletion. Extensive evaluations on various problem sizes show that our method significantly outperforms nearest-neighbor heuristics in both solution quality and risk management. We validate the Ra-DRL policy in a Gazebo-ROS SITL environment with a PX4-based custom UAV and Clearpath Husky UGV. The results demonstrate the robustness and adaptability of our policy, making it highly effective for mission planning in dynamic, uncertain scenarios.

Online optimal control of quadruped robots would enable them to adapt to varying inputs and changing conditions in real time. A common way of achieving this is linear model predictive control (LMPC), where a quadratic programming (QP) problem is formulated over a finite horizon with a quadratic cost and linear constraints obtained by linearizing the equations of motion and solved on the fly. However, the model linearization may lead to model inaccuracies. In this paper, we use the Koopman operator to create a linear model of the quadrupedal system in high dimensional space which preserves the nonlinearity of the equations of motion. Then using LMPC, we demonstrate high fidelity tracking and disturbance rejection on a quadrupedal robot. This is the first work that uses the Koopman operator theory for LMPC of quadrupedal locomotion.

Unmanned aerial vehicles (UAVs) possess the capability to survey vast areas, yet their operational range is limited by their battery capacity. Deploying mobile recharging stations via unmanned ground vehicles (UGVs) can significantly enhance the endurance and effectiveness of UAVs. However, optimizing the routes for both UAVs and UGVs, referred to as the UAV-UGV cooperative routing problem, requires a sophisticated planning framework to determine the vehicles’ routes and their recharging points. To address this, in this paper, we utilize a deep reinforcement learning (DRL) based framework equipped with multi-head attention layers. The framework is designed to sequentially select actions to construct routes for the UAV and UGV and to establish their rendezvous points for recharging. We evaluate our framework across various problem instance sizes and distributions, comparing it against recent heuristic-based methods and an existing learning-based method as baselines. Our proposed algorithm surpasses these baselines in terms of solution quality and runtime efficiency in the test scenarios, thus proving its effectiveness. Additionally, we investigate the application of our DRL policy in online mission planning to accommodate dynamic changes within the mission scenario.

We consider the motion planning problem of a hopper navigating a terrain comprising stepping stones while optimizing an energy metric. The most widely used approach of discrete searches (e.g., A-star) cannot handle boundary conditions (e.g., end path constraints on position, velocity). However, continuous optimizations can easily deal with the boundary value problem but are not widely used in motion planning because they are computationally intensive and possibly non-convex when one considers the terrain. Here we use a continuous optimization approach within a model predictive control framework. First, we generate a library comprising initial states at an instant in the locomotion cycle (e.g., apex), the controls (e.g., foot placement, amplitude of force), and the states at the same instant at the next step. Next, we fit these step-to-step models with low order polynomials (typically 2nd or 3rd order). Finally, the planner uses these low order step-to-step models to preview a fixed distance ahead and plans the optimal steps and controls. Thereafter, we implement the plan for the first step, followed by replanning. This process continues until the hopper reaches the end of the terrain. The main contributions are low-order polynomial models for fast computation and incorporation of the complex terrain as a cost function.

We present a sampling-based framework for feedback motion planning of legged robots. Our framework is based on switching between limit cycles at a fixed instance of motion, the Poincaré section (e.g., apex or touchdown), by finding overlaps between the regions of attraction (ROA) of two limit cycles. First, we assume a candidate orbital Lyapunov function (OLF) and define a ROA at the Poincaré section. Next, we solve multiple trajectory optimization problems, one for each sampled initial condition on the ROA to minimize an energy metric and subject to the exponential convergence of the OLF between two steps. The result is a table of control actions and the corresponding initial conditions at the Poincaré section. Then we develop a control policy for each control action as a function of the initial condition using deep learning neural networks. The control policy is validated by testing on initial conditions sampled on ROA of randomly chosen limit cycles. Finally, the rapidly-exploring random tree algorithm is adopted to plan transitions between the limit cycles using the ROAs. The approach is demonstrated on a hopper model to achieve velocity and height transitions between steps.

The main motivation of this paper is to understand the role of foot placement and ankle push-off control in stabilizing bipedal gaits. We modify the simplest walker (heavy torso, light legs) by incorporating a hip spring, a hip actuator, and a telescopic linear actuator. We consider two stability criteria: one-step dead-beat stabilization for full correction of disturbance in a single step and exponential orbital stabilization using discrete control Lyapunov function. Our findings are as follows: (1) Both control strategies have almost similar robustness as measured by the number of steps walked on stochastic terrain before failure, but both strategies are more robust for changing terrain with step down and less robust for step up. (2) One step dead-beat stabilization is more energy-efficient than exponential stabilization. (3) Control strategy for step up is to decrease foot placement or maintain push-off and for step down is to increase foot placement or decrease the push-off. However, it is most energy-efficient to use foot placement control for step up and push-off control for step down.

We show that the simplest slope walker can walk over wide combinations of step lengths and step velocities at a given ramp slope by proper choice of foot placement. We are able to find walking solutions up to slope of 15.42°, beyond which, the ground reaction force on the stance leg goes to zero, implying a flight phase. We also show that the simplest walker can walk at human-sized step length and step velocity at a slope of 6.62°. The central idea behind control using foot placement is to balance the potential energy gained during descent with the energy lost during collision at foot-strike. Finally, we give some suggestions on how the ideas from foot placement control and energy balance can be extended to realize walking motions on practical legged systems.