High-frequency control in continuous action and state spaces is essential for practical applications in the physical world. Directly applying end-to-end reinforcement learning to high-frequency control tasks struggles with assigning credit to actions across long temporal horizons, compounded by the difficulty of efficient exploration. The alternative, learning low-frequency policies that guide higher-frequency controllers (e.g., proportional-derivative (PD) controllers), can result in a limited total expressiveness of the combined control system, hindering overall performance. We introduce EvoControl, a novel bi-level policy learning framework for learning both a slow high-level policy (using PPO) and a fast low-level policy (using Evolution Strategies) for solving continuous control tasks. Learning with Evolution Strategies for the lower-policy allows robust learning for long horizons that crucially arise when operating at higher frequencies. This enables EvoControl to learn to control interactions at a high frequency, benefitting from more efficient exploration and credit assignment than direct high-frequency torque control without the need to hand-tune PD parameters. We empirically demonstrate that EvoControl can achieve a higher evaluation reward for continuous-control tasks compared to existing approaches, specifically excelling in tasks where high-frequency control is needed, such as those requiring safety-critical fast reactions.

Particle therapy and FLASH radiotherapy offer promising advancements in cancer treatment. However, accurately measuring radiation dose within the body, especially in real-time and high dose rates characteristic of FLASH radiotherapy, poses significant technological challenges. Ionoacoustic methodologies offer an innovative and non-invasive approach to particle therapy dosimetry, leveraging the generation of acoustic waves from energetic particle interactions. As the initial pressure distribution of a ionoacoustic wave is linearly proportional to the absorbed dose, it is possible to reconstruct the dose distribution from the acoustic signals. This work explores the viability of ionoacoustic dosimetry through computational simulations of 9 MeV FLASH electron beams using the k-Wave toolbox in MATLAB with a variable number of sensors with the goal of maximizing the gamma index with a 1%/1 mm standard. Two different imaging algorithms were compared, and the results clearly indicate that interpolated time reversal (ITR) outperforms standard time reversal (STR) across virtually all sensor configurations. Remarkably, ITR achieves high-quality image reconstruction with a significantly lower number of sensors. For instance, using ITR acceptable images (above 90% of the gamma index less than 1) were produced with as few as 24 sensors, 78% less than the ones necessary for STR, which required at least 108 sensors to reach a similar accuracy of 90.75%. When using 108 sensors, ITR demonstrated a 99.80% accuracy, representing an improvement of 46.25 times over STR.

This paper focuses on control techniques for an integrated Dual Active Bridge (DAB) converter designed for low-voltage (5V) and low-power (2W) applications. The analysis concentrates on the single-phase-shift (SPS) modulation, as it offers the best balance between implementation simplicity and power efficiency for a control loop circuit that must be embedded within the system. Moreover, an innovative modulation technique called controlled single-phase-shift (CSPS) is introduced, which dynamically adjusts the phase-shift to maximize current flow through the load. This approach enhances the output power range by 33%, achieving an increase in the average efficiency throughout the phase-shift interval up to 4% when compared to the conventional SPS modulation. Circuit-level simulations have been conducted in Cadence Virtuoso environment to verify the effectiveness of the proposed approach.

In this paper, we describe the design of the Barkour benchmark for measuring robot agility in navigating complex environments. Despite the growing interest in developing agile robot locomotion skills, the field lacks systematic benchmarks to measure the performance of robotic control systems and hardware in agility-focused tasks. This motivated us to propose the Barkour benchmark, an obstacle course designed to quantify agility across various robotic platforms. Inspired by dog agility competitions, the course features diverse obstacles and a time-based scoring mechanism, encouraging researchers to develop controllers that enable robots to move quickly, precisely, and with adaptability. This benchmark is challenging as it demands diverse motion skills and the time-based scoring requires control precision at high speed. Along with the design details presented in the paper, we release our simulated environment setups in MuJoCo-XLA and the CAD model of a custom-designed quadruped robot to facilitate future research to reproduce the Barkour setup (available at sites.google.com/view/barkour). We hope these together will accelerate the pace of robot agility research.

A common approach to the generation of walking patterns for humanoid robots consists in adopting a layered control architecture. This paper proposes an architecture composed of three nested control loops. The outer loop exploits a robot kinematic model to plan the footstep positions. In the mid layer, a predictive controller generates a Center of Mass trajectory according to the well-known table-cart model. Through a whole-body inverse kinematics algorithm, we can define joint references for position controlled walking. The outcomes of these two loops are then interpreted as inputs of a stack-of-task QP-based torque controller, which represents the inner loop of the presented control architecture. This resulting architecture allows the robot to walk also in torque control, guaranteeing higher level of compliance. Real world experiments have been carried on the humanoid robot iCub.

This paper presents a new algorithm for optimal control (OC) of nonlinear dynamical systems. The main feature of this algorithm is that it allows the specification of the control objectives as a hierarchy of tasks, each task representing an action that the robot should perform. Each task is described by a cost function that the algorithm tries to minimize, while not affecting the tasks of higher priority. The concept of strict priority allows for an easier and more robust specification of the control objectives, without hand tuning of task weights. The hierarchy also makes it possible to properly regularize the behavior of each task independently. For the first time, we properly define the problem of regularizing the task cost functions in the presence of a hierarchy and propose an algorithm to compute an approximate solution. Several simulated scenarios with different robots compare our solution with other state-of-the-art methods, validating the interest of the hierarchy in OC and empirically demonstrating the importance of regularization to generate feasible behaviors.

Envisioned applications for humanoid robots call for the design of balancing and walking controllers. While promising results have been recently achieved, robust and reliable controllers are still a challenge for the control community dealing with humanoid robotics. Momentum-based strategies have proven their effectiveness for controlling humanoids balancing, but the stability analysis of these controllers is still missing. The contribution of this paper is twofold. First, we numerically show that the application of state-of-the-art momentum-based control strategies may lead to unstable zero dynamics. Secondly, we propose simple modifications to the control architecture that avoid instabilities at the zero-dynamics level. Asymptotic stability of the closed loop system is shown by means of a Lyapunov analysis on the linearized system's joint space. The theoretical results are validated with both simulations and experiments on the iCub humanoid robot.

This paper deals with the generation of motion for complex dynamical systems (such as humanoid robots) to achieve several concurrent objectives. Hierarchy of tasks and optimal control are two frameworks commonly used to this aim. The first one specifies control objectives as a number of quadratic functions to be minimized under strict priorities. The second one minimizes an arbitrary user-defined function of the future state of the system, thus considering its evolution in time. Our recent work on prioritized optimal control merges the advantages of both these methods. This paper reformulates the original prioritized optimal control algorithm with the precise goal of improving its computational speed. We extend the dynamic programming method to work with a hierarchy of tasks. We compared our approach in simulation with both our previous algorithm and classical optimal control. The measured computational improvement represents another step towards the application of prioritized optimal control for online model predictive control of humanoid robots. We believe that this could be the key to unlock the (so far unexploited) dynamic capabilities of these mechanical systems.

Collocated adaptive control of underactuated mechanical systems is still a concern for the control community. The main difficulty comes from the nonlinearity of the collocated inverse dynamics with respect to the base parameters, which forbids the direct application of classical adaptive control schemes. This paper extends and encompasses the Slotine's adaptive control, which was developed for fully actuated mechanical systems, to stabilize the collocated state space of an underactuated mechanical system. The key point is to define the sliding variable as the difference between the system's velocity and an exogenous state whose dynamics is considered as control input. We first revisit the Slotine's result in view of this definition and then show how to extend it to the underactuated case. Stability and convergence of time-varying reference trajectories for the collocated dynamics are shown to be in the sense of Lyapunov. Global well-posedness of the control laws is achieved by means of a new algebraic property of the mass matrix. Simulations, comparisons to existing control strategies, and experimental results on a two-link manipulator verify the soundness of the proposed approach.

This paper presents a new technique to control highly redundant mechanical systems, such as humanoid robots. We take inspiration from two approaches. Prioritized control is a widespread multi-task technique in robotics and animation: tasks have strict priorities and they are satisfied only as long as they do not conflict with any higher-priority task. Optimal control instead formulates an optimization problem whose solution is either a feedback control policy or a feedforward trajectory of control inputs. We introduce strict priorities in multi-task optimal control problems, as an alternative to weighting task errors proportionally to their importance. This ensures the respect of the specified priorities, while avoiding numerical conditioning issues. We compared our approach with both prioritized control and optimal control with tests on a simulated robot with 11 degrees of freedom.