In this work, we developed a mathematical model and a simulation platform for a fish-inspired robotic template, namely Magnetic, Modular, Undulatory Robot$(\mu \text{Bot})$. Through this platform, we systematically explored the effects of robot design and fluid parameters on swimming performance via reinforcement learning. The mathematical model was composed of two interacting subsystems, the robotic dynamic model and the hydrodynamic model. The hydrodynamic model consisted of the reactive components (added-mass force and pressure forces) and the resistive components (drag and friction forces). These components were nondimensionalized for deriving key “control parameters” of the robot-fluid interaction. The$\mu\text{Bots}$were actuated via magnetic actuators controlled with harmonic voltage signals, which were optimized via EM-based Policy Hyper Parameter Exploration (EPHE) to maximize forward swimming speed. By varying the control parameters, a total of 36 cases with different robot template variations (Number of Actuators (NoA) and stiffness) and hydrodynamic parameters were simulated and optimized via EPHE. Results showed that the wavelength of the optimized gaits (i.e., backward traveling wave along the body) was independent of template variations and hydrodynamic parameters. Higher NoA yielded higher speed but lower speed per body length, suggesting a diminishing gain from added actuators. Body and caudal-fin dynamics were dominated by the interaction among fluid added-mass, spring, and actuation torque, with negligible contribution from fluid resistive drag. In contrast, thrust was dominated by the pressure force acting on the caudal fin, as steady swimming resulted from a balance between resistive force and pressure force, with minor contributions from added-mass force and body drag forces. Therefore, added-mass force only indirectly affected the thrust generation and forward swimming speed via the caudal fin dynamics.

Landing upside down on a ceiling is challenging as it requires a flier to invert its body and land against the gravity, a process that demands a stringent spatiotemporal coordination of body translational and rotational motion. Although such an aerobatic feat is routinely performed by biological fliers such as flies, it is not yet achieved in aerial robots using onboard sensors. This work describes the development of a bio-inspired inverted landing strategy using computationally efficient Relative Retinal Expansion Velocity (RREV) as a visual cue. This landing strategy consists of a sequence of two motions, i.e. an upward acceleration and a rapid angular maneuver. A policy search algorithm is applied to optimize the landing strategy and improve its robustness by learning the transition timing between the two motions and the magnitude of the target body angular velocity. Simulation results show that the aerial robot is able to achieve robust inverted landing, and it tends to exploit its maximal maneuverability. In addition to the computational aspects of the landing strategy, the robustness of landing is also significantly dependent on the mechanical design of the landing gear, the upward velocity at the start of body rotation, and timing of rotor shutdown.

In this work, we present an application of a policy gradient algorithm to a real-time robotic learning problem, where the goal is to maximize the average lift generation of a dynamically scaled robotic wing at a constant Reynolds number (Re). Compared to our previous work, the merit of this work is two-fold. First, a central pattern generator (CPG) model was used as the motion controller, which provided a smooth generation and transition of rhythmic wing motion patterns while the CPG was being updated by the policy gradient, thereby accelerating the sample generation and reducing the total learning time. Second, the kinematics included three degrees of freedom (stroke, deviation, pitching) and were also free of half-stroke symmetry constraint, together they yielded a larger kinematic space which later explored by the policy gradient to maximize the lift generation. The learned wing kinematics used the full range of stroke and deviation to maximize the lift generation, implying that the wing trajectories with larger disk area and lower frequencies were preferred for high lift generation at constant Re. Furthermore, the wing pitching amplitude converged to values between 45°-49° regardless of what the other parameters were. Notably, the learning agent was able to find two locally optimal wing motion patterns, which had distinct shapes of wing trajectory but generated similar cycle-averaged lift.

In this work, we present a successful application of a policy search algorithm to a real-time robotic learning problem, where the goal is to maximize the efficiency of lift generation on a dynamically scaled flapping robotic wing. The robotic wing has two degrees-of-freedom, i.e., stroke and pitch, and operates in a tank filled with mineral oil. For all experiments, the Reynolds number is maintained constant at 1000, where learning is performed for different prescribed stroke amplitudes to find the optimal wing pitching amplitude and the stroke-pitch phase difference that maximize the power loading (PL) of lift generation, a measure of aerodynamic efficiency. For the investigated stroke amplitude range (30°-90°), the efficiency is observed to increase with the stroke amplitude and the lift is mainly generated through the delayed stall, a quasi-steady aerodynamic mechanism. Furthermore, the wing rotation becomes more asymmetric with respect to stroke reversal as the stroke amplitude decreases, indicating an increased use of unsteady lift generation mechanisms at lower stroke amplitudes.