This work demonstrates universal dynamic perching capabilities for quadrotors of various sizes and on surfaces with different orientations. By employing a non-dimensionalization framework and deep reinforcement learning, we systematically assessed how robot size and surface orientation affect landing capabilities. We hypothesized that maintaining geometric proportions across different robot scales ensures consistent perching behavior, which was validated in both simulation and experimental tests. Additionally, we investigated the effects of joint stiffness and damping in the landing gear on perching behaviors and performance. While joint stiffness had minimal impact, joint damping ratios influenced landing success under vertical approaching conditions. The study also identified a critical velocity threshold necessary for successful perching, determined by the robot's maneuverability and leg geometry. Overall, this research advances robotic perching capabilities, offering insights into the role of mechanical design and scaling effects, and lays the groundwork for future drone autonomy and operational efficiency in unstructured environments.

Fish use their lateral lines to sense flows and pressure gradients, enabling them to detect nearby objects and organisms. Towards replicating this capability, we demonstrated successful leader-follower formation swimming using flow pressure sensing in our undulatory robotic fish ($\mu$Bot/MUBot). The follower$\mu$Bot is equipped at its head with bilateral pressure sensors to detect signals excited by both its own and the leader's movements. First, using experiments with static formations between an undulating leader and a stationary follower, we determined the formation that resulted in strong pressure variations measured by the follower. This formation was then selected as the desired formation in free swimming for obtaining an expert policy. Next, a long short-term memory neural network was used as the control policy that maps the pressure signals along with the robot motor commands and the Euler angles (measured by the onboard IMU) to the steering command. The policy was trained to imitate the expert policy using behavior cloning and Dataset Aggregation (DAgger). The results show that with merely two bilateral pressure sensors and less than one hour of training data, the follower effectively tracked the leader within distances of up to$200 \text{mm}(=1$body length) while swimming at speeds of$155 \text{mm} / \mathrm{s}(=0.8$body lengths/s). This work highlights the potential of fish-inspired robots to effectively navigate fluid environments and achieve formation swimming through the use of flow pressure feedback. Video—https://youtu.be/DIDYGi9Td0I

Inverted landing in a rapid and robust manner is a challenging feat for aerial robots, especially while depending entirely on onboard sensing and computation. In spite of this, this feat is routinely performed by biological fliers such as bats, flies, and bees. Our previous work has identified a direct causal connection between a series of onboard visual cues and kinematic actions that allow for reliable execution of this challenging aerobatic maneuver in small aerial robots. In this work, we utilized Deep Reinforcement Learning and a physics-based simulation to obtain a general, optimal control policy for robust inverted landing starting from any arbitrary approach condition. This optimized control policy provides a computationally-efficient mapping from the system's emulated observational space to its motor command action space, including both triggering and control of rotational maneuvers. This was accomplished by training the system over a large range of approach flight velocities that varied with magnitude and direction. Next, we performed a sim-to-real transfer and experimental validation of the learned policy via domain randomization, by varying the robot's inertial parameters in the simulation. Through experimental trials, we identified several dominant factors which greatly improved landing robustness and the primary mechanisms that determined inverted landing success. We expect the reinforcement learning framework developed in this study can be generalized to solve more challenging tasks, such as utilizing noisy onboard sensory data, landing on surfaces of various orientations, or landing on dynamically-moving surfaces.

Here we present the development of an autonomous modular swimming robot. This robot, named µBot 2.0, was upgraded from our previous robot platform µBot and features onboard computing, sensing, and power. Its compact size and modularity render the robot an ideal platform for studying bio-inspired robot swimming. The robot is equipped with a micro controller in its head that communicates with external computers through Bluetooth Low Energy (BLE) and sends motor commands to the body segments via Inter-Integrated Circuit (I2C) protocol. Each body segment has a customized printed circuit board (PCB) that receives commands and controls the electromagnetic actuator for generating body movements. The robot head is also equipped with an Inertial Measurement Unit (IMU) to measure its heading and a battery for power. In this work, a µBot 2.0 with three actuators was assembled and the swimming performance was tested. The robot actuators were activated via rhythmic motor input from a central pattern generator (CPG). Experimental results showed that the swimming speed was highly sensitive to the frequency of the motor input, with a maximum swimming speed of 130 mm/s (equivalent to 0.7 body length per second) at 6 Hz. The robot also had the capability to correct its heading with IMU feedback and follow desired paths using a line-of-sight (LOS) guidance law with an overhead camera. Our results demonstrate the effectiveness of the robot's design and its potential in a variety of aquatic applications.

Inverted landing is a challenging feat to perform in aerial robots, especially without external positioning. However, it is routinely performed by biological fliers such as bees, flies, and bats. Our previous observations of landing behaviors in flies suggest an open-loop causal relationship between their putative visual cues and the kinematics of the aerial maneuvers executed. For example, the degree of rotational maneuver (the amount of body inversion prior to touchdown) and the amount of leg-assisted body swing both depend on the flies' initial body states while approaching the ceiling. In this work, inspired by the inverted landing behavior of flies, we used a physics-based simulation with experimental validation to systematically investigate how optimized inverted landing maneuvers depend on the initial approach velocities with varied magnitude and direction. This was done by analyzing the putative visual cues (that can be derived from onboard measurements) during optimal maneuvering trajectories. We identified a three-dimensional policy region, from which a mapping to a global inverted landing policy can be developed without the use of external positioning data. Through simulation, we also investigated the effects of an array of landing gear designs on the optimized landing performance and identified their advantages and disadvantages. The above results have been partially validated using limited experimental testing and will continue to inform and guide our future experiments, for example by applying the calculated global policy.

In this work, we developed a mathematical model and a simulation platform for a fish-inspired robotic template, namely Magnetic, Modular, Undulatory Robot$(\mu \text{Bot})$. Through this platform, we systematically explored the effects of robot design and fluid parameters on swimming performance via reinforcement learning. The mathematical model was composed of two interacting subsystems, the robotic dynamic model and the hydrodynamic model. The hydrodynamic model consisted of the reactive components (added-mass force and pressure forces) and the resistive components (drag and friction forces). These components were nondimensionalized for deriving key “control parameters” of the robot-fluid interaction. The$\mu\text{Bots}$were actuated via magnetic actuators controlled with harmonic voltage signals, which were optimized via EM-based Policy Hyper Parameter Exploration (EPHE) to maximize forward swimming speed. By varying the control parameters, a total of 36 cases with different robot template variations (Number of Actuators (NoA) and stiffness) and hydrodynamic parameters were simulated and optimized via EPHE. Results showed that the wavelength of the optimized gaits (i.e., backward traveling wave along the body) was independent of template variations and hydrodynamic parameters. Higher NoA yielded higher speed but lower speed per body length, suggesting a diminishing gain from added actuators. Body and caudal-fin dynamics were dominated by the interaction among fluid added-mass, spring, and actuation torque, with negligible contribution from fluid resistive drag. In contrast, thrust was dominated by the pressure force acting on the caudal fin, as steady swimming resulted from a balance between resistive force and pressure force, with minor contributions from added-mass force and body drag forces. Therefore, added-mass force only indirectly affected the thrust generation and forward swimming speed via the caudal fin dynamics.

Here we developed an experimental platform with a magnetic, modular, undulatory robot (μBot) for studying fish-inspired underwater locomotion. This platform will enable us to systematically explore the relationship between body morphology, swimming gaits, and swimming performance via reinforcement learning methods. The μBot was designed to be easily modifiable in morphology, compact in size, easy to be controlled and inexpensive. The experimental platform also included a towing tank and a motion tracking system for real-time measurement of the μBot kinematics. The swimming gaits of μBot were generated by a central pattern generator (CPG), which outputs voltage signals to μBot's magnetic actuators. The CPG parameters were learned experimentally using the parameter exploring policy gradient (PGPE) method to maximize swimming speed. In the experiments, two μBot designs with the same body morphology but different caudal-fin shapes were tested. Results showed that swimming gaits with back-propagating traveling waves can be learned experimentally via PGPE, while the shape of the caudal fins had moderate influences on the learned gaits and the swimming speed. Furthermore, robot swimming speed was sensitive to the undulating frequency and the voltage magnitude of the last three posterior actuators. In contrast, swimming gaits and speed were relatively invariant to the variances within the inter-module connection weights of CPG and the voltage applied to the anterior actuator.

Landing upside down on a ceiling is challenging as it requires a flier to invert its body and land against the gravity, a process that demands a stringent spatiotemporal coordination of body translational and rotational motion. Although such an aerobatic feat is routinely performed by biological fliers such as flies, it is not yet achieved in aerial robots using onboard sensors. This work describes the development of a bio-inspired inverted landing strategy using computationally efficient Relative Retinal Expansion Velocity (RREV) as a visual cue. This landing strategy consists of a sequence of two motions, i.e. an upward acceleration and a rapid angular maneuver. A policy search algorithm is applied to optimize the landing strategy and improve its robustness by learning the transition timing between the two motions and the magnitude of the target body angular velocity. Simulation results show that the aerial robot is able to achieve robust inverted landing, and it tends to exploit its maximal maneuverability. In addition to the computational aspects of the landing strategy, the robustness of landing is also significantly dependent on the mechanical design of the landing gear, the upward velocity at the start of body rotation, and timing of rotor shutdown.

In this work, we present an application of a policy gradient algorithm to a real-time robotic learning problem, where the goal is to maximize the average lift generation of a dynamically scaled robotic wing at a constant Reynolds number (Re). Compared to our previous work, the merit of this work is two-fold. First, a central pattern generator (CPG) model was used as the motion controller, which provided a smooth generation and transition of rhythmic wing motion patterns while the CPG was being updated by the policy gradient, thereby accelerating the sample generation and reducing the total learning time. Second, the kinematics included three degrees of freedom (stroke, deviation, pitching) and were also free of half-stroke symmetry constraint, together they yielded a larger kinematic space which later explored by the policy gradient to maximize the lift generation. The learned wing kinematics used the full range of stroke and deviation to maximize the lift generation, implying that the wing trajectories with larger disk area and lower frequencies were preferred for high lift generation at constant Re. Furthermore, the wing pitching amplitude converged to values between 45°-49° regardless of what the other parameters were. Notably, the learning agent was able to find two locally optimal wing motion patterns, which had distinct shapes of wing trajectory but generated similar cycle-averaged lift.

In this work, we present a successful application of a policy search algorithm to a real-time robotic learning problem, where the goal is to maximize the efficiency of lift generation on a dynamically scaled flapping robotic wing. The robotic wing has two degrees-of-freedom, i.e., stroke and pitch, and operates in a tank filled with mineral oil. For all experiments, the Reynolds number is maintained constant at 1000, where learning is performed for different prescribed stroke amplitudes to find the optimal wing pitching amplitude and the stroke-pitch phase difference that maximize the power loading (PL) of lift generation, a measure of aerodynamic efficiency. For the investigated stroke amplitude range (30°-90°), the efficiency is observed to increase with the stroke amplitude and the lift is mainly generated through the delayed stall, a quasi-steady aerodynamic mechanism. Furthermore, the wing rotation becomes more asymmetric with respect to stroke reversal as the stroke amplitude decreases, indicating an increased use of unsteady lift generation mechanisms at lower stroke amplitudes.

Biologists proposed the tau theory to explain how animals control their motion using visual feedback for different tasks including landing and perching. Tau theory is based on a concept called time-to-contact, which is the required time to contact an object if the current velocity is maintained. Recently, tau theory has been applied to control robots' motion for similar tasks such as perching, docking, braking, or landing. However, existing tau theory can only work for the case with zero contact velocity. Some tasks such as perching actually require a non-zero contact velocity to make gripping mechanisms work. To address this problem, we extend the tau theory by proposing a two-stage strategy in one dimensional space to generate the reference trajectory for time-to-contact. Moreover, we propose a new coupling strategy to deal with the motion in three dimensional space. Simulation results demonstrate the effectiveness of proposed two-stage and coupling strategies. Moreover, we leverage a featureless method to estimate the time-to-contact from image sequences and implement it on a mobile robot platform. Experimental results also demonstrate that the non-zero contact velocity can be accomplished using onboard vision feedback. The research presented in this paper can be readily applied to control the motion of flying robots for perching with visual feedback.