With the research into development of quadruped robots picking up pace, learning based techniques are being explored for developing locomotion controllers for such robots. A key problem is to generate leg trajectories for continuously varying target linear and angular velocities, in a stable manner. In this paper, we propose a two pronged approach to address this problem. First, multiple simpler policies are trained to generate trajectories for a discrete set of target velocities and turning radius. These policies are then augmented using a higher level neural network for handling the transition between the learned trajectories. Specifically, we develop a neural network based filter that takes in target velocity, radius and transforms them into new commands that enable smooth transitions to the new trajectory. This transformation is achieved by learning from expert demonstrations. An application of this is the transformation of a novice user's input into an expert user's input, thereby ensuring stable manoeuvres regardless of the user's experience. Training our proposed architecture requires much less expert demonstrations compared to standard neural network architectures. Finally, we demonstrate experimentally these results in the in-house quadruped Stoch 2.

Humans and animals are believed to use a very minimal set of trajectories to perform a wide variety of tasks including walking. Our main objective in this paper is two fold 1) Obtain an effective tool to realize these basic motion patterns for quadrupedal walking, called the kinematic motion primitives (kMPs), via trajectories learned from deep reinforcement learning (D-RL) and 2) Realize a set of behaviors, namely trot, walk, gallop and bound from these kinematic motion primitives in our custom four legged robot, called the “Stoch”. D-RL is a data driven approach, which has been shown to be very effective for realizing all kinds of robust locomotion behaviors, both in simulation and in experiment. On the other hand, kMPs are known to capture the underlying structure of walking and yield a set of derived behaviors. We first generate walking gaits from D-RL, which uses policy gradient based approaches. We then analyze the resulting walking by using principal component analysis. We observe that the kMPs extracted from PCA followed a similar pattern irrespective of the type of gaits generated. Leveraging on this underlying structure, we then realize walking in Stoch by a straightforward reconstruction of joint trajectories from kMPs. This type of methodology improves the transferability of these gaits to real hardware, lowers the computational overhead on-board, and also avoids multiple training iterations by generating a set of derived behaviors from a single learned gait.

In this work, we provide a simulation framework to perform systematic studies on the effects of spinal joint compliance and actuation on bounding performance of a 16-DOF quadruped spined robot Stoch 2. Fast quadrupedal locomotion with active spine is an extremely hard problem, and involves a complex coordination between the various degrees of freedom. Therefore, past attempts at addressing this problem have not seen much success. Deep-Reinforcement Learning seems to be a promising approach, after its recent success in a variety of robot platforms, and the goal of this paper is to use this approach to realize the aforementioned behaviors. With this learning framework, the robot reached a bounding speed of 2.1m /s with a maximum Froude number of 2. Simulation results also show that use of active spine, indeed, increased the stride length, improved the cost of transport, and also reduced the natural frequency to more realistic values.

In this paper, we explore a specific form of deep reinforcement learning (D-RL) technique for quadrupedal walking—trajectory based policy search via deep policy networks. Existing approaches determine optimal policies for each time step, whereas we propose to determine an optimal policy for each walking step. We justify our approach based on the fact that animals including humans use “low” dimensional trajectories at the joint level to realize walking. We will construct these trajectories by using Bézier polynomials, with the coefficients being determined by a parameterized policy. In order to maintain smoothness of the trajectories during step transitions, hybrid invariance conditions are also applied. The action is computed at the beginning of every step, and a linear PD control law is applied to track at the individual joints. After each step, reward is computed, which is then used to update the new policy parameters for the next step. After learning an optimal policy, i.e., an optimal walking gait for each step, we then successfully play them in a custom built quadruped robot, Stoch 2, thereby validating our approach.

The axially translating flexible link in flexible manipulators with a prismatic joint can be modeled using the Euler-Bernoulli beam equation together with the convective terms. In general, the method of separation of variables cannot be applied to solve this partial differential equation. In this paper, we present a nondimensional form of the Euler-Bernoulli beam equation using the concept of group velocity and present conditions under which separation of variables and assumed modes method can be used. The use of clamped-mass boundary conditions lead to a time-dependent frequency equation for the translating flexible beam. We present a novel method to solve this time-dependent frequency equation by using a differential form of the frequency equation. We then present a systematic modeling procedure for spatial multi-link flexible manipulators having both revolute and prismatic joints. The assumed mode/Lagrangian formulation of dynamics is employed to derive closed form equations of motion. We show, using a model-based control law, that the closed-loop dynamic response of modal variables become unstable during retraction of a flexible link, compared to the stable dynamic response during extension of the link. Numerical simulation results are presented for a flexible spatial RRP configuration robot arm. We show that the numerical results compare favorably with those obtained by using a finite element-based model.

The equations describing the motion of a feedback controlled robot are known, to be non-linear. Several non-linear equations are known to exhibit chaos for certain ranges of parameters. In this paper, the authors explore the possibility of chaos in a system of differential equations which model a feedback controlled two link robot with rotary(R) joints. The authors examine a simple proportional and derivative controller and a model based controller for a 2R planar robot undergoing repetitive motions in a plane in the absence of gravity. The authors show that the differential equations describing such a system exhibits chaotic behavior for certain ranges of the proportional and derivative gains and for certain values of a parameter which quantifies the mismatch between the model and the actual system. The authors discuss the difficulty of obtaining analytical results and describe numerical schemes to test for chaos and to obtain ranges of gains and the mismatch which results in chaotic motions.

Wheeled mobile robots (WMRs) are known to be non-holonomic systems, and most dynamic models of WMRs assume that the wheels undergo rolling without slipping. This paper deals with the problem of modeling and simulation of motion of a WMR when the conditions for rolling are not satisfied at the wheels. The authors use a traction model where the adhesion coefficient between the wheels of a WMR and a hard flat surface is a function of the wheel slip. This traction model is used in conjunction with the dynamic equations of motion to simulate the motion of the WMR. The simulations show that controllers which do not take into account wheel slip give poor tracking performance for the WMR and path deviation is small only for large adhesion coefficients. This work shows the importance of wheel slip and suggests use of accurate traction models for improving tracking performance of a WMR.>