We address the unique challenge of vibration suppression for a flexible long-reach robotic manipulator system, namely, the through-wall deployment (TWD) system that is used in nuclear environments. This paper proposes a novel dynamics-based trajectory optimization approach, which minimizes both the acceleration and the jerk at the manipulator’s joints, as well as the vibrations of the flexible long-reach boom where the manipulator’s base is mounted. Firstly, we create an integrated model for the system dynamics based on the knowledge of the robotic manipulator and the acceleration data from the vibration tests. We then develop an original procedure for generating the high-order polynomial trajectory that guarantees the zero-boundary condition for a flexible number of optimization parameters and waypoints. Following the simulation of a multi-objective optimization scheme, the optimized trajectory is experimentally validated on the practical TWD system with around 28% vibration reduction on average compared to the benchmark. Importantly, this reduction is achieved without compromising on the average speed of motion. The methodology is transferable to a wider range of flexible robotic manipulator systems with similar characteristics.

In this article, we propose an algorithm that combines actor-critic-based off-policy method with consensus-based distributed training to deal with multiagent deep reinforcement learning problems. Specifically, convergence analysis of a consensus algorithm for a type of nonlinear system with a Lyapunov method is developed, and we use this result to analyze the convergence properties of the actor training parameters and the critic training parameters in our algorithm. Through the convergence analysis, it can be verified that all agents will converge to the same optimal model as the training time goes to infinity. To validate the implementation of our algorithm, a multiagent training framework is proposed to train each Universal Robot 5 (UR5) robot arm to reach the random target position. Finally, experiments are provided to demonstrate the effectiveness and feasibility of the proposed algorithm.

Sim-and-real training is a promising alternative to sim-to-real training for robot manipulations. However, the current sim-and-real training is neither efficient, i.e., slow con-vergence to the optimal policy, nor effective, i.e., sizeable real-world robot data. Given limited time and hardware budgets, the performance of sim-and-real training is not satisfactory. In this paper, we propose a Consensus-based Sim-And-Real deep reinforcement learning algorithm (CSAR) for manipulator pick-and-place tasks, which shows comparable performance in both sim-and- real worlds. In this algorithm, we train the agents in simulators and the real world to get the optimal policies for both sim-and-real worlds. We found two interesting phenomenons: (1) Best policy in simulation is not the best for sim-and-real training. (2) The more simulation agents, the better sim-and-real training. The experimental video is available at: https://youtu.be/mcHJtNIsTEQ.

Machines with high levels of autonomy such as robots and our growing need to interact with them creates challenges to ensure safe operation. The recent interest to create autonomous vehicles through the integration of control and decision-making systems makes such vehicles robots too. We therefore applied estimation and decision-making mechanisms currently investigated for human-robot interaction to human-vehicle interaction. In other words, we define the vehicle as an autonomous agent with which the human driver interacts, and focus on understanding the human intentions and decision-making processes. These are then integrated into the ro-bot`s/vehicle's own control and decision-making system not only to understand human behaviour while it occurs but to predict the next actions. To obtain knowledge about the human's intentions, this work relies heavily on the use of motion tracking data (i.e. skeletal tracking, body posture)gathered from drivers whilst driving. We use a data-driven approach to both classify current driving manoeuvres and predict future manoeuvres, by using a fixed prediction window and augmenting a standard set of manoeuvres. Results are validated against drivers of different sizes, seat preferences and levels of driving expertise to evaluate the robustness of the methods; precision and recall metrics higher than 95% for manoeuvre classification and 90% for manoeuvre prediction with time-windows of up to 1.3 seconds are obtained. The idea of prediction adds a highly novel aspect to human-robot/human-vehicle interaction, allowing for decision and control at a later point.

In this paper, an alternative solution for adaptive optimal tracking control of nonlinear completely unknown systems is proposed. Firstly, an adaptive identifier is used to estimate the unknown system dynamics. Then, a recently developed system augmentation approach is adopted to design the optimal control, where the reference signal is incorporated into the augmented system. Thus, both the feedforward control and feedback control can be obtained simultaneously. Then, a critic neural network (NN) is used to estimate the augmented performance index, and calculate the optimal control action. Thus, the widely used actor NN is not needed. Finally, a new adaptive law recently proposed by the authors is used to online update the NN weight. The closed-loop stability and the convergence of the optimal control are all proved. The feasibility of the suggested approach is demonstrated by a simulation example.

The soft robotics approach is widely considered to enable robots in the near future to leave their cages and move freely in our modern homes and manufacturing sites. Musculoskeletal robots are such soft robots which feature passively compliant actuation, while leveraging the advantages of tendon-driven systems. Even though these robots have been intensively researched within the last decade, high-performance feedback control laws have only very recently been developed. In [1], a controller was developed utilizing Dynamic Surface Control (DSC), an extension to backstepping, with an adaptive neural network compensator for joint as well as muscle friction. We compare these novel control strategies to Computed Force Control (CFC), an existing technique from the field of tendon-driven control, yielding highly improved trajectory tracking. The musculoskeletal robot Anthrob [2] serves as a benchmark.

An adaptive multi-dimensional compliance model reference controller was implemented in real-time on a 4 degrees of freedom (DOF) of the humanoid Bristol-Elumotion-Robotic-Torso II (BERT II) arm in Cartesian space. The robot manipulator has been controlled in such a way as to follow the compliant passive behaviour of a reference mass-spring-damper system model subject to externally sensed forces/torques in all DOF. The relevant reference model converts all measured torques into their equivalent forces at the end-effector and reacts accordingly. The suggested control scheme takes in particular account of the multi-variable aspect and the problem of body own torques when measuring external torques. The redundant DOF were used to control the robot motion in a human-like pattern via effort minimization. Associated actuator saturation issues were addressed by incorporating a novel anti-windup (AW) compensator.

This paper presents a conditioning scheme for a linear control system which is enhanced by a neural network (NN) controller and subjected to a control signal amplitude limit. The NN controller improves the performance of the linear control system by directly estimating an actuator-matched, unmodeled, nonlinear disturbance, in closed-loop, and compensating for it. As disturbances are generally known to be bounded, the nominal NN-control element is modified to keep its output below the disturbance bound. The linear control element is conditioned by an antiwindup (AW) compensator which ensures performance close to the nominal controller and swift recovery from saturation. For this, the AW compensator proposed is of low order, designed using convex linear matrix inequalities (LMIs) optimization.

Servo actuators used in hard disk drives have to be able to move from one track to another track as fast as possible in order to enable fast access to the data. This paper improves short-span seeking performance of a proposed compensator based control system for a dual-stage servo actuator. Short-span seeking performance is improved by replacing a constant saturation non-linearity in the secondary loop of the dual-stage control system by a variable saturation non-linearity. The variable saturation ensures secondary actuator signal boundedness and at the same time modulates the control error to be fed into the secondary control loop. This results in a significantly faster response of the secondary actuator and hence faster seeking times. The improvement in seeking times is most significant for short range seeks that are within the range of the secondary actuator. Both simulation and experimental results comparing the performance of a dual-stage servo actuator with constant and with variable saturation in the secondary loop are presented. The use of a variable saturation shows significant improvement in short-span seeking performance