The use of reinforcement learning (RL) has led to huge advancements in the field of robotics. However data scarcity, brittle convergence and the gap between simulation & real world environments, mean that most common RL approaches are subject to over fitting and fail to generalise to unseen environments. Hardware agnostic policies would mitigate this by allowing a single network to operate in a variety of test domains, where dynamics vary due to changes in robotic morphologies or internal parameters. We utilise the idea that learning to adapt a known and successful control policy is easier and more flexible than jointly learning numerous control policies for different morphologies.This paper presents the idea of Hardware Agnostic Reinforcement Learning using Adversarial selection (HARL-A). In this approach training examples are sampled using a novel adversarial loss function. This is designed to self regulate morphologies based on their learning potential. Simply applying our learning potential based loss function to current state-of-the-art already provides ~ 30% improvement in performance. Meanwhile experiments using the full implementation of HARL-A report an average increase of 70% to a standard RL baseline and 55% compared with current state-of-the-art.

The successful performance of any system is dependant on the hardware of the agent, which is typically immutable during RL training. In this work, we present ORCHID (Optimisation of Robotic Control and Hardware In Design) which allows for truly simultaneous optimisation of hardware and control parameters in an RL pipeline. We show that by forming a complex differential path through a trajectory rollout we can leverage a vast amount of information from the system that was previously lost in the ‘black-box’ environment. Combining this with a novel hardware-conditioned critic network minimises variance during training and ensures stable updates are made. This allows for refinements to be made to both the morphology and control parameters simultaneously. The result is an efficient and versatile approach to holistic robot design, that brings the final system nearer to true optimality. We show improvements in performance across 4 different test environments with two different control algorithms - in all experiments the maximum performance achieved with ORCHID is shown to be unattainable using only policy updates with the default design. We also show how re-designing a robot using ORCHID in simulation, transfers to a vast improvement in the performance of a real-world robot.

A space robot working in a controlled-floating mode can be used for performing in-orbit telescope assembly through simultaneously controlling the motion of the spacecraft base and its robotic arm. Handling and assembling optical mirrors requires the space robot to achieve slow and precise manoeuvres regardless of the disturbances and errors in the trajectory. The robustness offered by the nonlinear H∞controller, in the presence of environmental disturbances and parametric uncertainties, makes it a viable solution. However, using fixed tuning parameters for this controller does not always result in the desired performance as the arm's trajectory is not known a priori for orbital assembly missions. In this paper, a complete study on the impact of the different tuning parameters is performed and a new adaptive H∞controller is developed based on bounded functions. The simulation results presented show that the proposed adaptive H∞controller guarantees robustness and precise tracking using a minimal amount of forces and torques for assembly operations using a small space robot.