In many engineering problems, including those related to robotics, optimization of the control policy for multiple conflicting criteria is required. However, this can be very challenging because of the existence of noise, which may be input dependent or heteroscedastic, and restrictions regarding the number of evaluations owing to the costliness of the experiments in terms of time and/or money. This paper presents a multiobjective optimization algorithm for expensive-to-evaluate noisy functions for robotics. We present a method for model selection between heteroscedastic and standard homoscedastic Gaussian process regression techniques to create suitable surrogate functions from noisy samples, and to find the point to be observed at the next step. This algorithm is compared against an existing multiobjective optimization algorithm, and then used to optimize the speed and head stability of the sidewinding gait of a snake robot.

In many robotic problems, optimization of the policy for multiple conflicting criteria is required. However this is very challenging due to the existence of noise, which may be input dependent, or heteroscedastic, and the restriction in the number of evaluations, due to robotic experiments which are expensive in time and/or money. This paper presents a multiobjective optimization (MOO) algorithm for expensive-to-evaluate noisy functions for robotics. We present a method for model selection between heteroscedastic and standard homoscedastic Gaussian process regression techniques to create suitable surrogate functions from noisy samples and find the point to be observed at the next step. This algorithm is compared against an existing MOO algorithm which assumes homoscedastic noise, and is then used to optimize the speed and head stability of the sidewinding gait of a snake robot.

Sidewinding is an efficient gait adopted by biological and robotic snakes for locomoting on various terrains. The mechanics of this motion on flat ground and steady state terrains have been thoroughly investigated, while its capability to adapt to changing environments is not as well studied. We demonstrate the capability of a snake robot to automatically adjust gait parameters to optimally move up and down slopes of varying angle. This capability is achieved by three components. First, an efficient offline learning algorithm finds a policy mapping the estimated slope angle to the optimal gait parameters. Next, a robust online state estimation technique infers the local terrain characteristics. Finally, the precomputed policy is consulted online to select the optimal gait parameters for this slope. The efficacy of this approach is verified by robot experiments.

Real world systems often have parameterized controllers which can be tuned to improve performance. Bayesian optimization methods provide for efficient optimization of these controllers, so as to reduce the number of required experiments on the expensive physical system. In this paper we address Bayesian optimization in the setting where performance is only observed through a stochastic binary outcome – success or failure of the experiment. Unlike bandit problems, the goal is to maximize the system performance after this offline training phase rather than minimize regret during training. In this work we define the stochastic binary optimization problem and propose an approach using an adaptation of Gaussian Processes for classification that presents a Bayesian optimization framework for this problem. We propose an experiment selection metric for this setting based on expected improvement. We demonstrate the algorithm’s performance on synthetic problems and on a real snake robot learning to move over an obstacle.

Many practical optimization problems in robotics involve multiple competing objectives - from design trade-offs to performance metrics of the physical system such as speed and energy efficiency. Proper treatment of these objective functions, while commonplace in fields such as economics, is often overlooked in robotics. Additionally, optimization of the performance of robotic systems can be restricted due to the expensive nature of testing control parameters on a physical system. This paper presents a multi-objective optimization (MOO) algorithm for expensive-to-evaluate functions that generates a Pareto set of solutions. This algorithm is compared against another leading MOO algorithm, and then used to optimize the speed and head stability of the sidewinding gait for a snake robot.

The design of a hyper-redundant serial-linkage snake robot is the focus of this paper. The snake, which consists of many fully enclosed actuators, incorporates a modular architecture. In our design, which we call the Unified Snake, we consider size, weight, power, and speed tradeoffs. Each module includes a motor and gear train, an SMA wire actuated bistable brake, custom electronics featuring several different sensors, and a custom intermodule connector. In addition to describing the Unified Snake modules, we also discuss the specialized head and tail modules on the robot and the software that coordinates the motion.

Many controlled systems must operate over a range of external conditions. In this paper, we focus on the problem of learning a policy to adapt a system's controller based on the value of these external conditions in order to always perform well (i.e., maximize system output). In addition, we are concerned with systems for which it is expensive to run experiments, and therefore restrict the number that can be run during training. We formally define the problem setup and the notion of an optimal control policy. We propose two algorithms which aim to find such a policy while minimizing the number of system output evaluations. We present results comparing these algorithms and various other approaches and discuss the inherent tradeoffs in the proposed algorithms. Finally, we use these methods to train both simulated and physical snake robots to automatically adapt to changing terrain, and demonstrate improved performance on test courses with changing environments.

Several categories of optimization problems suffer from expensive objective function evaluation, driving the need for smart selection of subsequent experiments. One such category of problems involves physical robotic systems, which often require significant time, effort, and monetary expenditure in order to run tests. To assist in the selection of the next experiment, there has been a focus on the idea of response surfaces in recent years. These surfaces interpolate the existing data and provide a measure of confidence in their error, serving as a low-fidelity surrogate function that can be used to more intelligently choose the next experiment. In this paper, we robustly implement a previous algorithm based on the response surface methodology with an expected improvement criteria. We apply this technique to optimize open-loop gait parameters for snake robots, and demonstrate improved locomotive capabilities.

This paper presents an approach to accurate and scalable multiple-model (MM) state estimation for hybrid systems with intermittent, cyclical, multimodal dynamics. The approach consists of using discrete-state estimation to identify a system's dynamical and behavioral contexts and determine which motion models appropriately represent current dynamics and which individual and MM filters are appropriate for state estimation. Furthermore, the heirarchical structure of the dynamics is explicitly encoded, which enables detection not only of rapid transitions between motion models but of higher level behavioral transitions as well. This improves the accuracy and scalability of conventional MM state estimation, which is demonstrated experimentally on a mobile robot that exhibits fast-switching, multimodal dynamics.