The automatic design of soft robots is an intertwined process of evolving morphology and learning control. As reinforcement learning is repeatedly used to learn the control policy for each candidate robot design, the design process becomes time-consuming. So far, the common design paradigm in robotics has been based on a single task. In fact, there is control similarity between different tasks. Learning a controller with combinatorial generalization capabilities across a variety of tasks can significantly reduce the computational cost of the design process. To this end, we propose a cross-task collaborative evolutionary algorithm that constructs a universal controller capable of solving a group of tasks simultaneously. Instead of “one robot, one controller, one task" paradigm, the proposed universal controller is to learn a control policy, which can generalize to unseen morphologies. After the controller learning on easy tasks, the universal controller can be further transferred to new hard tasks. Furthermore, the knowledge transfer is incorporated in the search strategy to enhance the performance of the universal controller. The experimental results on 13 test tasks demonstrate that the proposed algorithm outperforms the SOTA design algorithms on 8 of them. Compared to these algorithms, the proposed algorithm reduces the computational cost by 55% while achieving comparable performance, particularly for unseen hard tasks.

Soft robot design is an intricate field with unique challenges due to its complex and vast search space. In the past literature, evolutionary computation algorithms, including novel probabilistic generative models (PGMs), have shown potential in this realm. However, these methods are sample inefficient and predominantly focus on rigid robots in locomotion tasks, which limit their performance and application in robot design automation. In this work, we propose MorphVAE, an innovative PGM that incorporates a multi-task training scheme and a meticulously crafted sampling technique termed ``continuous natural selection'', aimed at bolstering sample efficiency. This method empowers us to gain insights from assessed samples across diverse tasks and temporal evolutionary stages, while simultaneously maintaining a delicate balance between optimization efficiency and biodiversity. Through extensive experiments in various locomotion and manipulation tasks, we substantiate the efficiency of MorphVAE in generating high-performing and diverse designs, surpassing the performance of competitive baselines.

In practical applications, combinatorial optimization problems demonstrate intrinsic complexity, predominantly characterized by discrete decision variables and fitness evaluations that are both expensive and subject to noise. Surrogate-assisted evolutionary algorithms (SAEAs) are commonly used to solve expensive optimization problems in which expensive fitness evaluations are replaced by computationally cheaper surrogate models. The quality and quantity of training data are two crucial factors affecting surrogate models' accuracy, especially in noisy environments. Implicit and explicit averaging are two straightforward and effective noise-tolerant techniques, both entirely applicable to combinatorial optimization problems. In scenarios where fitness evaluations are subject to noise, and the allotted number of evaluations is constrained, implicit averaging tends to yield a considerable quantity of training data with diminished quality, whereas explicit averaging exhibits the opposite trend. This paper discusses which of these two noise-tolerant techniques is more suitable for embedding into SAEAs. The results of six multidimensional knapsack problems show that explicit averaging is a good choice, regardless of whether the noise type is additive or multiplicative.

In the field of robotic control, designing individual controllers for each robot leads to high computational costs. Universal control policies, applicable across diverse robot morphologies, promise to mitigate this challenge. Predominantly, models based on Graph Neural Networks (GNN) and Transformers are employed, owing to their effectiveness in capturing relational dynamics across a robot’s limbs. However, these models typically employ homogeneous graph structures that overlook the functional diversity of different limbs. To bridge this gap, we introduce HeteroMorpheus, a novel method based on heterogeneous graph Transformer. This method uniquely addresses limb heterogeneity, fostering better representation of robot dynamics of various morphologies. Through extensive experiments we demonstrate the superiority of HeteroMorpheus against state-of-the-art methods in the capability of policy generalization, including zero-shot generalization and sample-efficient transfer to unfamiliar robot morphologies.

Hypervolume (HV)-based evolutionary algorithms have been widely used to handle many-objective optimization problems. In such algorithms, HV-based environmental selection (HVES), which aims at selecting a subpopulation with the maximal HV from the current population, plays a crucial role in guiding evolution. However, the computation time of HV increases exponentially with the number of objectives, making the HVES task an expensive optimization problem. In this article, we propose an efficient surrogate-assisted greedy inclusion algorithm to deal with computationally expensive HVES tasks. It uses a lightweight surrogate model, radial basis function network, to replace the most time-consuming calculations. In addition, an$L_{1}$-norm distance-based filter is performed as a preselection operator to reduce the search space and avoid some unnecessary calculations. Considering the inevitable approximation errors of surrogate models, we also design an online sampling strategy to enhance the reliability of selected solutions. The proposed algorithm is tested on two types of datasets and compared with six state-of-the-art greedy algorithms. Experimental results show that the proposed algorithm performs excellently on most datasets.

The automatic design of soft robots characterizes as jointly optimizing structure and control. As reinforcement learning is gradually used to optimize control, the time-consuming controller training makes soft robots design an expensive optimization problem. Although surrogate-assisted evolutionary algorithms have made a remarkable achievement in dealing with expensive optimization problems, they typically suffer from challenges in constructing accurate surrogate models due to the complex mapping among structure, control, and task performance. Therefore, we propose an action inheritance-based evolutionary algorithm to accelerate the design process. Instead of training a controller, the proposed algorithm uses inherited actions to control a candidate design to complete a task and obtain its approximated performance. Inherited actions are near-optimal control policies that are partially or entirely inherited from optimized control actions of a real evaluated robot design. The action inheritance plays the role of surrogate models where its input is the structure and output is the near-optimal control actions. We also propose a random perturbation operation to estimate the error introduced by inherited control actions. The effectiveness of our proposed method is validated by evaluating it on a wide range of tasks, including locomotion and manipulation. Experimental results show that our algorithm is better than the other three state-of-the-art algorithms on most tasks when only a limited computational budget is available. Compared with the algorithm without surrogate models, our algorithm saves about half the computing cost.

Various evolutionary algorithms (EAs) have been proposed to address feature selection (FS) problems, in which a large number of fitness evaluations are needed. With the rapid growth of data scales, the fitness evaluation becomes time consuming, which makes FS problems expensive optimization problems. Surrogate-assisted EAs (SAEAs) have been widely used to solve expensive optimization problems. However, the SAEAs still face difficulties in solving expensive FS problems due to their high-dimensional discrete decision variables. To address this issue, we propose an SAEA with parallel random grouping for expensive FS problems, in which three main components consist. First, a constraint-based sampling strategy is proposed, which considers the influence of the constraint boundary and the number of selected features. Second, a high-dimensional FS problem is randomly divided into several low-dimensional subproblems. Surrogate models are then constructed in these low-dimensional decision spaces. After that, all the subproblems are optimized in parallel. The process of random grouping and parallel optimization continues until the termination condition is met. Finally, a final solution is chosen from the best solution in the historical search and the best solution in the last population using a random, distance-, or voting-based method. Experimental results show that the proposed algorithm generally outperforms traditional, ensemble, and evolutionary FS methods on 14 datasets with up to 10 000 features, especially when the required number of real fitness evaluations is limited.