Reinforcement learning (RL) has achieved outstanding success in complex robot control tasks, such as drone racing, where the RL agents have outperformed human champions in a known racing track. However, these agents fail in unseen track configurations, always requiring complete retraining when presented with new track layouts. This work aims to develop RL agents that generalize effectively to novel track configurations without retraining. The naïve solution of training directly on a diverse set of track layouts can overburden the agent, resulting in suboptimal policy learning as the increased complexity of the environment impairs the agent's ability to learn to fly. To enhance the generalizability of the RL agent, we propose an adaptive environment-shaping framework that dynamically adjusts the training environment based on the agent's performance. We achieve this by leveraging a secondary RL policy to design environments that strike a balance between being challenging and achievable, allowing the agent to adapt and improve progressively. Using our adaptive environment shaping, one single racing policy efficiently learns to race in diverse challenging tracks. Experimental results validated in both simulation and the real world show that our method enables drones to successfully fly complex and unseen race tracks, outperforming existing environment-shaping techniques. Website: http://rpg.ifi.uzh.ch/env_as_policy.

It is difficult for traditional multi-objective evolutionary algorithms (MOEAs) to find equivalent and approximate equivalent solutions for multi-modal multi-objective optimization problems (MMOPs). This results in decision-makers having no available alternatives if changes in the external environment render the original solution un-executable. Although some multi-modal multi-objective evolutionary algorithms (MMOEAs) have been designed to solve this problem, their search capabilities are weak. This paper designs a new MMOEA named BIMOEA. It uses improved local convergence fitness for environment selection, in which the distances between an individual and its neighbors are used to average the number of times that this individual is dominated. This improvement can reduce the occurrence probability of the same fitness and enhances the algorithm's ability to screen individuals. In addition, a new crowding distance calculation method is designed to promote population diversity in decision space. It not only calculates the crowding distances between individuals but also calculates the crowding distances between individuals and the boundary of the decision space, which promotes a more even distribution of the population in the decision space and improves the population's ability to explore unknown regions. The experiments on 17 benchmark problems show that BIMOEA can effectively improve search capabilities and provide decision-makers with more comprehensive solutions.