Multi-goal long-horizon problems are prevalent in real-world applications. The additional goal space introduced by multi-goal problems intensifies the spatial complexity of exploration; meanwhile, the long interaction sequences in long-horizon problems exacerbate the temporal complexity of exploration. Addressing the great exploration challenge posed by multi-goal long-horizon problems depends not only on the design of algorithms but also on the design of environments and the availability of demonstrations to assist in training. To facilitate the above research, we propose a multi-goal long-horizon Reinforcement Learning (RL) environment based on realistic fixed-wing UAV's velocity vector control, named VVC-Gym, and generate multiple demonstration sets of various quality. Through experimentation, we analyze the impact of different environment designs on training, assess the quantity and quality of demonstrations and their influence on training, and assess the effectiveness of various RL algorithms, providing baselines on VVC-Gym and its corresponding demonstrations. The results suggest that VVC-Gym is suitable for studying: (1) the influence of environment designs on addressing multi-goal long-horizon problems with RL. (2) the assistance that demonstrations can provide in overcoming the exploration challenges of multi-goal long-horizon problems. (3) the RL algorithm designs with the least possible impact from environment designs on the efficiency and effectiveness of training.

This paper addresses the challenge of Velocity Vector Control (VVC) for fixed-wing UAVs using Reinforcement Learning (RL) in the presence of imperfect demonstrations. The multi-objective and long-horizon nature of VVC introduces significant spatial and temporal complexities, complicating RL's exploration. While demonstration-based RL methods can help mitigate exploration challenges, their effectiveness is often limited by the quality of the provided demonstrations. To tackle this, we propose V-Pilot, a novel approach that integrates: (1) a controller equipped with a control law model to reduce action oscillation, thus alleviating temporal exploration issues, and (2) a VVC-specific training workflow for iterative policy refinement and demonstration quality improvement. This framework is designed to enhance the performance of demonstration-based RL under imperfect demonstrations. We evaluate V-Pilot on the fixed-wing UAV RL environment, VVCGym. Experimental results demonstrate that V-Pilot outperforms PID and Behavioral Cloning across multiple performance metrics.

Robotic technologies and applications have recently witnessed remarkable advancements. A major challenge is the shortest-path planning problem of a curvature-bounded vehicle from the known starting configuration to visit a target point and finally return to the starting configuration in an obstacle environment. Spurred by this significant issue in robotic surveillance and patrolling applications, this paper proposed the Two-trip Obstacle-environment Relaxed Dubins Problem (TORDP). In TORDP, the vehicle’s target-visiting heading is a critical variable. Analytical approaches have existed for simpler scenarios than TORDP. However, these approaches are unavailable when solving the complex TORDP simultaneously with bounded curvature, variable target heading and unified ability to tackle with- or without- obstacle cases. Hence, we develop the mixed-integer piecewise-linear program (MIPWLP) approach, making the otherwise intractable complex scenario unifiedly solved with guaranteed good quality. Extensive experiments demonstrate that the proposed approach demonstrates effective performance. Furthermore, the objective approximation error in some cases was analyzed to achieve a length near the optimal length within$h^{2}/(2\sqrt {2})$tolerance wherehis the approximation piece length. The proposed MIPWLP approach could also offer a generalizable optimization framework for broader robotic path-planning applications in constrained environments.

In recent years, the multiple traveling salesmen problem (MTSP or multiple TSP) has received increasing research interest and one of its main applications is coordinated multirobot mission planning, such as cooperative search and rescue tasks. However, it is still challenging to solve MTSP with improved inference efficiency as well as solution quality in varying situations, e.g., different city positions, different numbers of cities, or agents. In this article, we propose an attention-based multiagent reinforcement learning (AMARL) approach, which is based on the gated transformer feature representations for min-max multiple TSPs. The state feature extraction network in our proposed approach adopts the gated transformer architecture with reordering layer normalization (LN) and a new gate mechanism. It aggregates fixed-dimensional attention-based state features irrespective of the number of agents and cities. The action space of our proposed approach is designed to decouple the interaction of agents' simultaneous decision-making. At each time step, only one agent is assigned to a non-zero action so that the action selection strategy can be transferred across tasks with different numbers of agents and cities. Extensive experiments on min-max multiple TSPs were conducted to illustrate the effectiveness and advantages of the proposed approach. Compared with six representative algorithms, our proposed approach achieves state-of-the-art performance in solution quality and inference efficiency. In particular, the proposed approach is suitable for tasks with different numbers of agents or cities without extra learning, and experimental results demonstrate that the proposed approach realizes powerful transfer capability across tasks.

Dubins coverage has been extensively researched to address the coverage path planning (CPP) problem of a known environment for the curvature-constrained robot. However, its fixed-speed assumption prevents the robot from accelerating to reduce the time and limits its flexibility to avoid obstacles. Therefore, this paper presents a collision-free CPP approach (CFC) for the obstacle-constrained environment, which enhances time efficiency by constructing the variable-speed Dubins paths and ensures robot safety by building a risk potential surface for representing the possibility of collision. Furthermore, CFC models the CPP problem as an asymmetric traveling salesman problem (ATSP) and utilizes a graph pruning strategy to reduce the computational cost. Comparison tests with other Dubins coverage methods demonstrate that CFC provides shorter coverage times and better runtimes than the other Dubins coverage methods while preventing collision risk between the robot and obstacles. Physical experiments in a laboratory setting demonstrate the applicability of CFC to the physical robot.

Complete coverage, which is the foundation of many robotic applications, aims to cover an area as quickly as possible. This study investigates the time-limited version of multi-robot complete coverage problem, that is, to find the least number of robots and allocate tasks properly to them such that they can finish a known mission within the time limit. This version of problem can be tackled straightforwardly based on optimizing the task-allocation to a fixed number of robots and enumerating the number. However, the number-fixed problem is NP-hard and the existing algorithm for the number-fixed problem allows intersecting tasks (possibly causing robots' interference) and endures high approximation factor. In this study, the time-limited complete coverage problem is tackled with a multi-objective approach, instead of enumerating robots' number and optimizing each number-fixed problem one by one. The multi-objective GA, Mofint, at first estimates the lower and upper bounds of the number of robots. It abstracts each task as a weighted node of a graph. Then, Mofint evolves individuals, each individual being a forest containing a certain number (within the bounds) of non-intersecting trees. Mofint can finally obtain higher precision than existing work with less time: the approximation factor for Mofint is 1.1 to 1.5 times the ideal allocation when robots' number is fixed, while for existing work is 1.5 to 2. Due to its higher precision, the least number of robots obtained in the experiments by Mofint is 0.6 times of existing work.

Parameterized optimization problems (POPs) belong to a class of NP problems which are hard to be tackled by traditional methods. However, the relationship of the parameters (usually represented as k) makes a POP different from ordinary NP-complete problem in designing algorithms. In this paper, GEAS, an evolutionary computing algorithm (also can be seen as a framework) to solve POPs is proposed. This algorithm organically unifies genetic algorithm (GA) framework and the idea of evolutionary strategy (ES). It can maintain diversity while with a small population and has an intrinsic parallelism property:each individual in the population can solve a same problem that only has a different parameter. GEAS is delicately tested on an NP-complete problem, the Critical Link Set Problem. Experiment results show that GEAS can converge much faster and obtain more precise solution than GA which uses the same genetic operators.