We present an approach for learning a reactive maneuver policy for a UAV involved in a close-quarters one-on-one aerial engagement. Specifically, UAVs with behaviors learned through reinforcement learning can match or improve upon simple, but effective behaviors for intercept. In this paper, a framework for developing reactive policies that can learn to exploit behaviors is discussed. In particular, the A3C algorithm with a deep neural network is applied to the aerial combat domain. The efficacy of the learned policy is demonstrated in Monte Carlo experiments. An architecture that can transfer the learned policy from simulation to an actual aircraft and its effectiveness in live-flight are also demonstrated.

On typical multi-robot teams, there is an implicit assumption that robots can be trusted to effectively perform assigned tasks. The multi-robot patrolling task is an example of a domain that is particularly sensitive to reliability and performance of robots. Yet reliable performance of team members may not always be a valid assumption even within homogeneous teams. For instance, a robot's performance may deteriorate over time or a robot may not estimate tasks correctly. Robots that can identify poorly performing team members as performance deteriorates, can dynamically adjust the task assignment strategy. This paper investigates the use of an observation based trust model for detecting unreliable robot team members. Robots can reason over this model to perform dynamic task reassignment to trusted team members. Experiments were performed in simulation and using a team of indoor robots in a patrolling task to demonstrate both centralized and decentralized approaches to task reassignment. The results clearly demonstrate that the use of a trust model can improve performance in the multi-robot patrolling task.

Multi-robot patrolling is a problem that has important applications in security and surveillance. However, the optimal task assignment is known to be NP-hard. We consider evenly spacing the robots in a cyclic Traveling Salesman Problem (TSP) tour or partitioning the graph of the environment. The trade-off in performance, overall team travel cost and coordination is analyzed in this paper. We provide both a theoretical analysis and simulation results across multiple environments. The results demonstrate that generally cyclic-based strategies are superior, especially when small teams are used but at the expense of greater team cost, whereas partitioning strategies are especially suitable for larger teams and unbalanced graph topologies. The reported results show that graph topology and team size are fundamental to determine the best choice for a patrol strategy.

One of the common applications for outdoor robots is to follow a path in large scale unknown environments. This task is challenging due to the intensive memory requirements to represent the map, uncertainties in the location estimate of the robot and unknown terrain type and obstacles on the way to the goal. We develop a novel graph-based path planner that is based on only local perceptual information to plan a path in such environments. In order to extend the capabilities of the graph representation, we introduce Exploration Bias, which is a node attribute that can implicitly encode obstacle features at immediate surrounding of a node in the graph, the uncertainty of the planner about a node location and also the frequency of visiting a location. Through simulation experiments, we demonstrate that the resulting path cost and distance that the robot traverses to reach the goal location is not significantly different from those of the previous approaches.

The Rapidly Exploring Random Tree (RRT) algorithm can be applied to the robotic path planning problem and performs well in challenging, dynamic domains. Traditional RRT methods use a binary cost function and they select portions of the tree for expansion based on the Euclidean distance to the target. However, in outdoor navigation, the relative cost of terrain can also provide useful input to a planning algorithm that traditional RRT methods cannot take advantage of. We present the Metric Adaptive RRT (MA-RRT), which integrates planning and fast execution for generating paths over a cost map. The MA-RRT algorithm considers underlying cost of a path when calculating the distance function for tree expansion. A heuristic value is also used for determining distance from a point to the target and an adaptive mechanism is employed for adjusting the heuristic on-line. We have implemented our approach in offline simulations and in outdoor robot experiments, and show that the MA-RRT algorithm can improve upon the quality of the path returned when cost is considered. The trade off between cost consideration and runtime performance is also presented.