Command and Control (C2) for Unmanned Aerial Vehicle (UAV) missions requires the dynamic coordination of task planning, environmental perception, and navigation strategies. While Large Language Model (LLM-) modulo planning frameworks have shown promise in addressing such complexity, due to their capabilities to understand natural language and common sense, their limited robustness remains a challenge. Recent adaptions of such frameworks made online planning possible, but they do not scale well with the number of actions. This limits the applicability in robotic frameworks with large sets of parameterized actions, such as those encountered in UAV C2. We introduce Say’n’Fly, an LLM-modulo online planning framework that streamlines the action space by discarding infeasible actions using domain-specific knowledge, while leveraging online heuristic search to mitigate reward uncertainty and automate the C2 processes for UAVs. Test results for our Search and Rescue (SAR) validation scenarios show that Say’n’Fly is 70 % more efficient compared to existing frameworks while maintaining or exceeding success rates.

This paper examines the integration of cognitive mixed-initiative assistance via a Planning Assistance Agent in the context of Manned-Unmanned Teaming operations. The aim is to enhance mission planning, replanning, and execution in complex military air operations. The agent employs a hybrid Mixed-Initiative-Planning approach to autonomously adjust and optimize mission plans in real time, with the objective of reducing pilot workload, increasing situational awareness, and maintaining high mission success rates without increasing the risk of losses of unmanned assets. The agent integrates current environmental data and tactical situation changes into its planning processes, thereby closing the so-called “cognitive loop”. This is made possible by the use of sophisticated algorithms and planning problem modeling languages. The effectiveness of the Agent was evaluated through the participation of German Air Force pilots in both static and dynamic mission simulations. The dynamic simulations were conducted in a fully integrated Manned-Unmanned-Teaming fighter simulator, while the static missions required the pilots to create mission plans on a separate workstation. The simulations assessed the impact of the Agent on mission success and the pilot performance under varying levels of assistance. The results demonstrated that a situation-adapted assistance, which allows for dynamic and autonomous tactical adjustments by the Agent, most effectively enhances operational performance and pilot engagement without overwhelming the pilot or causing the pilot to over rely on automated systems.

Modern ultralight aircraft in general aviation are equipped with an onboard Pilot Assistance System (PAS) as a companion system, meant to guide the pilot in decision-making, e.g. with plan suggestions, especially in critical situations. For more meaningful guidance, the PAS must possess a continuous understanding of the context, i.e. the pilot’s intention, so that decision-making support is relevant. However, in realistic settings, the pilot’s intention is not communicated manually, but can only be proactively monitored by the PAS. This paper explores the possibility of embedding domain expertise using Hierarchical Task Network (HTN) planning to track the pilot’s intention, by recognising the pilot’s current goal task judging from the pilot’s actions. Furthermore, by leveraging probability theory for state estimation, we derive belief values to be associated with the recognised goal task, inferred from already executed actions which are in turn inferred from in-cockpit observable measurement data. Statistical evaluation using data collected from human-in-the-loop tests shows that our method for tracking the pilot’s intention is reliable enough to provide the PAS with a contextual understanding in real time.

Manned-Unmanned Teaming (MUM-T) is an ensemble of manned and unmanned vehicles operating as a team to achieve the same set of goals. Such teaming is highly beneficial, notably in overcoming the limits of direct communication link to the unmanned vehicles, as well as in enhancing capabilities of a manned vehicle by leveraging multiple accompanying unmanned vehicles. However, this can in times overstrain the command and control capacity of the human operator(s) on board of the manned vehicle(s), unless if the unmanned vehicles possess some “understanding” of the human operators’ decision-making behaviors, in which case, they can act like real “team players” to proactively support the manned vehicle, instead of waiting passively for successive commands. In this study, we investigate the possibility of learning decision-making behaviors of a human operator on board of a manned vehicle in charge of commanding multiple accompanying unmanned vehicles. We base our investigation on a rescue mission involving a manned helicopter and several unmanned aerial vehicles to collect training and validation data using software-in-the-loop simulations. By extracting meaningful features and by performing a clustering on the features on the training dataset, we validate and analyze the learned pattern of commanding the unmanned vehicles.

High-Altitude Pseudo-Satellites (HAPS) are long-endurance, fixed-wing, lightweight Unmanned Aerial Vehicles (UAVs) that operate in the stratosphere and offer a flexible alternative for ground activity monitoring/imaging at specific time windows. As their missions must be planned ahead (to let them operate in controlled airspace), this paper presents a Genetic Algorithm (GA)-guided Hierarchical Task Network (HTN)-based planner for multiple HAPS. The HTN allows to compute plans that conform with airspace regulations and operation protocols. The GA copes with the exponentially growing complexity (with the number of monitoring locations and involved HAPS) of the combinatorial problem to search for an optimal task decomposition (that considers the time-dependent mission requirements and the time-varying environment). Besides, the GA offers a flexible way to handle the problem constraints and optimization criteria: the former encodes the airspace regulations, while the latter measures the client satisfaction, the operation efficiency and the normalized expected mission reward (that considers the wind effects in the uncertainty of the arrival-times at the monitoring-locations). Finally, by integrating the GA into the HTN planner, the new approach efficiently finds overall good task decompositions, leading to satisfactory task plans that can be executed reliably (even in tough environments), as the results in the paper show.

Navigating in a vector field is a challenging problem for many autonomous vehicles. This article focuses on a High-Altitude Pseudo-Satellite (HAPS) operating in a wind field with wind magnitude comparable to its airspeed. In addition to navigating from a start to a goal point, the HAPS is expected to carry out patrolling missions spanning long hours/ days, within which the physical environment (e.g. wind field, weather-critical no-go areas, fly zones etc.) vary, resulting in either an improvement or a deterioration of mission fulfillment. A time-dependent hybrid mission planning framework is proposed in this work which consists firstly of a hierarchical strategic planner that produces very quickly multiple sequences of tasks (or rather "premature plans") that are ranked with roughly estimated objective or penalty values, and secondly of a tactical planner that refines the values of each premature plan to help the operator assess the plans better. The framework was implemented on the long-term offline planner for HAPS in patrolling missions. Test results using an independent six degrees-of-freedoms HAPS simulator and historical weather data are provided and analyzed.