Along with the advance of low-altitude economy, a variety of applications based on Unmanned Aerial Vehicles (UAVs) have been developed to accomplish diverse tasks. In this paper, we focus on the scenario of multiple UAVs performing Federated Learning (FL) tasks. Specifically, a group of UAVs is scheduled to repeatedly visit some Points of Interest (PoIs), collect the data produced by these PoIs, and jointly train a machine learning model based on the collected data. The most challenging issue is how to schedule UAVs to collect data so as to optimize the generalization and convergence of model training under the case that the distributions of the data produced by PoIs have not been known in advance. To address this issue, we propose a novel framework for online FL on distributed unknown data, named OFL-UD2, which is dedicated to online decision-making for UAVs to optimize model training performance. Concretely, we formulate the optimization problem while considering the convergence and quality of trained models as well as energy constraints. Then, we define a utility metric for the data quality of different PoIs and conduct a rigorous convergence analysis for OFL-UD2. Based on the analysis results, we design a two-stage algorithm to determine the scheduling of UAVs. Extensive simulations demonstrate that OFL-UD2can improve model accuracy and speed up running time compared to existing benchmarks significantly.

As a revolutionary technology, autonomous driving is changing the traditional ways of transportation. However, due to the high complexity and uncertainty of the road environment, existing autonomous driving systems still face numerous challenges in practical applications. To address these challenges, we propose a fuzzy knowledge-driven with reward shaping Human-in-the-loop Reinforcement Learning (FR-HRL) framework. The framework incorporates human expert knowledge through a fuzzy rule base and utilizes a Lagrangian method to regulate expert intervention frequency. Additionally, reward shaping is introduced to integrate domain knowledge, while a dual experience replay mechanism is employed to separately manage expert demonstrations and self-generated experiences. Experimental results in an autonomous driving scenario show that FR-HRL achieves improvements of 19.5%-250.0% in success rate, 13.5%-115.3% in total reward, and 90.0%-97.8% in environmental cost reduction compared to baseline methods. The framework performs particularly well in early training stages, where expert guidance significantly accelerates learning while ensuring safety.

In this paper, we investigate the competitive content placement problem in Mobile Edge Caching (MEC) systems, where Edge Data Providers (EDPs) cache appropriate contents and trade them with requesters at a suitable price. Most of the existing works ignore the complicated strategic and economic interplay between content caching, pricing, and content sharing. Therefore, we propose a joint Mean-Field Game framework for mobile edge Caching and Pricing (MFG-CP) in large-scale dynamic MEC systems, which can facilitate distributed optimal decision-making based on the mean-field game theory. Specifi-cally, we first formulate the competitive content placement issue among EDPs as a non-cooperative stochastic differential game. To significantly reduce the communication and computation complexity, we further devise a mean-field model to approximate the collective impact of all EDPs on caching, trading, and sharing, by which each EDP can quickly estimate some unknown information without considerable interactions. Then, we develop a distributed best response scheme based on iterative learning, enabling each EDP to solely customize its optimal caching strategy and pricing policy. Besides, we theoretically prove the existence of a unique MFG equilibrium. Finally, trace-driven simulations demonstrate the effectiveness of MFG-CP compared with some baselines.