Generative Adversarial Imitation Learning (GAIL) replicates expert behaviors by employing adversarial training involving the discriminator and the generator. In theory, GAIL can balance discriminator and generator through considerable online interactions to learn well. But real-world adversarial training is tricky and unstable. Early mistakes by the discriminator can lead to bad learning outcomes, or the generator might just copy average expert actions (mode collapse) instead of diverse strategies. To address these, we propose Process Adversarial Diffusion Imitation Learning (PADIL). Employing a conditional diffusion model as the generator facilitates the generation of a multi-modal strategy, thereby reducing the likelihood of mode collapse and enhancing imitation learning performance with fewer online interactions. At the same time, it lessens the generator’s vulnerability to the fluctuating signals or errors conveyed by the discriminator throughout the training process. Furthermore, we have revised the sample extraction approach of the diffusion policy to tackle the concern of iterative instability under rapidly fluctuating reward signals. Experimental results demonstrate that our method achieves better performance compared to baseline methods.

This paper addresses the asynchronous multi-user dynamic spectrum access (DSA) problem in half-duplex cognitive radio networks (HD-CRNs). The core challenge lies in resolving the decision asynchrony caused by variable action duration under HD constraints, which traditional deep reinforcement learning (DRL) methods fail to address. The problem is formulated as a Macro-Action Decentralized Partially Observable Markov Decision Process (MacDec-POMDP), with a decentralized macro-action-based multi-agent Dueling Double Deep Q-Network (D3QN) framework being proposed. An extra reward mechanism is specifically designed to encourage sustained conflict-free access. Simulation results demonstrate that our approach outperforms synchronous fixed-duration methods by dynamically adjusting access duration. The reward function design significantly improves adaptability, reducing collisions to both primary and secondary users while optimizing the spectrum utilization under HD constraints, validating its effectiveness in asynchronous multi-user DSA.

Imitation learning aims to reproduce expert behaviors without relying on an explicit reward signal. However, real-world demonstrations often present challenges, such as multi-modal, data imbalance, and expensive labeling processes. In this work, we propose a novel semi-supervised imitation learning architecture that learns disentangled behavior representations from imbalanced demonstrations using limited labeled data. Specifically, our method consists of three key components. First, we adapt the concept of semi-supervised generative adversarial networks to the imitation learning context. Second, we employ a learnable latent distribution to align the generated and expert data distributions. Finally, we utilize a regularized information maximization approach in conjunction with an approximate label prior to further improve the semi-supervised learning performance. Experimental results demonstrate the efficiency of our method in learning multi-modal behaviors from imbalanced demonstrations compared to baseline methods.