Due to network delays and scalability limitations, clustered ad hoc networks widely adopt Reinforcement Learning (RL) for on-demand resource allocation. Albeit its demonstrated agility, traditional Model-Free RL (MFRL) solutions struggle to tackle the huge action space, which generally explodes exponentially along with the number of resource allocation units, enduring low sampling efficiency and high computational complexity. To mitigate these limitations, Model-Based RL (MBRL) offers a solution by generating simulated samples through an environment model, which boosts sample efficiency and stabilizes the training by avoiding extensive real-world interactions. However, establishing an accurate dynamic model for complex and noisy environments necessitates a careful balance between model accuracy and computational complexity & stability. To address these issues, we propose a conditional Diffusion Model (DM) as high-dimensional offline resource allocation planner in multifrequency time division multiple access (MF-TDMA) wireless ad hoc networks. By leveraging the astonishing generative capability of DMs, our approach takes advantage of generated high-quality samples to guide exploration and learn optimal policy. Extensive experiments show that our model outperforms MFRL in average reward and Quality of Service (QoS) while demonstrating comparable performance to other MBRL algorithms.

In modern clustered ad hoc networks, efficient and dynamic resource allocation is crucial for ensuring Quality of Service (QoS) under dynamic and uncertain environments. However, the challenges posed by limited sample efficiency, high interaction cost, and high-dimensional action space limit the effectiveness of the widely adopted Model-Free Reinforcement Learning (MFRL) solutions. In contrast, Model-Based RL (MBRL) offers an alternative approach to boost sample efficiency and stabilize the training by explicitly leveraging a learned environment model. Nevertheless, designing accurate and stable dynamics models in noisy, real-world communication scenarios remains a key bottleneck. To address these issues, we propose a Conditional Diffusion Model Planner (CDMP) for high-dimensional offline resource allocation in clustered ad hoc networks. By leveraging the powerful generative capability of Diffusion Models (DMs), our approach enables the accurate modeling of complex environmental dynamics and utilizes an inverse dynamics model for effective policy planning. Beyond simply adopting DMs in offline RL, we further incorporate the CDMP algorithm with a theoretically guaranteed, uncertainty-aware penalty metric, which theoretically and empirically manifests itself in mitigating the Out-of-Distribution (OOD)-induced distributional shift, a common issue for offline settings with scarce training data. Extensive experiments also show that our model outperforms MFRL in average reward and QoS, while demonstrating superior performance over other MBRL algorithms. These results highlight the practicality and scalability of our model in real-world network resource allocation tasks.

Graph Neural Networks (GNNs) have been gaining more attention due to their excellent performance in modeling various graph-structured data. However, most of the current GNNs only consider fixed-neighbor discrete message-passing, disregarding the importance of the local structure of different nodes and the implicit information between nodes for smoothing features. Previous approaches either focus on adaptive selection for aggregation structures or treat discrete graph convolution as a continuous diffusion process, but none of them comprehensively considered the above issues, significantly limiting the model's performance. To this end, we present a novel approach called Flexible Diffusion Convolution (Flexi-DC), which exploits the neighborhood information of nodes to set a particular continuous diffusion for each node to smooth features. Specifically, Flexi-DC first extracts the local structure knowledge based on the degrees of nodes in the graph data and then injects it into the diffusion convolution module to smooth features. Additionally, we utilize the extracted knowledge to smooth labels. Flexi-DC is an efficient framework that can significantly improve the performance of most GNN architectures. Experimental results demonstrate that Flexi-DC outperforms their vanilla implementations by an average accuracy of 13.24% (GCN), 16.37% (JKNet), and 11.98% (ARMA) on nine graph datasets with different homophily ratios.

Graph neural networks (GNNs) have garnered significant attention for their competitive performance on graph-structured data. However, many existing methods are commonly constrained by the homophily assumption, making them overly reliant on the uniform neighbor propagation, which limits their ability to generalize to heterophilous graphs. Although some approaches extend aggregation to multi-hop neighbors, adapting neighborhood sizes on a per-node basis remains a significant challenge. In view of this, we propose an Evolutionary Graph Neural Network (EGNN) with adaptive structure-level aggregation and label smoothing, offering a novel solution to the aforementioned drawback. The core innovation of EGNN lies in assigning each node apersonalizedneighborhood structure utilizingbehavior-levelcrossover and mutation. Specifically, we first adaptively search for the optimal structure-level neighborhoods for nodes within the solution space, leveraging the exploratory capabilities of evolutionary computation. This approach enhances the exchange of information between the target node and surrounding nodes, achieving a smooth vector representation. Subsequently, we adopt the optimal structure obtained through evolutionary search to perform label smoothing, further boosting the robustness of the framework. We conduct experiments on nine real-world networks with different homophily ratios, where outstanding performance demonstrates that the ability of EGNN can match or surpass SOTA baselines.

Exploring the state space efficiently is a crucial problem in reinforcement learning as it holds significant importance for learning optimal policies. One effective approach involves learning different sub-policies to cover various sub-spaces of the state space, with each sub-policy corresponding to a specific goal. However, the unevenness of the state probability distribution may lead to exploration difficulties in deep reinforcement learning. To overcome this challenge, we propose a Phase State Graph Exploration framework (PSGE), guiding the agent towards more promising directions for exploration. Specifically, we design a graph-based state space exploration framework to separate the combination space into sub-spaces and define the combination space and evaluation criteria for the agent’s sub-policies. In addition, hypernetwork is leveraged to decouple sub-policies and sub-goals, ensuring diversity among the agent’s sub-policies and reward shaping is used to provide dense internal reward signals for policy training, which encourages the agent to learn more efficiently. Experiments on combining control and navigation tasks demonstrate that PSGE performs well in controlling agent across various difficulty level tasks.