This work presents an energy-efficient reinforcement learning (RL) solution based on Neuromorphic Computing (NC) to enable opportunistic spectrum access in partially observable wireless environments. To improve the energy efficiency of the underlying spectrum access strategy, we explore Neuromorphic Computing and adopt spiking neural networks. Additionally, the time-dependent aspect of the problem and the necessity for sample efficiency drive us to liquid state machines, a variant of reservoir computing. Nevertheless, a priori hyperparameter optimization of the spiking reservoir is essential for handling state- and time-varying inputs in RL agents; yet, this can undermine model robustness and impede deployment. In response, we examine homeostatic regulation for self-modulating the small-world reservoir’s dynamics, thereby maintaining desired near-chaotic behavior throughout operation. The RL model for opportunistic spectrum access is evaluated under both dynamic spectrum access (DSA), where agents identify temporal spectrum holes for transmission, and spatiotemporal spectrum access (SSA), where agents also aim to minimize coverage overspill without coordination or sharing location data. Numerical analysis demonstrates that the proposed model outperforms existing learning models in the literature for both DSA and SSA, while significantly reducing power consumption.

Mobile distributed MIMO (MD-MIMO) is an innovative extension of distributed MIMO, where mobile radio nodes with antenna arrays connect wirelessly to a base station. To explore the potential of these systems, we developed a software-defined radio (SDR) testbed and created a prototype implementation. This testbed serves as a platform for research and prototyping of MD-MIMO systems.

This paper focuses on advancing reinforcement learning for challenging environments characterized by partial observability and non-stationarity, such as dynamic spectrum access (DSA). In the literature, the Deep Recurrent Q-Network was introduced to capitalize on the inherent temporal correlations present in DSA. Nevertheless, its practicality is still questionable due to sample inefficiency and slow convergence. We introduce Dyna-ESN, leveraging both model-based and model-free methods by employing Reservoir Computing for generative modeling. Specifically, we utilize Echo State Networks (ESNs) to synthesize samples for enhancing the sample efficiency of a model-free Deep Echo State Q-network, enabling effective operation of agent given limited genuine relevant samples obtained through interaction with environment. To mitigate potential adverse effects of synthetic samples, an evaluation algorithm guides the sample selection process, ensuring reliability. A sample augmentation technique is also introduced to allow agents to collect adequate samples despite controlling the sensing rate and duration of secondary transmissions. Our analysis explores trade-offs between data evaluation and sample efficiency, as well as the bias-variance trade-off of the model, identifying optimal design parameters. Evaluating the performance of Dyna-ESN in DSA scenarios demonstrates its performance benefits over existing methods, paving the way for more efficient and effective techniques in complex dynamic environments.

In the midst of the machine learning revolution, there is hope to thrive the ever-growing demand for limited spectrum resources imposed by the growth of wireless devices with a paradigm shift to more intelligent ways to manage and share the radio spectrum. This requirement mandates very energy-efficient solutions that can tackle the rapid changes of the wireless environment. This work considers spiking neural networks, which have been shown to drastically reduce the energy consumption compared to conventional neural networks in a reinforcement learning setup designed for the dynamic spectrum sharing scenario. Moreover, the temporal aspect of the problem and the necessity of sample efficiency motivates incorporating liquid state machines into this design. However, the agents’ state- and time-variant inputs impose a burden of a posteriori hyperparameter optimization for liquid state machines, rendering the deployment of reliable models whose reservoirs operate in favorable regimes very challenging in such a setting. Therefore, we employ a homeostatic learning rule for adaptively tuning small-world reservoir connections to maintain near-chaotic behavior during operation. Simulation results prove the performance of the introduced solution compared with several existing techniques.

The performance of federated learning systems is bottlenecked by communication costs and training variance. The communication overhead problem is usually addressed by three communication-reduction techniques, namely, model compression, partial device participation, and periodic aggregation, at the cost of increased training variance. Different from traditional distributed learning systems, federated learning suffers from data heterogeneity (since the devices sample their data from possibly different distributions), which induces additional variance among devices during training. Various variance-reduced training algorithms have been introduced to combat the effects of data heterogeneity, while they usually cost additional communication resources to deliver necessary control information. Additionally, data privacy remains a critical issue in FL and, thus, there have been attempts at bringing Differential Privacy to this framework as a mediator between utility and privacy requirements. This article investigates the tradeoffs between communication costs and training variance under a resource-constrained federated system theoretically and experimentally, and studies how communication reduction techniques interplay in a differentially private setting. The results provide important insights into designing practical privacy-aware federated learning systems.