Recurrent Neural Networks (RNNs) are powerful models for sequential tasks, but their training can be computationally intensive. On the other hand, Echo State Networks (ESNs), a specific RNN architecture, simplify this by configuring a fixed, random reservoir of recurrent weights. Instead of setting random weights as in the ESN case, in this work, we investigate the recurrent weight configuration problem of the fully-fledged RNN for the task of Linear Time-Invariant (LTI) system approximation. Our investigation focuses on a specific RNN architecture with a network of recurrent neurons limited to self-loops and linear activation. We demonstrate that as the recurrent weights of this RNN are trained on large datasets, their distribution converges to a near-identical match of an optimal distribution that can be analytically derived using the available domain knowledge of the LTI system. This insight establishes that domain-informed weight configuration is a highly efficient alternative to data-driven training. Building upon this, we propose a novel deterministic algorithm to set the recurrent weights, which significantly improves approximation accuracy. Numerical results show our domain-informed RNN weight configuration achieves up to a four-order-of-magnitude performance gain over conventional ESNs.

Mobile distributed MIMO (MD-MIMO) is an innovative extension of distributed MIMO, where mobile radio nodes with antenna arrays connect wirelessly to a base station. To explore the potential of these systems, we developed a software-defined radio (SDR) testbed and created a prototype implementation. This testbed serves as a platform for research and prototyping of MD-MIMO systems.

Network slicing plays a critical role in enabling multiple virtualized and independent network services to be created on top of a common physical network infrastructure. In this paper, we introduce a deep reinforcement learning (DRL)-based radio resource management (RRM) solution for radio access network (RAN) slicing under service-level agreement (SLA) guarantees. The objective of this solution is to minimize the SLA violation. Our method is designed with a two-level scheduling structure that works seamlessly under Open Radio Access Network (O-RAN) architecture. Specifically, at an upper level, a DRL-based inter-slice scheduler is working on a coarse time granularity to allocate resources to network slices. And at a lower level, an existing intra-slice scheduler such as proportional fair (PF) is working on a fine time granularity to allocate slice dedicated resources to slice users. This setting makes our solution O-RAN compliant and ready to be deployed as an ‘xApp’ on the RAN Intelligent Controller (RIC). For performance evaluation and proof of concept purposes, we develop two platforms, one industry-level simulator and one O-RAN compliant testbed; evaluation on both platforms demonstrates our solution’s superior performance over conventional methods.

This paper focuses on advancing reinforcement learning for challenging environments characterized by partial observability and non-stationarity, such as dynamic spectrum access (DSA). In the literature, the Deep Recurrent Q-Network was introduced to capitalize on the inherent temporal correlations present in DSA. Nevertheless, its practicality is still questionable due to sample inefficiency and slow convergence. We introduce Dyna-ESN, leveraging both model-based and model-free methods by employing Reservoir Computing for generative modeling. Specifically, we utilize Echo State Networks (ESNs) to synthesize samples for enhancing the sample efficiency of a model-free Deep Echo State Q-network, enabling effective operation of agent given limited genuine relevant samples obtained through interaction with environment. To mitigate potential adverse effects of synthetic samples, an evaluation algorithm guides the sample selection process, ensuring reliability. A sample augmentation technique is also introduced to allow agents to collect adequate samples despite controlling the sensing rate and duration of secondary transmissions. Our analysis explores trade-offs between data evaluation and sample efficiency, as well as the bias-variance trade-off of the model, identifying optimal design parameters. Evaluating the performance of Dyna-ESN in DSA scenarios demonstrates its performance benefits over existing methods, paving the way for more efficient and effective techniques in complex dynamic environments.

In dynamic spectrum access (DSA) networks, secondary users (SUs) need to opportunistically access primary users’ (PUs) radio spectrum without causing significant interference. Since the SU-PU interaction is limited, deep reinforcement learning has been introduced to help SUs conduct spectrum access. Specifically, deep recurrent Q network (DRQN) has been utilized in DSA networks for SUs to aggregate information from recent experiences to make spectrum access decisions. DRQN is notorious for its sample efficiency since it needs a rather large number of training samples to tune its parameters which is a computationally demanding task. Deep echo state network (DEQN) has been introduced for DSA networks to address the sample efficiency issue of DRQN. In this work, we compare the convergence of DRQN and DEQN by comparing the upper bounds we obtain on their covering number, a notion of richness. Furthermore, we introduce a method to determine the right hyper-parameters for DEQN, providing system design guidance for DEQN-based DSA networks. Extensive performance evaluation confirms that DEQN-based DSA strategy is the superior choice with regard to computational power while outperforming DRQN-based ones.

In this paper, we derive a theoretical upper bound on the generalization error of reservoir computing (RC), a special category of recurrent neural networks (RNNs). The specific RC implementation considered in this paper is the echo state network (ESN), and an upper bound on its generalization error is derived via the empirical Rademacher complexity (ERC) approach. While recent work in deriving risk bounds for RC frameworks makes use of a non-standard ERC measure and a direct application of its definition, our work uses the standard ERC measure and tools allowing fair comparison with conventional RNNs. The derived result shows that the generalization error bound obtained for ESNs is tighter than the existing bound for vanilla RNNs, suggesting easier generalization for ESNs. With the ESN applied to symbol detection in MIMO-OFDM (Multiple Input Multiple Output-Orthogonal Frequency Division Multiplexing) systems, we show how the derived generalization error bound can guide underlying system design. Specifically, the derived bound together with the empirically characterized training loss is utilized to identify the optimum reservoir size in neurons for the ESN-based symbol detector. Finally, we corroborate our theoretical findings with results from simulations that employ 3GPP standards-compliant wireless channels, signifying the practical relevance of our work.