Deep reinforcement learning has advanced signifi-cantly in recent years, and it is now used in embedded systems in addition to simulators and games. Reinforcement Learning (RL) algorithms are currently being used to enhance device operation so that they can learn on their own and offer clients better services. It has recently been studied in a variety of industrial applications. However, reinforcement learning, especially when controlling a large number of agents in an industrial environment, has been demonstrated to be unstable and unable to adapt to realistic situations when used in a real-world setting. To address this problem, the goal of this study is to enable multiple reinforcement learning agents to independently learn control policies on their own in dynamic industrial contexts. In order to solve the problem, we propose a dynamic multi-agent reinforcement learning (dynamic multi-RL) method along with adaptive exploration (AE) and vector-based action selection (VAS) techniques for accelerating model convergence and adapting to a complex industrial environment. The proposed algorithm is tested for validation in emergency situations within the telecommunications industry. In such circumstances, three unmanned aerial vehicles (UAV-BSs) are used to provide temporary coverage to mission-critical (MC) customers in disaster zones when the original serving base station (BS) is destroyed by natural disasters. The algorithm directs the participating agents automatically to enhance service quality. Our findings demonstrate that the proposed dynamic multi-RL algorithm can proficiently manage the learning of multiple agents and adjust to dynamic industrial environments. Additionally, it enhances learning speed and improves the quality of service.

Reinforcement learning, particularly deep reinforcement learning, has made remarkable progress in recent years and is now used not only in simulators and games but is also making its way into embedded systems as another software-intensive domain. However, when implemented in a real-world context, reinforcement learning is typically shown to be fragile and incapable of adapting to dynamic environments. In this paper, we provide a novel dynamic reinforcement learning algorithm for adapting to complex industrial situations. We apply and validate our approach using a telecommunications use case. The proposed algorithm can dynamically adjust the position and antenna tilt of a drone-based base station to maintain reliable wireless connectivity for mission-critical users. When compared to traditional reinforcement learning approaches, the dynamic reinforcement learning algorithm improves the overall service performance of a drone-based base station by roughly 20%. Our results demonstrate that the algorithm can quickly evolve and continuously adapt to the complex dynamic industrial environment.

In this paper, we study the performance of cooperative millimeter wave (mmWave) networks with hybrid precoding architectures. Considering joint transmissions and BS silence strategy, we propose hybrid precoding algorithms which minimize the sum power consumption of the base stations (BSs), for both fully- and partially-connected hybrid precoding (FHP and PHP, respectively) schemes, for single-carrier and orthogonal frequency-division multiplexing systems. We reformulate the analog precoding part as an equal-gain transmission problem, which only depends on the channel information, and the digital precoding part as a relaxed convex semidefinite program subject to per-user quality-of-service constraints that gives the optimal sum power consumption in terms of the BS silence strategy. In order to reduce the complexity of the hybrid precoding algorithm with optimal BS silence strategy, we propose a sub-optimal hybrid precoding algorithm that iteratively put BSs with small power into the silent mode. The simulation results show that, depending on the parameter settings, the power consumption of the PHP may be dominated by the RF transmit power and it may result in a larger power consumption than the FHP. For the cases with 2 BSs and 4 users, implementation of the FHP and the PHP in cooperative networks reduces the required RF transmit power, compared to the case in a non-cooperative network, by 71% and 56%, respectively.

In this paper, we make theoretical analysis on cross-tier handover in Small Cell Networks (SCNs). The cross-tier handover is defined as the handover occurred from a macro cell tier to a small cell tier, or vice versa. Based on stochastic geometry, we first propose a linear approximation approach to study the coverage of small cells. Using the proposed method, we then derive analytical expressions for cross-tier handover rate and sojourn time inside a small cell, which are the key parameters in mobility performance evaluation of an SCN. According to the analysis, we show that the small cell coverage area can be well represented by a biased circle, and the expected value of small cell size is inversely proportional to the square root of the BS density in the macro cell tier. Another important observation is that the handover rate and sojourn time change linearly with the reciprocal of small cell radius. These analytical results could provide fundamental supports for improving mobility management in SCNs.

In this paper, we consider the uplink of a single-cell massive multiple-input multiple-output (MIMO) system with inphase and quadrature-phase imbalance (IQI). This scenario is of particular importance in massive MIMO systems, where the deployment of lower-cost, lower-quality components is desirable to make massive MIMO a viable technology. Particularly, we investigate the effect of IQI on the performance of massive MIMO employing maximum-ratio combining (MRC) receivers. In order to study how IQI affects channel estimation, we derive a new channel estimator for the IQI-impaired model and show that IQI can substantially downgrade the performance of MRC receivers. Moreover, a low-complexity IQI compensation scheme, suitable for massive MIMO, is proposed which is based on the IQI coefficients' estimation and it is independent of the channel gain. The performance of the proposed compensation scheme is analytically evaluated by deriving a tractable approximation of the ergodic achievable rate and providing the asymptotic power scaling laws assuming transmission over Rayleigh fading channels with log-normal large-scale fading. Finally, we show that massive MIMO effectively suppresses the residual IQI effects, as long as, the compensation scheme is applied.

This paper considers the downlink of a heterogeneous network, where multiple base stations (BSs) can serve the users by non-coherent multiflow beamforming. We assume imperfect channel state information at both BSs and users. The objective is to jointly optimize the precoding, load balancing, and BS operation mode (active or sleep) for improving the energy efficiency of the network. The considered problem is to minimize the weighted total power consumption (both circuit power and dynamic transmit power), while satisfying per-user quality of service constraints and per-BS transmit power constraints. This problem is non-convex, but we prove that for each combination of BS modes, the considered problem has a hidden convexity structure. Thus, the global optimal solution is obtained by an exhaustive search over all possible BS mode combinations. Furthermore, by iterative convex approximations of the non-convex power consumption functions, a heuristic algorithm is proposed to obtain a local optimal solution with low complexity. Simulation results illustrate that our proposed algorithms significantly reduce the total power consumption, compared to the scheme where all BSs are continuously active. This implies that putting a BS into sleep mode by proper load balancing is an important solution for energy savings in heterogeneous networks.

This paper considers a downlink heterogeneous network, where different types of multiantenna base stations (BSs) communicate with a number of single-antenna users. Multiple BSs can serve the users by spatial multiflow transmission techniques. Assuming imperfect channel state information at both BSs and users, the precoding, load balancing, and BS operation mode are jointly optimized for improving the network energy efficiency. We minimize the weighted total power consumption while satisfying quality-of-service constraints at the users. This problem is nonconvex, but we prove that for each BS mode combination, the considered problem has a hidden convexity structure. Thus, the optimal solution is obtained by an exhaustive search over all possible BS mode combinations. Furthermore, by iterative convex approximations of the nonconvex objective function, a heuristic algorithm is proposed to obtain a suboptimal solution of low complexity. We show that although multicell joint transmission is allowed, in most cases, it is optimal for each user to be served by a single BS. The optimal BS association condition is parameterized, which reveals how it is impacted by different system parameters. Simulation results indicate that putting a BS into sleep mode by proper load balancing is an important solution for energy savings.

This paper studies the impact of in-phase and quadrature-phase imbalance (IQI) in two-way amplify-and-forward (AF) relaying systems. In particular, the effective signal-to-interference-plus-noise ratio (SINR) is derived for each source node, considering four different linear detection schemes, namely, uncompensated (Uncomp) scheme, maximal-ratio-combining (MRC), zero-forcing (ZF) and minimum mean-square error (MMSE) based schemes. For each proposed scheme, the outage probability (OP) is investigated over independent, non-identically distributed Nakagami-m fading channels, and exact closed-form expressions are derived for the first three schemes. Based on the closed-form OP expressions, an adaptive detection mode switching scheme is designed for minimizing the OP of both sources. An important observation is that, regardless of the channel conditions and transmit powers, the ZF-based scheme should always be selected if the target SINR is larger than 3 (4.77dB), while the MRC-based scheme should be avoided if the target SINR is larger than 0.38 (-4.20dB).

We investigate the performance of dual-hop two-way amplify-and-forward (AF) relaying in the presence of inphase and quadrature-phase imbalance (IQI) at the relay node. In particular, the effective signal-to-interference-plus-noise ratio (SINR) at both sources is derived. These SINRs are used to design an instantaneous power allocation scheme, which maximizes the minimum SINR of the two sources under a total transmit power constraint. The solution to this optimization problem is analytically determined and used to evaluate the outage probability (OP) of the considered two-way AF relaying system. Both analytical and numerical results show that IQI can create fundamental performance limits on two-way relaying, which cannot be avoided by simply improving the channel conditions.

We analyze the performance of amplify-and-forward dual-hop relaying systems in the presence of in-phase and quadrature-phase imbalance (IQI) at the relay node. In particular, an exact analytical expression for and tight lower bounds on the outage probability are derived over independent, non-identically distributed Nakagami-m fading channels. Moreover, tractable upper and lower bounds on the ergodic capacity are presented at arbitrary signal-to-noise ratios (SNRs). Some special cases of practical interest (e.g., Rayleigh and Nakagami-0.5 fading) are also studied. An asymptotic analysis is performed in the high SNR regime, where we observe that IQI results in a ceiling effect on the signal-to-interference-plus-noise ratio (SINR), which depends only on the level of I/Q impairments, i.e., the joint image rejection ratio. Finally, the optimal I/Q amplitude and phase mismatch parameters are provided for maximizing the SINR ceiling, thus improving the system performance. An interesting observation is that, under a fixed total phase mismatch constraint, it is optimal to have the same level of transmitter (TX) and receiver (RX) phase mismatch at the relay node, while the optimal values for the TX and RX amplitude mismatch should be inversely proportional to each other.

We analyze the performance of dual-hop two-way amplify-and-forward relaying in the presence of in-phase and quadrature-phase imbalance (IQI) at the relay node. In particular, two power allocation schemes, namely, fixed power allocation and instantaneous power allocation, are proposed to improve the system reliability and robustness against IQI under a total transmit power constraint. For each proposed scheme, the outage probability is investigated over independent, non-identically distributed Nakagami- m fading channels, and exact closed-form expressions and bounds are derived. Our theoretical analysis indicates that, without IQI compensation, IQI can create fundamental performance limits on two-way relaying. However, these limits can be avoided by performing IQI compensation at source nodes. Compared with the equal power allocation scheme, our numerical results show that the two proposed power allocation schemes can significantly improve the outage performance, thus reducing the IQI effects, particularly when the total power budget is large.

We study the effect of unreliable backhaul links on the performance of Coordinated Multi-Point (CoMP) techniques. CoMP has emerged as a powerful scheme to mitigate co-channel interference. Economically viable deployment of Heterogeneous Networks (HetNets) will require the use of lower-performance backhaul options, e.g. non-line-of-sight (NLOS) microwave links. Motivated by HetNets, a backhauling model is introduced, by assigning Link Failure Probability (LFP) to backhaul links, for the cooperative clusters. In this paper we analyze the centralized and semi-distributed CoMP architectures. We investigate the probability of deficient backhaul links reducing quality of service, by impeding transmission. By evaluating the average sum rate of users within a CoMP cluster, we show how backhaul link reliability affects the performance of the cooperative cluster. We conclude, that the performance gains offered by CoMP quickly diminish, as the unreliability of the backhaul links grows.

Coordinated multi-point (CoMP) transmission has received a lot of attention, as a way to improve the system throughput in an interference limited cellular system. For joint processing in CoMP, the user equipments (UEs) need to feed back the channel state information (CSI), typically to their serving base stations (BSs). The BS forwards the CSI to a central coordination node (CCN) for precoding. These precoding weights need to be forwarded from the CCN to the corresponding BSs to serve the UEs. In this work, a feedback load reduction technique is employed via partial joint processing to alleviate the CSI feedback overhead. Similarly, to achieve backhaul load reduction due to the precoding weights, scheduling approaches are proposed. The state of the art block diagonalization solution is compared with our proposed constrained and unconstrained scheduling. Our main contribution is the method of choosing the best subset of the BSs and UEs at the CCN that yields the best sum rate under the constraint of efficient backhaul use. In particular, with constrained scheduling, the choice of a smaller subset proportionally reduces the backhaul load. Simulation results based on a frequency selective WINNER II channel model, show that our proposed constrained scheduling outperforms the block diagonalization approach in terms of the average sum rate per backhaul use.

We study the power allocation problem for the downlink of a cooperative system with different node activeness, i.e., each receiving node requests for data transmission according to a certain probability. Data symbols of the active nodes are jointly transmitted from cooperative transmission points using zero-forcing precoding. The problem is cast in form of maximizing the ergodic achievable rate subject to per-transmit-point average total power constraints. The derived power allocation indicates that depending on the channel conditions and receiving nodes' activation probability, the optimal solution takes the form of either greedy power allocation or power sharing allocation.

Coordinated Multi-Point transmission is a promising technique to improve the performance of the users at the cell-edge. To achieve this, in case of a centralized approach, users need to unicast the quantized channel state information (CSI), typically to the anchor base station (BS), and then each BS forwards this information to a central coordination node for precoding and scheduling. In the case of a decentralized approach, users broadcast the quantized CSI such that the coordinating BSs could simultaneously receive the CSI. The advantage of a decentralized approach is that it does not require a central coordination node, thereby not imposing stringent latency constraints on the backhaul. The CSI transmission over the erroneous feedback channel in the uplink gives rise to precoding loss and scheduling loss. In the decentralized framework, the feedback errors could result in BSs receiving a different version of the CSI. In this work, we propose a decentralized opportunistic scheduling approach, which only requires a minimal sharing of scheduling information between BSs. The results show that the sum rate achieved with the proposed method is comparable to that of the centralized approach even when there is a high bit error probability introduced by the feedback channel. We also show that when the bit error probabilities in the feedback channel are less than 10-4, the decentralized approach achieves the sum rate of the centralized approach.

Coordinated multi-point (CoMP) transmission is considered as an efficient technique to improve cell-edge performance as well as system spectrum efficiency. In CoMP-enabled systems, a cluster of coordinated base stations (BSs) are typically assumed to be connected to a control unit (CU) via backhaul links, and the provided performance gain relies heavily on the quality of the channel state information (CSI) available at the CU side. In this paper, we consider the downlink of a CoMP cluster and compare three different CoMP transmission schemes: zero-forcing coherent joint transmission, non-coherent joint transmission and coordinated scheduling. Moreover, for each of the analyzed schemes, the performance in terms of average sum rate of the CoMP cluster is studied with predicted CSI, considering the effects of the feedback and backhaul latency, as well as the user mobility. Compared to zero-forcing coherent joint transmission, we show that non-coherent joint transmission and coordinated scheduling are more robust to channel uncertainty. In addition, depending on the latency, user mobility and user locations, different schemes would achieve the highest average sum rate performance. Hence, a system could switch between the transmission schemes to improve the sum rate.

In this paper, the issues upon user scheduling in the downlink packet transmission for multiple services are addressed for coordinated multi-point (CoMP) OFDMA networks. We consider mixed traffic with voice over IP (VOIP) and best effort (BE) services. In order to improve cell-edge performance and guarantee diverse quality of service (QoS), a utility-based joint scheduling algorithm is proposed, which consists of two steps: ant colony optimization (ACO) based joint user selection and greedy subchannel assignment. We compare the proposed algorithm with the optimal algorithm and the greedy user selection (GUS) based scheme. Via simulation results, we show that the performance of our proposed algorithm well approaches to that of the optimal solution. It is also observed that 95% of BE users are satisfied with average cell-edge data rate greater than 200kbps by using either of the two algorithms. Whereas, our proposed algorithm ensures that more than 95% of VoIP users are satisfied with packet drop ratio less than 2%, compared to 78% by the GUS based algorithm.

This paper addresses the problem of limited CSI feedback in coordinated multi-point (CoMP) networks. Specifically, the system throughput is obtained for block-fading CoMP channels with quantized CSI feedback, and the effect of feedback bit allocation on the system throughput is investigated for different user locations and fading distributions. The analytical and simulation results show that substantial throughput increment is achieved via CoMP joint transmission with very limited number of feedback bits per base station. The effect of optimal bit allocation becomes more important for the user that is located in the CoMP cluster edge areas. Also, the standard Zonal-sampling scheme provides the best bit allocation strategy in many cases, maximizing the system throughput.

In this paper, we study the subchannel (SC) power allocation for orthogonal frequency division multiplexing (OFDM) multiple access points (APs) systems with non-coherent cooperative transmission. The objective is to maximize the total capacity under per-AP power constraints. It can be proved that the optimal solution can be obtained by the combination of an optimal SC partition search and the power allocation across SCs for each feasible partition. Existing work exhaustively searched the optimal SC partition and used Lagrange dual method to compute the power allocation across SCs. Since the entire complexity increases exponentially with the number of SCs, the existing method is unsuitable for practical implementation. In this paper, we propose a novel optimal power allocation algorithm for non-coherent cooperative transmission with a much lower complexity. Firstly, a concept of “cut-off SC” is proposed for searching the optimal SC partition. Then, an efficient optimal power allocation algorithm across SCs is proposed for any given cut-off SC. Simulation results demonstrate that the proposed algorithm is optimal with a polynomial complexity, and ends within an acceptable number of iterations.

In this paper, we address the problem of designing a joint scheduling and power control algorithm in a downlink coordinated multi-point (CoMP) cluster supporting CoMP joint transmission. The objective is to maximize the cell-edge throughput under per-point power constraints. By an analytical derivation, binary power control is proved to be the optimal solution for any given selected user group. Utilizing this analytical result, a centralized and a semi-distributed version of joint user selection and power control algorithms are proposed. Compared to algorithms without considering joint transmission and algorithms without considering power control, simulation results show that the proposed algorithms achieve a good trade-off between joint transmission and interference coordination, which helps to improve the cell-edge performance.

In this paper, we consider a static cluster of base stations where joint processing is allowed in the downlink. The partial joint processing scheme is a user-centric approach where subclusters or active sets of base stations are dynamically defined for each user in the cluster. In frequency selective channels, the definition of the subclusters or active set thresholding of base stations can be frequency adaptive (per resource block) or non-adaptive (averaged over all the resource blocks). Frequency adaptive thresholding improves the average sum-rate of the cluster, but at the cost of an increased user data interbase information exchange with respect to the non-adaptive frequency thresholding case. On the other hand, the channel state information available at the transmitter side to design the beamforming matrix is very limited and rank deficiency problems arise for low values of active set thresholding and users located close to the base station. To solve this problem, an algorithm is proposed that defines a cooperation area over the cluster where the partial joint processing scheme can be performed, frequency adaptive or non-adaptive, for a given active set threshold value.

Coordinated Multi-Point (CoMP) transmission is considered in 3GPP LTE-Advanced as a key technique to improve the cell-edge performance. In order to support joint resource allocation among coordinate cells in CoMP systems, efficient frequency reuse schemes need to be designed. However, most of the existing frequency reuse schemes are not suitable for CoMP transmission due to not considering multi-cell joint transmission scenario in their frequency reuse rule. To solve this problem, a cooperative frequency reuse (CFR) scheme is proposed in this paper, which divides the cell-edge area of each cell into two types of zones, and defines a frequency reuse rule to support CoMP transmission for users in these zones. Compared with the conventional soft frequency reuse (SFR) scheme, simulation results demonstrate that the CFR scheme reduces the blocking probability by more than 50%, and improves the cell-edge throughput by 30~40%, with 5~9% additional cell-average throughput.

In this paper, a novel channel allocation method is proposed to reduce the time-varying effects in downlink OFDMA systems. This method makes use of a Kalman filter in order to predict the user SINR and rate in the next time slot. Based on this estimated rate, the available subcarriers are assigned according to a rate greeding algorithm. Compared with the rate-craving greedy (RCG) algorithm and the rate allocation with fixed increase (RAFI) algorithm, the numerical results show that the proposed algorithm can effectively improve the throughput for a single user, averagely 3 to 7 % more than the RCG algorithm and 1 to 3 % more than the RAFI algorithm. Moreover, the total outage probability is also reduced as the number of users increases, averagely 8 to 11% more than the RCG algorithm and 3 to 7% more than the RAFI algorithm.