Mobile edge computing (MEC) is a key technology to support computationally intensive mobile applications with stringent latency requirements. With MEC servers deployed at network edge (e.g., base stations), the computational tasks generated by various applications can be offloaded to nearby edge nodes (ENs) and timely processed there. Meanwhile, future mobile applications will be more diverse and complex, which will need to be supported by a large number of complicated programs. As the storage space of ENs is limited, it is infeasible for each EN to store the program codes of all applications. Thus, it is necessary to optimize program placement at ENs to fully harvest the potential of MEC. In this paper, we investigate the problem of program placement and user association in storage-limited MEC networks. Such a problem is formulated as a sequential decision-making problem. We first consider the single EN scenario and propose an online learning-based solution. We then propose a solution framework for the multi-EN scenario, where we decompose the original problem into three subproblems and iteratively solve them with low-complexity approaches. Simulation results show that the average latency achieved by our proposed schemes is 30% to 70% lower than two benchmark schemes and is on average less than 10% higher than a lower bound.

In this paper, we investigate the problem of task offloading and resource allocation in non-orthogonal multiple access (NOMA) based multi-beam satellite mobile edge computing (SMEC) systems, aiming to minimize the average task completion latency over a finite number of time steps. Such a problem is formulated as a sequential mixed-integer programming problem, which cannot be solved by standard optimization techniques. Thus, we convert the problem into a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) with hybrid action space and propose a deep reinforcement learning-based solution. In particular, a multi-agent hybrid Proximal Policy Optimization algorithm (MAHPPO) is designed to address the hybrid action space. Simulation results show that the proposed scheme outperforms several benchmark schemes.

In this paper, a low-complexity beam hopping pattern design with frequency reuse (FR) is proposed to efficiently adapt to the non-uniform traffic demands characteristics of the terrestrial cells. Concretely, an FR-based beam hopping (BH) downlink transmission model is conceived for multi-beam satellite systems. Next, a heuristic BH pattern optimization algorithm with FR and load balancing (BH-FR-LB) is proposed to improve the communication capacity, which is divided into two sub-problems with low complexity. Specifically, the cells are allocated into different frequency sub-bands to optimize the traffic load. The selection of the BH pattern is updated based on the remaining traffic demands of the terrestrial cells at each time slot, thereby achieving a trade-off between spectrum efficiency and inter-beam interference mitigation. Simulation results show that the proposed scheme obtains higher traffic satisfaction rate, while only requiring fewer time slots to achieve the data transmission under low traffic demands.

The proliferation of intelligent Internet of Things (IoT) applications has led to an increase in the complexity of tasks generated by IoT devices putting pressure on the timely execution of these tasks. Mobile edge computing (MEC) has emerged as a promising paradigm to deliver low-latency computing services, enabled by task offloading from users to MEC servers. Meanwhile, as the IoT applications become increasingly diversified, the demand for communication and computing resources significantly varies over different tasks, highlighting the importance of efficient task offloading and resource allocation strategies in supporting low-latency task processing. Considering the heterogeneity of tasks, this article investigates the problem of task offloading and resource allocation strategies in the MEC system with heterogeneous tasks and propose a deep reinforcement learning (DRL)-based solution. Specifically, we consider task offloading strategies across various combinations of different task types and focus on optimizing channel allocation to minimize task completion delay. The effectiveness of proposed approach in reducing task completion latency is demonstrated through simulation results.

Mobile Edge Computing (MEC) deploys servers at cellular base stations to provide computing services for Industrial Internet of Things (IIoT) applications. However, in remote areas lacking cellular coverage, traditional MEC systems are ineffective. The advancement of Low Earth Orbit (LEO) satellite communication networks introduces satellite-based MEC, deploying servers on satellites, as a promising solution for remote areas. Yet, the limited resources for IIoT devices pose challenges for delivering low-latency computing services. In this paper, we investigate the problem of task partitioning and resource allocation in satellite-assisted MEC systems, aiming to minimize the task completion latency of IIoT devices. Tasks can be divided into multiple subtasks executed by local IIoT devices, MEC servers, or cloud servers. The subtasks can be independent of or dependent on each other. A proposed approach based on Deep Recurrent Q-learning Networks (DRQN) utilizes recurrent neural networks to learn temporal features resulting from satellite movement and optimize accordingly. Given the difficulty in obtaining global state information in large-scale networks, federated learning (FL) is integrated into the DRQN framework to enhance efficient decision-making by aggregating local models. The simulation validates the effectiveness of the proposed approach, demonstrating significant reductions in average latency.

By providing an immersive experience with panoramic views, 360-degree video streaming has gained increasing popularity recently. In many cases, videos are transmitted to mobile users over cellular networks. However, due to the high bandwidth requirement of 360-degree videos and the growing number of users, it is challenging to provide high-quality live streaming services to all users with limited bandwidth. Improving spectral efficiency and reducing bandwidth consumption are two major approaches to address this issue, which can be achieved with non-orthogonal multiple access (NOMA) and scalable video coding (SVC), respectively. In this paper, we apply NOMA and SVC to 360-degree video streaming over a cellular network and propose a multicast scheme called SVCast, aiming to maximize the sum quality of experience of users served by a base station. Such a problem is formulated as an NP-hard problem, and we decompose it into two levels of subproblems. The lower-level subproblem is inter-group spectrum allocation, which is solved by a knapsack approach. The higher-level subproblem is intra-group multicast scheduling, and we propose a recursive algorithm to solve it. Simulation results demonstrate that SVCast improves the system utility by 31.9% on average. Furthermore, SVCast eliminates the need for viewport prediction by aggregating the contents from the viewports of multiple users.

In this paper, we propose an age of incorrect information (AoII)-aware data dissemination scheme for distributed multi-agent systems (MASs). In the proposed scheme, AoII is utilized to measure the importance of data in terms of timeliness and content. We formulate the joint optimization of time slot allocation and agent selection as a decentralized partially observable Markov decision process (Dec-POMDP), with the objective of minimizing the AoII. To solve the Dec-POMDP, a novel multi-agent reinforcement learning algorithm (DV-MAPPO) is proposed. In particular, to tackle challenges posed by the partial observability of global system information, each agent estimates the global system state using variational inference. Moreover, to improve the accuracy of global system state estimation, each agent is given an intrinsic reward that is dominated by the accuracy of estimates. The proposed data dissemination scheme is implemented and evaluated in various missions. Simulation results show that the proposed data dissemination scheme outperforms traditional data dissemination schemes in terms of AoII. Furthermore, in typical multi-agent collaborative tasks, the proposed scheme facilitates more efficient cooperation among multiple agents compared to the data distribution mechanisms that ignore the importance of data.

In this paper, we consider adaptive user equipments (UE) link selection and user association in millimeter-wave (mmWave) networks. We formulate a joint optimization of link selection, resource allocation, and user association, aiming to maximize the sum logarithmic rate of all UEs. The formulated problem is solved by decomposing it into two levels of subproblems. The lower-level subproblem is link selection and resource allocation with a given user association, which is solved by a three-stage process. In the first stage, we establish the D2D relaying architecture by assuming that all UEs are served via D2D relaying. Based on the relaying architecture, we derive the optimal resource allocation in the second stage. Finally, an adaptive link selection algorithm is proposed in the third stage to determine the set of UEs that switch from D2D relaying to multi-beam reflection. The high-level subproblem is user association, for which we solve it with a dual decomposition-based approach. Simulation results indicate that compared to benchmark schemes, the average data rate achieved by the proposed scheme is significantly higher than the benchmark schemes and is close to an upper bound. Besides, the proposed scheme achieves a good tradeoff between system performance and fairness.

Massive multiple input multiple output (mMIMO) assisted grant-free random access (RA) (mGFRA) has been considered a promising RA scheme for future machine-type communication (MTC). In mGFRA, the nature that RA user equipments (UEs) blindly interplay each other in the presence of preamble collision degrades the performance of RA UEs. To address the issue, a mMIMO assisted semi-grant-free RA (mSGFRA) scheme is considered in this paper, where a downlink feedback based on the preamble detection after the preamble phase is introduced. With such a feedback, RA UEs experiencing preamble collision are enforced to keep silent in data-transmission phase, which in turn enhances the performance of RA UEs without experiencing preamble collision. To understand the performance behaviours of mSGFRA, we first analyse the preamble detection performance in mSGFRA and reveal that accurate collided-preamble detection can be achieved with the assistance of mMIMO. Then, we analyse and compare the performance of mSGFRA and mGFRA in terms of success probability. Simulation results validate theoretical analysis and confirm the potential superiority of mSGFRA to mGFRA.

The use of highly directional antennas in millimeter wave (mmWave) cellular networks necessitates precise beam alignment between a base station (BS) and a user equipment (UE), which requires beam sweeping over a large number of directions and causes high initial access (IA) delay. Intuitively, such delay can be lowered by using wider beams, as fewer directions need to be swept. However, this results in a weak received signal and higher misdetection probability, which in turn increases the IA delay as more rounds of beam sweeping would be required to discover a UE. In this paper, we propose a multi-armed bandit approach for beamwidth optimization in 5G New Radio (NR) mmWave cellular networks. We aim to find the optimal beamwidths at the BS and the UE that minimize the beam sweeping delay for a successful IA. We first formulate the beamwidth optimization problem based on analyzing the interplay among beamwidth, beam sweeping overhead, and misdetection probability. Then, we propose a two-stage solution framework based on a multi-armed bandit approach. In the first stage, an initial solution of the BS beamwidth and the optimal solution of UE beamwidth are derived. In the second stage, each BS learns its optimal beamwidth by solving a multi-armed bandit problem with a Thompson sampling-based algorithm. Our extensive simulation results show that, the proposed algorithms can decrease the IA delay by more than 50% compared to the traditional fixed-beamwidth schemes.

Accurate identification of the signal type in shared-spectrum networks is critical for efficient resource allocation and fair coexistence. It can be used for scheduling transmission opportunities to avoid collisions and improve system throughput, especially when the environment changes rapidly. In this paper, we develop deep neural networks (DNNs) to detect coexisting signal types based on In-phase/Quadrature (I/Q) samples without decoding them. By using segments of the samples of the received signal as input, a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN) are combined and trained using categorical cross-entropy (CE) optimization. Classification results for coexisting Wi-Fi, LTE LAA, and 5G NR-U signals in the 5-6 GHz unlicensed band show high accuracy of the proposed design. We then exploit spectrum analysis of the I/Q sequences to further improve the classification accuracy. By applying Short-time Fourier Transform (STFT), additional information in the frequency domain can be presented as a spectrogram. Accordingly, we enlarge the input size of the DNN. To verify the effectiveness of the proposed detection framework, we conduct over-the-air (OTA) experiments using USRP radios. The proposed approach can achieve accurate classification in both simulations and hardware experiments.

Mobile edge computing (MEC) is a promising technology to support computationally intensive mobile applications with stringent delay requirements. As MEC applications become much more diverse and complex, it becomes more challenging for an edge node (EN) with limited storage to keep the program codes of all tasks. In this paper, we investigate the problem of program placement and user association in storage-limited MEC systems. Formulating the problem as a sequential decision-making problem, we first derive the solution for a single EN by transforming the formulation into a multi-armed bandit (MBA) problem and solving it via a Thompson sampling (TS) algorithm. We then propose a solution framework for the multi-EN scenario, where we decompose the original problem into three subproblems and solve them with low-complexity approaches. The first subproblem is to learn the task popularity, which we also formulate as a MAB problem and solve it via a TS algorithm. The second subproblem is optimizing program placement under a given user association and we propose a greedy algorithm to solve it. The last subproblem relates to user association, which is solved by a dual decomposition-based approach. Simulation results show that the average latency achieved by our proposed schemes is 30% to 100% lower than two benchmark schemes and is on average less than 10% higher than a lower bound.

Preamble resources are scarce and precious in random access (RA), which need to be efficiently utilized to support critical machine-type communication (MTC) with stringent access requirements. In this article, we study a grant-free RA scenario for critical MTC in the context of massive multiple-input–multiple-output (MIMO). To enhance the access reliability of delay-sensitive devices within a predefined latency budget, two dynamic preamble-resource partitioning (DPP) schemes are proposed. Particularly, by leveraging massive MIMO, we first analytically investigate the feasibility of DPP in the considered RA scenario and demonstrate its performance superiority to the conventional scheme. Based on the analytical results, we then propose a greedy DPP scheme that performs locally optimal preamble-resource partitioning in each RA slot. To find the globally optimal DPP solution, we further propose a reinforcement learning (RL)-based scheme by modeling the considered RA scenario as a Markov decision process. Simulation results show the practicality and effectiveness of the proposed DPP schemes. In particular, to achieve a target access failure rate of$1\times 10^{-2}$, the proposed RL-based scheme is able to enhance the RA traffic load by 42% and improve the preamble resource utilization by over 25% compared to the conventional baseline scheme.

Mobile edge computing (MEC) is a key enabler of delay-sensitive vehicle-to-everything (V2X) applications. Determining where to execute a task necessitates accurate estimation of the offloading latency. In this paper, we propose a latency prediction framework that integrates machine learning and statistical approaches. Aided by extensive latency measurements collected during driving, we first preprocess the data and divide it into two components: one that follows a trackable trend over time and the other that behaves like random noise. We then develop a Long Short-Term Memory (LSTM) network to predict the first component. This LSTM network captures the trend in latency over time. We further enhance the prediction accuracy of this technique by employing a k-medoids classification method. For the second component, we propose a statistical approach using a combination of Epanechnikov Kernel and moving average functions. Experimental results show that the proposed prediction approach reduces the prediction error to half of a standard deviation (STD) of the raw data.

Millimeter-wave (mmW) spectrum is a major candidate to support the high data rates of 5G systems. However, due to directionality of mmW communication systems, misalignments between the transmit and receive beams occur frequently, making link maintenance particularly challenging and motivating the need for fast and efficient beam tracking. In this paper, we propose a multi-armed bandit framework, called MAMBA, for beam tracking in mmW systems. We develop a reinforcement learning algorithm, called adaptive Thompson sampling (ATS), that MAMBA embodies for the selection of appropriate beams and transmission rates along these beams. ATS uses prior beam-quality information collected through the initial access and updates it whenever an ACK/NACK feedback is obtained from the user. The beam and the rate to be used during next downlink transmission are then selected based on the updated posterior distributions. Due to its model-free nature, ATS can accurately estimate the best beam/rate pair, without making assumptions regarding the temporal channel and/or user mobility. We conduct extensive experiments over the 28 GHz band using a 4x8 phased- array antenna to validate the efficiency of ATS, and show that it improves the link throughput by up to 182%, compared to the beam management scheme proposed for 5G.

In this paper, we investigate the problem of frame design, resource allocation, and user association in a massive multiple input multiple output (MIMO) heterogeneous network (HetNet) with wireless backhaul (WB) and linear processing. The objective is to maximize the sum downlink rate of all users, subject to constraints on data rates of WBs and fairness-aware constraints. Such a problem is formulated as an integer programming problem with both coupled variables and coupled constraints. We first develop a centralized scheme in which we decompose the original problem into two subproblems and iteratively solve them until convergence to achieve a near-optimal solution. We then propose a distributed scheme by formulating a repeated game among all users and prove that the game converges to a Nash Equilibrium. Simulation studies show that the proposed schemes are adaptive to different network scenarios and traffic patterns, and achieve considerable gains over several benchmark schemes.

Device to device (D2D) relaying and multi-beam reflection are two effective approaches to deal with the blockage problem in millimeter-wave (mmWave) communication, each with its own limitations when serving a large number of user equipments (UE). A combination of D2D relaying and multibeam reflection is expected to enhance the performance, but the selection of UEs to be served by each approache remains a challenge. In this paper, we consider adaptive mode selection between D2D relaying and multi-beam reflection in a time division duplex (TDD) mmWave network. We formulate a joint mode selection and resource sharing problem with the objective of maximizing the sum logarithm rate, and propose a two-stage solution algorithm. In the first stage, we derive the optimal resource sharing solution under the case that all UEs are served by D2D relaying. In the second stage, an adaptive algorithm is proposed to determine the set of UEs that switch from D2D relaying to multi-beam reflection. Simulation results demonstrate that the proposed scheme achieves considerable performance gain compared to several benchmark schemes.

In this paper, we investigate the problem of pilot optimization, resource allocation, and user association in a massive MIMO heterogeneous network (HetNet) with wireless backhaul (WB) and linear processing. The objective is to maximize the sum downlink rate of all users, subject to constraints on data rate of WB and fairness-aware constraints. Such a problem is formulated as an integer programming problem with both coupled variables and coupled constraints. We first develop a centralized scheme in which we decompose the original problem into two subproblems and iteratively solve them until convergence to achieve a near-optimal solution. We then propose a distributed scheme by formulating a repeated game among all users and prove that the game converges to a Nash Equilibrium (NE). Simulation studies show that the proposed schemes are adaptive to different network scenarios and traffic patterns, and achieve considerable gains over several benchmark schemes.

We investigate the problem of base station (BS) ON-OFF switching, user association, and power control in a heterogeneous network (HetNet) with massive multiple input multiple output (MIMO), aiming to turn OFF under-utilized BS's and maximize the system energy efficiency. With a mixed integer programming problem formulation, we first develop a centralized scheme to derive the near optimal BS ON-OFF switching, which is an iterative framework with proven convergence. We further propose two distributed schemes based on game theory, with a bidding game between users and BS's, and a pricing game between wireless service provider and users. Both games are proven to achieve a Nash Equilibrium. Simulation studies demonstrate the efficacy of the proposed schemes.

The nested array, which is implemented by nonuniform antenna placement, is an effective approach to achieve O(N2) degrees of freedom (DoF) with an antenna array of N antennas. Such DoF refers to the number of directions of incoming signals that can be resolved. With the increased number of DoF, an important application of nested array is to nullify the interference signals from multiple directions. In this paper, we apply nested array in a massive MIMO heterogeneous network (HetNet) for interference management. With nested array based interference nulling, each base station (BS) can nullify a certain number of interference signals. A key design issue is to select the interference sources to be nullified at each BS. We formulate this problem as an integer programming problem. The objective is to maximize the sum rate of all users, subject to BS DoF constraints. We propose an approximation scheme to solve this problem and derive a performance upper bound. Simulation results show that the proposed scheme effectively improves the sum rate and achieves a near optimal performance.

In this paper, we investigate the problem of optimal base station (BS) ON-OFF switching and user association in a heterogeneous network (HetNet) with massive MIMO, with the objective to maximize the system energy efficiency (EE). The joint BS ON-OFF switching and user association problem is formulated as an integer programming problem. We first develop a centralized scheme, in which we relax the integer constraints and employ a series of Lagrangian dual methods that transform the original problem into a standard linear programming (LP) problem. Due to the special structure of the LP, we prove that the optimal solution to the relaxed LP is also feasible and optimal to the original problem. We then propose a distributed scheme by formulating a repeated bidding game for users and BS's, and prove that the game converges to a Nash Equilibrium (NE). Simulation studies demonstrate that the proposed schemes can achieve considerable gains in EE over several benchmark schemes in all the scenarios considered.

To provide ubiquitous Internet access under the explosive increase of applications and data traffic, the current network architecture has become highly heterogeneous and complex, making network management a challenging task. To this end, software-defined networking (SDN) has been proposed as a promising solution. In the SDN architecture, the control plane and the data plane are decoupled, and the network infrastructures are abstracted and managed by a centralized controller. With SDN, efficient and flexible network control can be achieved, which potentially enhances network performance. To harvest the benefits of SDN in wireless networks, the software-defined wireless network (SDWN) architecture has been recently considered. In this paper, we first analyze the applications of SDN to different types of wireless networks. We then discuss several important technical aspects of performance enhancement in SDN-based wireless networks. Finally, we present possible future research directions of SDWN.

In this paper, we propose a novel additive cancellation signal (ACS) method for sidelobe suppression in non-contiguous orthogonal frequency division multiplexing (NC-OFDM) based CR systems. The key idea of the proposed method is to dynamically add several additive cancellation symbols on both the primary user (PU) subcarriers and the secondary user (SU) subcarriers, to generate the additive cancellation signals for suppressing the sidelobe power of NC-OFDM signals. Moreover, the ACS method formulates the problem of sidelobe suppression as a quadratically constrained quadratic program (QCQP), and the optimal additive cancellation signal can be obtained by the standard interior-point method. Simulation results show that the proposed ACS method can provide significant sidelobe suppression performance.

In this paper, we investigate the problem of incorporating full-duplex (FD) transmission in cognitive femtocell networks (CFN) to achieve higher spectrum utilization. We aim to maximize the sum rate of a full-duplex cognitive femtocell network (FDCFN) as well as guaranteeing the quality of service (QoS) of users in the form of a required signal to interference plus noise ratios (SINR). We propose a duplex mode selection strategy based on stable roommate matching, as well as a greedy channel allocation algorithm with a proven performance bound. Numerical results show that the proposed schemes effectively improve the sum rate of the FDCFN.

Femtocells are attractive candidates in the future cellular system such as long-term evolution (LTE). However, the interference introduced by femtocells may hinder the improved system performance in LTE networks. In this paper, we consider the challenging problem of interference management in LTE networks with femtocells, propose two frequency reuse patterns and a novel interference avoidance method with the frequency resources of the macrocell reused by femtocells. The key idea of the interference avoidance method is to identify the macrocell user equipments (MUEs) which are near femtocells with the help of spectrum sensing performed by femtocells and avoid MUEs and nearby femtocells using the same frequency resources through spectrum scheduling. Moreover, we conduct simulations to verify that our proposed method is successful in avoiding the potential interference and improving the network performance.