Deep reinforcement learning (DRL) has been extensively applied to Multi-Unmanned Aerial Vehicle (UAV) network (MUN) to effectively enable real-time adaptation to complex, time-varying environments. Nevertheless, most of the existing works assume a stationary user distribution (UD) or a dynamic one with predicted patterns. Such considerations may make the UD-specific strategies insufficient when a MUN is deployed in unknown environments. To this end, this paper investigates distributed user connectivity maximization problem in a MUN with generalization to arbitrary UDs. Specifically, the problem is first formulated into a time-coupled combinatorial nonlinear non-convex optimization with arbitrary underlying UDs. To make the optimization tractable, a multi-agent CNN-enhanced deep Q learning (MA-CDQL) algorithm is proposed. The algorithm integrates a ResNet-based CNN to the policy network to analyze the input UD in real time and obtain optimal decisions based on the extracted high-level UD features. To improve the learning efficiency and avoid local optimums, a heatmap algorithm is developed to transform the raw UD to a continuous density map. The map will be part of the true input to the policy network. Simulations are conducted to demonstrate the efficacy of UD heatmaps and the proposed algorithm in maximizing user connectivity as compared to K-means methods.

Enhancing the resilience of fundamental infrastructures such as the power system and the communication system are crucial considering the increasingly frequent occurrence of natural disasters. Unmanned aerial vehicles (UAVs) have been extensively discussed as flexible and effective disaster rescue devices for the communication system but their service quantity and quality are severely constrained by their limited battery capacities. In this paper, we leverage the power provision and computing offloading capabilities of electric vehicles (EVs) to help UAVs offload computing tasks and recharge UAVs to prolong their service period. Different from existing literature, the proposed work explores the potential of ground EVs to serve as both power source and computing offloading devices to help extend the operation period of UAVs. Specifically, a cooperative UAV-EV rescue framework is developed to characterize the cooperation procedure of UAV-EV pairs. Then, a two-tier matching problem is formulated where the lower tier maximizes the UAV operation period leveraging EV’s computing and recharging services while the upper tier matches UAVs with EVs considering the optimized UAV operation period and statuses of different outage regions. The matching problem is an integer linear programming problem in nature and can be efficiently solved by CVX. Simulation results validate the effectiveness of the cooperative framework on the service utility and UAV operation period extension compared to benchmarks.

With the electrification of the automobile system, the overload problem is being incurred by the increased charging demands from the electric vehicles (EVs). To avoid overloading in the power grid, microgrids (MGs) can be integrated to assist the power balancing. In addition, a coordinated charging strategy among EVs can mitigate the overload problem based on spatially and temporally varying distribution of vehicle traffic in transportation. However, few works have been done on the integration of power system and transportation system in large-scale realistic EV networks. In this paper, both the power distribution and transportation systems are integrated in the high-fidelity and at-scale co-simulation models. Specifically, an LSTM-based prediction model of vehicle traffic distribution is first built and trained over realistic vehicle trace files. The predicted vehicle traffic distribution is exploited to forecast the future EV charging demand. The distribution system is then simulated to describe how the loads (e.g., controllable loads and EVs) and supplies (e.g., distributed generations and energy storages in MGs) impact a power system across the region at scale. Based on the forecast EV loads and co-simulation results from the integrated system, a spatio-temporal coordinated fast EV charging strategy is developed and executed in a distributed way to improve the reliability and resilience of the power systems. Numerical results demonstrate that our proposed strategy can improve the total EV charging performance in the power system while maintaining the power balance of the networked MGs.

The increasingly frequent occurrence of natural disasters has severely interfered with the operation of fundamental infrastructures such as power, transportation, and communication systems. For these decades-old infrastructures, enhancing the system resilience requires extremely high upgrade expenditure. Therefore, more flexible and cost-efficient solutions are in urgent demand. Equipped with on-broad large-capacity batteries, electric vehicles (EVs) could serve as mobile post-disaster rescue devices, namely mobile energy storage (MES). This paper proposes a flexible post-disaster rescue scheme using mobile and connected EVs as MESs to supply emergency resources before the fundamental infrastructures fully recover. Different from existing literature, this paper uncovers the potential energy supply and communication capabilities of MESs to provide damaged areas with on-demand energy and communication resources. Specifically, the uncertainty of natural disasters of tornadoes and flooding is modelled during different scenario generations. Then, a two-stage stochastic programming problem is formulated to determine the MES deployment location in the pre-disaster stage and the MES service operation in the post-disaster stage. The generated disaster scenarios are integrated into the formulated problem to ensure a statistically optimal result. Simulation results validate the optimality of the proposed scheme compared to benchmark schemes.

Multi-agent reinforcement learning has been applied to Unmanned Aerial Vehicle (UAV) based communication networks (UCNs) to effectively solve the problem of time-coupled sequential decision making while achieving scalability. Nevertheless, a transverse comparison on the impact of different levels of inter-agent information exchange on the learning convergence has not been well studied. In this work, we study a distributed user connectivity maximization problem in a UCN, aiming to obtain a trajectory design to optimally guide UAVs' movements in a time horizon to maximize the accumulated number of connected users. Specifically, the problem is first formulated into a time- coupled mixed-integer non-convex optimization problem. A two- stage user association policy is proposed to determine the UAV- user connectivity. A multi-agent deep Q learning algorithm is then designed to solve the optimization, featuring four different levels of information exchange and reward function design. Simulations are conducted to compare the convergence speed and total number of connected users per episode between different levels. The results show that exchanging state information with a deliberated task-specific reward function design yields the best convergence performance in both cases of stationary and dynamic user distributions.

This work studies optimal solar charging for solar-powered self-sustainable UAV communication networks, considering the day-scale time-variability of solar radiation and user service demand. The objective is to optimally trade off between the user coverage performance and the net energy loss of the network by proactively assigning UAVs to serve, charge, or land. Specifically, the studied problem is first formulated into a time-coupled mixed-integer non-convex optimization problem, and further decoupled into two sub-problems for tractability. To solve the challenge caused by time-coupling, deep reinforcement learning (DRL) algorithms are respectively designed for the two sub-problems. Particularly, a relaxation mechanism is put forward to overcome the “dimension curse” incurred by the large discrete action space in the second sub-problem. At last, simulation results demonstrate the efficacy of our designed DRL algorithms in trading off the communication performance against the net energy loss, and the impact of different parameters on the tradeoff performance.

In recent years, with the continuous development of internet of things (IoT) technology, many fields have benefited a lot, including the maritime transportation system (MTS). But there are also corresponding risks, such as security and privacy, interference attacks, ransomware attacks, and so on. How to ensure the reliability and efficiency of information transmission is very important for maritime transportation system. In order to solve this problem, we propose an IoT-enabled maritime transport communication system, which is a distributed system composed of base stations and offshore buoys, and uses the unique structure of the blockchain to solve the problems of security and reliability in the network. There are two main advantages: First, the decentralized network is reliable and can handle node failures. Second, the use of blockchain technology can integrate computing resources into the entire network to support different tasks, while taking into account information security and transaction security. On this basis, with the help of edge computing technology, we have also improved the energy efficiency and performance of IoT devices in the system.

Vehicular communication networks hold promise to significantly improve road safety by giving both automated vehicles and human drivers improved awareness and advanced warning to emergencies. The emergency messages are broadcasted upon emergency detection, but this does not guarantee recipients will be able to avoid collision. In this paper, we introduce a method to relate the delay tolerance of each vehicle in the network directly to the transmission range by taking into consideration the reaction time of the drivers in order to ensure each vehicle in the network can avoid a collision. The system utilizes a digital-twin system to maintain network awareness and accounts for the coexistence of automated vehicles and human driving vehicles and allows the network to minimize transmission range while effectively assuring the safety. The proposed strategy is tested in simulated road scenarios generated from measured highway traffic data. The simulation results demonstrate the efficacy of the proposed strategy through extensive evaluation of multiple traffic scenarios.

The rechargeable battery of a plug-in electric vehicle (PEV) endows the PEV with dual roles in the power grid as power load and mobile energy storage (MES). Owing to the technical advancement of autonomous driving, private PEVs that are parked most of the day can be used as private MESs (PMESs) to autonomously deliver energy for overloaded charging stations (CSs). In this paper, we investigate an energy compensation problem where PMESs are scheduled to deliver energy to overloaded CSs so that the energy balance can be achieved while the energy delivery time can be minimized. Based on the time-variant CS operation status and traffic conditions, we propose a pricing-based scheduling scheme that considers both PMES navigation and incentive price design. First, to navigate PMESs in the energy-capacitated transportation system, a minimum-cost flow problem is formulated to minimize the energy delivery time. Then, the incentive price is determined to encourage PMESs to follow the optimal navigation results for energy delivery. Simulations are conducted based on the traffic data of California highway to validate the effectiveness of the proposed scheduling scheme.

Internet of Everything (IoE) brings unprecedented challenges regarding scability, mobility, availability, and security to wireless communications. Cybertwin emerges as a promising paradigm for the next-generation mobile network, i.e., 6G. Basically, it serves as the communication anchor of a user at the edge and performs fundamental authentication and network resources control functionalities. Cybertwin is also an indispensable enabler of the cloud native network paradigm and can efficiently support the digital twin and metaverse. With cybertwin, heterogeneous access networks can be easily exploited in a synergic manner, such that advanced applications, such as multiscreen multistream rich media delivery, can be realized with guaranteed QoS. Furthermore, a user’s activities in cyberspace can be recorded naturally which becomes his/her/its digital asset. In the future, cybertwin may become the personal assistant and even an immortal second life of the user.

Multi-Unmanned Aerial Vehicle (UAV) control is one of the major research interests in UAV-based networks. Yet few existing works focus on how the network should optimally react when the UAV lineup and user distribution change. In this work, proactive self-regulation (PSR) of UAV-based networks is investigated when one or more UAVs are about to quit or join the network, with considering dynamic user distribution. We target at an optimal UAV trajectory control policy which proactively relocates the UAVs whenever the UAV lineupis about tochange, rather than passively dispatches the UAVsafterthe change. Specifically, a deep reinforcement learning (DRL)-based self-regulation approach is developed to maximize the accumulated user satisfaction (US) score for a certain period within which at least one UAV will quit or join the network. To handle the changed dimension of the state-action space before and after the lineup changes, the state transition is deliberately designed. To accommodate continuous state and action space, an actor-critic based DRL, i.e., deep deterministic policy gradient (DDPG), is applied with better convergence stability. To effectively promote learning exploration around the timing of lineup change, an asynchronous parallel computing (APC) learning structure is proposed. Referred to as PSR-APC, the developed approach is then extended to the case of dynamic user distribution by incorporating time as one of the agent states. Finally, numerical results are presented to demonstrate the convergence and superiority of PSR-APC over a passive reaction method, and its capability in jointly handling the dynamics of both UAV lineup and user distribution.

Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one of the major research interests in UAV based networking. Yet few existing works have focused on how the network should react around the timing when the UAV lineup is changed. In this work, we study proactive self-remedy of energy-constrained UAV networks when one or more UAVs are short of energy and about to quit for charging. We target at an energy-aware optimal UAV control policy which proactively relocates the UAVs when any UAV is about to quit the network, rather than passively dispatches the remaining UAVs after the quit. Specifically, a deep reinforcement learning (DRL)-based self remedy approach, named SREC-DRL, is proposed to maximize the accumulated user satisfaction scores for a certain period within which at least one UAV will quit the network. To handle the continuous state and action space in the problem, the state-of-the-art algorithm of the actor-critic DRL, i.e., deep deterministic policy gradient (DDPG), is applied with better convergence stability. Numerical results demonstrate that compared with the passive reaction method, the proposed SREC-DRL approach shows a 12.12% gain in accumulative user satisfaction score during the remedy period.

Supported by the technical development of electric battery and charging facilities, plug-in electric vehicle (PEV) has the potential to be mobile energy storage (MES) for energy delivery from resourceful charging stations (RCSs) to limited-capacity charging stations (LCSs). In this paper, we study the problem of using on-road PEVs as MESs for energy compensation service to compensate charging station (CS) overload. A price-incentive scheme is proposed for power system operator (PSO) to stimulate on-road MESs fulfilling energy compensation tasks. The price-service interaction between the PSO and MESs is characterized as a one-leader, multiple-follower Stackelberg game. The PSO acts as a leader to schedule on-road MESs by posting service price and on-road MESs respond to the price by choosing their service amount. The existence and uniqueness of the Stackelberg equilibrium are validated, and an algorithm is developed to find the equilibrium. Simulation results show the effectiveness of the proposed scheme in utility optimization and overload mitigation.

Hybrid beamforming (BF) is a widely considered strategy to enable downlink multiuser transmission for mmWave communication systems. However, current mmWave WiFi standard, the IEEE 802.11 ad, does not support hybrid BF because it could only serve one user at a time. Thus, it is important to implement hybrid BF based on IEEE 802.11.ad such that it could be applied in future mmWave WiFi. In this paper, we propose a hybrid BF scheme compatible with the IEEE 802.11 ad, and then analyze its BF overhead and throughput gain. Theoretical analysis and simulation results show that with the increasing number of users, BF overhead increases linearly, whereas the throughput gain increases and then decreases. Specifically, we analyze the tradeoff between the hybrid BF overhead and the throughput gain of multiuser transmission enabled by hybrid BF based on an IEEE 802.11 ad setting. Our finding suggests that there is an optimal number of users that a hybrid BF enabled mmWave communication systems should support.

This paper investigates quality of service (QoS) provisioning for Internet of Things (IoT) in long-term evolution advanced (LTE-A) heterogeneous networks (HetNets) with partial spectrum usage (PSU). In HetNets, the IoT users with ubiquitous mobility support or low-rate services requirement can connect with macrocells (MCells), while femtocells (FCells) with PSU mechanism can be deployed to serve the IoT users requiring high-data-rate transmissions within small coverage. Despite the great potentials of HetNets in supporting various IoT applications, the following challenges exist: 1) how to depict the unplanned random behaviors of the IoT-oriented FCells and cope with the randomness in user QoS provisioning and 2) how to model the interplay of resource allocation (RA) between MCells and FCells under PSU mechanism. In this work, the stochastic geometry (SG) theory is first exploited to statistically analyze how the unplanned random behaviors of the IoT-oriented FCells impact the user performance, considering the user QoS requirements and FCell PSU policy. Particularly, to satisfy the QoS requirements of different IoT user types, the concept of effective bandwidth (EB) is leveraged to provide the users with probabilistic QoS guarantee, and a heuristic algorithm named QA-EB algorithm is proposed to make the EB determination tractable. Then, the interplay of RA between the MCells and FCells is formulated into a two-level Stackelberg game, where the two parties try to maximize their own utilities through optimizing the macro-controlled interference price and the femto-controlled PSU policy. A backward induction method is proposed to achieve the Stackelberg equilibrium. Finally, extensive simulations are conducted to corroborate the derived SINR and ergodic throughput performance of different user types and demonstrate the Stackelberg equilibrium under varying user QoS requirements and spectrum aggregation capabilities.

In this paper, a new MAC protocol for LTE over unlicensed spectrum (LTE-U) is presented that allows friendly co-existence of LTE-U with other unlicensed wireless networks, including Wi-Fi. Specifically, in a time-slotted LTE-U system, LTE- U users can transmit continuously for a period after a successful channel reservation during the spectrum sensing period. Following each LTE transmission period, a certain duration is reserved for asynchronous Wi-Fi transmissions. By adaptively adjusting the periods of LTE transmissions, Wi-Fi transmissions, and spectrum sensing, different levels of Wi-Fi protection can be achieved. Based on the proposed MAC, an analytical model is developed to study the throughput performance of both LTE-U and Wi-Fi, considering the asynchronous transmission nature of Wi-Fi within the time-slotted MAC structure. Impacts of the protocol parameters, i.e., the periods of LTE/Wi-Fi transmissions and spectrum sensing, on the throughput performance of LTE-U and Wi-Fi are also investigated. Extensive simulation results are provided to validate the analysis.

A vehicle-to-vehicle (V2V) (dis)charging strategy can provide charging plans for gridable electric vehicles (GEVs), aiming to offload the heavy power loads from the electric power system. However, designing an efficient online V2V (dis)charging strategy to achieve optimal energy utilization is still an open issue. In this paper, we propose a semi-distributed online V2V (dis)charging strategy at a swapping station based on price control. Specifically, based on the electricity price control strategy, GEVs are motivated to contribute to a V2V energy transaction due to expected high revenue for discharging GEVs and low cost for charging GEVs. The Oligopoly game and Lagrange duality optimization techniques are exploited to address the associated optimal V2V (dis)charging strategies. Simulation results are presented to demonstrate the performance of the proposed V2V (dis)charging strategy.

Carrier Aggregation is considered as a key revolution in Long Term Evolution-Advanced systems to meet the explosively increasing aspiration for high data rates. Unlike extensive simulative evaluations on CA in literature, current theoretical analysis on CA is not convincing due to lack of effective interference modeling. In this paper, we exploit the theory of stochastic geometry to provide tractable statistical interference modeling for downlink CA in LTE-A systems. Our objective is to demonstrate the benefits of CA by comparing the user performance between the legacy LTE users and LTE-A users. Specifically, we first model the distributions of base stations and users into Poisson Point Processes. Then, the user service probability and subchannel usage in each carrier are calculated for LTE and LTE-A users, respectively. The obtained probabilities are applied to derive the user SINR distribution and ergodic rates. To better clarify the impact of system/user parameters on the investigated performance, a special case is presented where the network is interference-limited and the channel fast fading is considered as Rayleigh fading. Finally, simulation results validate our analytical model and demonstrate that LTE-A users can achieve significantly better SINR and ergodic-rate performance than LTE users when the cell is not heavily loaded.

Drive-thru Internet has become a popular solution vehicular Internet access. However, the quality-of-service provisioning for high-data-rate drive-thru Internet services poses significant challenges upon medium access control (MAC) in a large-scale and highly dynamic vehicular environment. In this paper, to achieve high throughput in drive-thru Internet, we exploit the mobility interdependency between neighboring vehicles and propose a density-adaptive MAC protocol by predicting the vehicle-density dynamics. Specifically, to predict the vehicle-density fluctuations in the drive-thru Internet scenario, we leverage Navier-Stokes equations to characterize the mobility interdependence and the dynamically waving vehicle-density which can be observed from the simulated vehicle traces in VISSIM. In tune with the dynamic vehicle-density, we propose an enhanced MAC protocol to dynamically adjust the contention window (CW) setting in order to improve the throughput. Extensive simulations validate the effectiveness of the utilized mobility model, and demonstrate the efficiency of the proposed MAC protocol in improving the overall resultant system throughput.

Coordinated charging can provide efficient charging plans for electric vehicles (EVs) to improve the overall energy utilization while preventing an electric power system from overloading. However, designing an efficient coordinated charging strategy to route mobile EVs to fast-charging stations for globally optimal energy utilization is very challenging. In this paper, we investigate a special smart grid with enhanced communication capabilities, i.e., a VANET-enhanced smart grid. It exploits vehicular ad-hoc networks (VANETs) to support real-time communications among road-side units (RSUs) and highly mobile EVs for collecting real-time vehicle mobility information or dispatching charging decisions. Then, we propose a mobility-aware coordinated charging strategy for EVs, which not only improves the overall energy utilization while avoiding power system overloading, but also addresses the range anxieties of individual EVs by reducing the average travel cost. Specifically, the mobility-incurred travel cost for an EV is considered in two aspects: 1) the travel distance from the current position of the EV to a charging station; and 2) the transmission delay for receiving a charging decision via VANETs. The optimal mobility-aware coordinated EV charging problem is formulated as a time-coupled mixed-integer linear programming problem. By solving this problem based on Lagrange duality and branch-and-bound-based outer approximation techniques, an efficient charging strategy is obtained. To evaluate the performance of the proposed strategy, a realistic suburban scenario is developed in VISSIM to track vehicle mobility through the generated simulation traces, based on which the travel cost of each EV can be accurately calculated. Extensive simulation results demonstrate that the proposed strategy considerably outperforms the traditional EV charging strategy without VANETs on the metrics of the overall energy utilization, the average EV travel cost, and the number of successfully charged EVs.

In this paper, we investigate the asymptotic performance limits (throughput capacity and average packet delay) of social-proximity vehicular networks. The considered network involves N vehicles moving and communicating on a scalable grid-like street layout following the social-proximity model: Each vehicle has a restricted mobility region around a specific social spot and transmits via a unicast flow to a destination vehicle that is associated with the same social spot. Moreover, the spatial distribution of the vehicle decays following a power-law distribution from the central social spot toward the border of the mobility region. With vehicles communicating using a variant of the two-hop relay scheme, the asymptotic bounds of throughput capacity and average packet delay are derived in terms of the number of social spots, the size of the mobility region, and the decay factor of the power-law distribution. By identifying these key impact factors of performance mathematically, we find three possible regimes for the performance limits. Our results can be applied to predict the network performance of real-world scenarios and provide insight on the design and deployment of future vehicular networks.

In this paper, we analyze the user accommodation capabilities of LTE-A systems with carrier aggregation for the LTE users and LTE-A users, respectively. The adopted performance metric is equivalent capacity (EC), defined as the maximum number of users allowed in the system given the user QoS requirements. Specifically, both LTE and LTE-A users are divided into heterogeneous user classes with different QoS requirements, traffic characteristics and bandwidth weights. Two bandwidth allocation strategies are studied, i.e., the fixed-weight strategy and the cognitive-weight strategy, where the bandwidth weights of different user classes are prefixed under the former and dynamically changing with the cell load conditions under the latter. For each strategy, closed-form expressions of ECs of different user classes are derived for LTE and LTE-A users, respectively. A net-profit-maximization problem is further formulated to discuss the tradeoff among the bandwidth weights. Extensive simulations are conducted to corroborate our analytical results, and demonstrate an interesting discovery that only a slightly higher spectrum utilization of LTE-A users than LTE users can result in a significant EC gain when the user traffic is bursty. Moreover, the cognitive-weight strategy is shown to outperform considerably the fixed-weight one due to stronger adaptability to the cell load conditions.

The Long Term Evolution - Advanced (LTE-A) standard is widely accepted for the 4th generation mobile systems to satisfy the explosive growth of high-data-rate demand. Carrier Aggregation (CA) is considered as one of the most momentous techniques adopted in LTE-A standard. Many studies have been done to analyze the performance of LTE-A systems with CA in terms of average user throughput. However, the system-level capacity analysis of LTE-A systems has not been well studied. In this paper, we explore the downlink admission control process in LTE-A systems with CA to compare the capacities between LTE users and LTE-A users, based on the metric - equivalent capacity. Specifically, taking into account the user heterogeneity, the system evolution is modeled as a birth-death process for each user class based on an effective user traffic generation model. A closed-form relationship between the equivalent capacity and system bandwidth is then derived for a single-carrier LTE-A system with the help of binomial-normal approximation. The relationship is further extended to multi-carrier case for both LTE users and LTE-A users. Finally, simulation results are provided to verify our analytical ones, and demonstrate that the equivalent capacity of LTE-A users surpasses that of LTE users significantly.

In vehicular ad hoc networks (VANETs), improving uploading efficiency is crucial to enabling the copious applications such as reporting sensed data for traffic management or environment monitoring. Depending on the applications, the contents to be uploaded can be of large volumes. Therefore, there exist the fundamental demands of the delivery with high throughput. In this paper, we derive the achievable throughput capacity scaling law for such applications in VANETs as Θ(1/log n), with the number of road-side units scaling as Θ(n/log n). Furthermore, by exploring the mobility diversity among vehicles, we propose a novel two-hop forwarding scheme to improve the throughput performance approaching the throughput capacity. Specifically, the source vehicle distributes the contents to multiple relay vehicles with the largest mobility diversity so that the number of concurrent transmissions can be increased. The simulation results demonstrate the effectiveness of the proposed transmission scheme in terms of the increased throughput performance.

In this paper, the asymptotic capacity and delay performance of social-proximity urban vehicular networks with inhomogeneous vehicle density are analyzed. Specifically, we investigate the case of N vehicles in a grid-like street layout while the number of road segments increases linearly with the population of vehicles. Each vehicle moves in a localized mobility region centered at a fixed social spot and communicates to a destination vehicle in the same mobility region via a unicast flow. With a variant of the two-hop relay scheme applied, we show that social-proximity urban networks are scalable: a constant average per-vehicle throughput can be achieved with high probability. Furthermore, although the throughput and delay of a unicast flow may degrade in a high density area, almost constant per-vehicle throughput Ω(1/log (N)) and almost constant delay O(log2(N)) (except for the polylogarithmic factor) are still achievable with high probability. By identifying the key impact factors of performance mathematically, our results should provide insight on the design and deployment of future vehicular networks.