In recent years, extensive research has been conducted on radio resource allocation (RRA) in vehicular networks. Many studies have employed multi-agent Deep Reinforcement Learning (DRL) as an effective approach for making decentralized RRA decisions in highly dynamic and uncertain vehicular environments. However, a systematic evaluation and comparison of various multi-agent DRL algorithms in vehicular contexts remain lacking. In this paper, we address this gap by framing RRA problems in Cellular Vehicle-to-Everything (C-V2X) networks as a series of multi-agent interference games with ascending complexity as more realistic factors are introduced. We benchmark performance of classical multi-agent DRL algorithms in these environments. Our results offer insights into the relative significance of different Multi-Agent Reinforcement Learning (MARL) challenges in C-V2X RRA tasks, along with a comparative evaluation of multiple algorithms.

Collaborative autonomous driving holds significant potential to improve the performances of Connected Autonomous Vehicles (CAVs). This paper presents a communication-aware hierarchical driving control mechanism designed to operate under non-ideal Vehicle-to-Vehicle (V2V) communication conditions. To address the impact of delayed and partial observations of CAV, a state augmentation method is first introduced to convert the resulting random-delay partially observable Markov decision process (RD-POMDP) into a standard Markov decision process (MDP), enabling the application of deep reinforcement learning (DRL) algorithms with theoretical convergence guarantees. Based on this formulation, a hierarchical DRL framework is developed, comprising an event-triggered upper-level for driving behavior adaptation at a coarse time scale and a periodic lower-level for motion control at a fine time scale. A modified Twin Delayed Deep Deterministic Policy Gradient with Prioritized Experience Replay (TD3-PER) algorithm is used to train the lower-level motion control policy, while an option-critic framework is employed to train the upper-level behavior policy, leveraging the pretrained low-level policies. Simulation results demonstrate the effectiveness of the proposed mechanism in collaborative driving scenarios with imperfect V2V communications.

The growing adoption of electric buses (EBs) represents a significant step toward sustainable development. By utilizing Internet of Things (IoT) systems, charging stations can autonomously determine charging schedules based on real-time data. However, optimizing EB charging schedules remains a critical challenge due to uncertainties in travel time, energy consumption, and fluctuating electricity prices. Moreover, to address real-world complexities, charging policies must make decisions efficiently across multiple time scales and remain scalable for large EB fleets. In this article, we propose a hierarchical deep reinforcement learning (HDRL) approach that reformulates the original Markov decision process (MDP) into two augmented MDPs. To solve these MDPs and enable multitimescale decision-making, we introduce a novel HDRL algorithm, namely, double actor-critic multiagent proximal policy optimization enhancement (DAC-MAPPO-E). Scalability challenges of the double actor-critic (DAC) algorithm for large-scale EB fleets are addressed through enhancements at both decision levels. At the high level, we redesign the decentralized actor network and integrate an attention mechanism to extract relevant global state information for each EB, decreasing the size of neural networks. At the low level, the multiagent proximal policy optimization (MAPPO) algorithm is incorporated into the DAC framework, enabling decentralized and coordinated charging power decisions, reducing computational complexity and enhancing convergence speed. Extensive experiments with real-world data demonstrate the superior performance and scalability of DAC-MAPPO-E in optimizing EB fleet charging schedules.

In distributed systems, data may partially overlap in sample and feature spaces, that is, horizontal and vertical data partitioning. By combining horizontal and vertical federated learning (FL), hybrid FL emerges as a promising solution to simultaneously deal with data overlapping in both sample and feature spaces. Due to its decentralized nature, hybrid FL is vulnerable to model poisoning attacks, where malicious devices corrupt the global model by sending crafted model updates to the server. Existing work usually analyzes the statistical characteristics of all updates to resist model poisoning attacks. However, training local models in hybrid FL requires additional communication and computation steps, increasing the detection cost. In addition, due to data diversity in hybrid FL, solutions based on the assumption that malicious models are distinct from honest models may incorrectly classify honest ones as malicious, resulting in low accuracy. To this end, we propose a secure and efficient hybrid FL against model poisoning attacks. Specifically, we first identify two attacks to define how attackers manipulate local models in a harmful yet covert way. Then, we analyze the execution time and energy consumption in hybrid FL. Based on the analysis, we formulate an optimization problem to minimize training costs while guaranteeing accuracy considering the effect of attacks. To solve the formulated problem, we transform it into a Markov decision process and model it as a multiagent reinforcement learning (MARL) problem. Then, we propose a malicious device detection (MDD) method based on MARL to select honest devices to participate in training and improve efficiency. In addition, we propose an alternative poisoned model detection (PMD) method considering model change consistency. This method aims to prevent poisoned models from being used in the model aggregation. Experimental results validate that under the random local model poisoning attack, the proposed MDD method can save over 50% training costs while guaranteeing accuracy. When facing the advanced adaptive local model poisoning (ALMP) attack, utilizing both the proposed MDD and PMD methods achieves the desired accuracy while reducing execution time and energy consumption.

In Part I of this two-part paper (Multitimescale Control and Communications with deep reinforcement learning (DRL)—Part I: Communication-Aware Vehicle Control), we decomposed the multitimescale control and communications (MTCCs) problem in cellular vehicle-to-everything (C-V2X) system into a communication-aware DRL-based platoon control (PC) subproblem and a control-aware DRL-based radio resource allocation (RRA) subproblem. We focused on the PC subproblem and proposed the MTCC-PC algorithm to learn an optimal PC policy given an RRA policy. In this article (Part II), we first focus on the RRA subproblem in MTCC assuming a PC policy is given, and propose the MTCC-RRA algorithm to learn the RRA policy. Specifically, we incorporate the PC advantage function in the RRA reward function, which quantifies the amount of PC performance degradation caused by observation delay. Moreover, we augment the state space of RRA with PC action history for a more well-informed RRA policy. In addition, we utilize reward shaping and reward backpropagation prioritized experience replay (RBPER) techniques to efficiently tackle the multiagent and sparse reward problems, respectively. Finally, a sample- and computational-efficient training approach is proposed to jointly learn the PC and RRA policies in an iterative process. In order to verify the effectiveness of the proposed MTCC algorithm, we performed experiments using real driving data for the leading vehicle, where the performance of MTCC is compared with those of the baseline DRL algorithms.

An intelligent decision-making system enabled by vehicle-to-everything (V2X) communications is essential to achieve safe and efficient autonomous driving (AD), where two types of decisions have to be made at different timescales, i.e., vehicle control and radio resource allocation (RRA) decisions. The interplay between RRA and vehicle control necessitates their collaborative design. In this two-part paper (Part I and Part II), taking platoon control (PC) as an example use case, we propose a joint optimization framework of multitimescale control and communications (MTCCs) MTCCs based on deep reinforcement learning (DRL). In this article (Part I), we first decompose the problem into a communication-aware DRL-based PC subproblem and a control-aware DRL-based RRA subproblem. Then, we focus on the PC subproblem assuming an RRA policy is given, and propose the MTCC- PC algorithm to learn an efficient PC policy. To improve the PC performance under random observation delay, the PC state space is augmented with the observation delay and PC action history. Moreover, the reward function with respect to the augmented state is defined to construct an augmented state Markov decision process (MDP). It is proved that the optimal policy for the augmented state MDP is optimal for the original PC problem with observation delay. Different from most existing works on communication-aware control, the MTCC- PC algorithm is trained in a delayed environment generated by the fine-grained embedded simulation of cellular vehicle-to-everything communications rather than by a simple stochastic delay model. Finally, experiments are performed to compare the performance of MTCC- PC with those of the baseline DRL algorithms.

In autonomous vehicle platooning, Vehicle-to-Everything (V2X) communications are leveraged in cooperative adaptive cruise control (CACC) to improve control performance. Since exchanging information at all times incurs significant communication overhead in vehicular networks, it is important to determine when V2X communication is necessary. To solve this problem, we propose a Deep Reinforcement Learning (DRL)-based algorithm named Attention-DDPG, which learns platoon control with Deep Deterministic Policy Gradient (DDPG), and learns when to communicate with an attention network. Specifically, each preceding vehicle is equipped with a deep neural network (DNN), which takes as input its local state and platoon control action and determines whether to transmit its acceleration or not to the following vehicle at each time step. The attention network of a preceding vehicle is trained using the feedback from the following vehicle on the value of V2X information in the form of an advantage function. In order to evaluate Attention-DDPG, simulations are performed using real driving data, and performance is compared with those of two baselines that communicate and do not communicate at all times, respectively. The results demonstrate that Attention-DDPG strikes a competitive tradeoff between control performance and communication overhead while ensuring platoon string stability.

Autonomous vehicles in a platoon determine the control inputs based on the system state information collected and shared by the Internet of Things (IoT) devices. Deep reinforcement learning (DRL) is regarded as a potential method for car-following control and has been mostly studied to support a single following vehicle. However, it is more challenging to learn an efficient car-following policy with convergence stability when there are multiple following vehicles in a platoon, especially with unpredictable leading vehicle behavior. In this context, we adopt an integrated DRL and dynamic programming (DP) approach to learn autonomous platoon control policies, which embeds the deep deterministic policy gradient (DDPG) algorithm into a finite-horizon value iteration framework. Although the DP framework can improve the stability and performance of DDPG, it has the limitations of lower sampling and training efficiency. In this article, we propose an algorithm, namely, finite-horizon-DDPG with sweeping through reduced state space using stationary approximation (FH-DDPG-SS), which uses three key ideas to overcome the above limitations, i.e., transferring network weights backward in time, stationary policy approximation for earlier time steps, and sweeping through reduced state space. In order to verify the effectiveness of FH-DDPG-SS, simulation using real driving data is performed, where the performance of FH-DDPG-SS is compared with those of the benchmark algorithms. Finally, platoon safety and string stability for FH-DDPG-SS are demonstrated.

In this article, we investigate the scheduling issue of diesel generators (DGs) in an Internet of Things (IoT)-Driven isolated microgrid (MG) by deep reinforcement learning (DRL). The renewable energy is fully exploited under the uncertainty of renewable generation and load demand. The DRL agent learns an optimal policy from history renewable and load data of previous days, where the policy can generate real-time decisions based on observations of past renewable and load data of previous hours collected by connected sensors. The goal is to reduce operating cost on the premise of ensuring supply–demand balance. In specific, a novel finite-horizon partial observable Markov decision process (POMDP) model is conceived considering the spinning reserve. In order to overcome the challenge of discrete-continuous hybrid action space due to the binary DG switching decision and continuous energy dispatch (ED) decision, a DRL algorithm, namely, the hybrid action finite-horizon RDPG (HAFH-RDPG), is proposed. HAFH-RDPG seamlessly integrates two classical DRL algorithms, i.e., deep$Q$-network (DQN) and recurrent deterministic policy gradient (RDPG), based on a finite-horizon dynamic programming (DP) framework. Extensive experiments are performed with real-world data in an IoT-driven MG to evaluate the capability of the proposed algorithm in handling the uncertainty due to interhour and interday power fluctuation and to compare its performance with those of the benchmark algorithms.

Autonomous driving vehicles can reduce congestion and improve safety while increasing traffic efficiency. To reflect the quality of driving more comprehensively, the driving safety, efficiency, and occupant comfort should be jointly optimized for autonomous vehicles. Furthermore, in order to cope with complicated traffic environments and achieve satisfactory driving performance, a powerful behavior decision-making module is indispensable for autonomous vehicles. Toward this end, we study a reinforcement-learning (RL)-based method to intelligently make the behavior decision in this article. A Markov decision process (MDP) model is first formulated with a comprehensive reward function, including the effects of driving safety, efficiency, and comfort. The knowledge of the surrounding vehicles is also leveraged to exploit the behavior prediction of the target vehicle. We then propose a behavior decision strategy based on the actor–critic (AC) mechanism, which can efficiently learn both a Gaussian policy function and a linear value function. Finally, the real traffic data are used to build up the simulations for evaluating the performances of the proposed method thoroughly. Simulation results show that our proposed method can significantly reduce the collision rate for autonomous vehicles.

Microgrids (MGs) are small, local power grids that can operate independently from the larger utility grid. Combined with the Internet of Things (IoT), a smart MG can leverage the sensory data and machine learning techniques for intelligent energy management. This article focuses on deep reinforcement learning (DRL)-based energy dispatch for IoT-driven smart isolated MGs with diesel generators (DGs), photovoltaic (PV) panels, and a battery. A finite-horizon partial observable Markov decision process (POMDP) model is formulated and solved by learning from historical data to capture the uncertainty in future electricity consumption and renewable power generation. In order to deal with the instability problem of DRL algorithms and unique characteristics of finite-horizon models, two novel DRL algorithms, namely, finite-horizon deep deterministic policy gradient (FH-DDPG) and finite-horizon recurrent deterministic policy gradient (FH-RDPG), are proposed to derive energy dispatch policies with and without fully observable state information. A case study using real isolated MG data is performed, where the performance of the proposed algorithms are compared with the other baseline DRL and non-DRL algorithms. Moreover, the impact of uncertainties on MG performance is decoupled into two levels and evaluated, respectively.

Electricity theft can be considered as a Nontechnical Loss (NTL) in smart grids, which is very harmful to the power system. Electricity Theft Detection (ETD) is a procedure to detect atypical behaviours in smart grids, which can be achieved via the massive amount of data that is generated by these networks due to using smart meter tools and Information and Communications Technology (ICT). Since the existing methods are not exceptionally robust to detect this type of attack, also considering the strength of the convolutional neural network (CNN), an Ensemble Deep Convolutional Neural Network (EDCNN) algorithm for ETD in smart grids has been proposed. As the first layer of the model, a random under bagging technique is applied to deal with the imbalance data, then deep CNNs are utilized on each subset, and finally, a voting system is embedded as the last part. This study has been conducted on a dataset which contains consumption information of more than 42,000 customers over 24 months. Various performance parameters containing AUC, precision, recall, f1-score and accuracy have been reported as the results.

By leveraging mobile edge computing (MEC), a huge amount of data generated by Internet of Things (IoT) devices can be processed and analyzed at the network edge. However, the MEC system usually only has the limited virtual resources, which are shared and competed by IoT edge applications. Thus, we propose a resource allocation policy for the IoT edge computing system to improve the efficiency of resource utilization. The objective of the proposed policy is to minimize the long-term weighted sum of average completion time of jobs and average number of requested resources. The resource allocation problem in the MEC system is formulated as a Markov decision process (MDP). A deep reinforcement learning approach is applied to solve the problem. We also propose an improved deep Q-network (DQN) algorithm to learn the policy, where multiple replay memories are applied to separately store the experiences with small mutual influence. Simulation results show that the proposed algorithm has a better convergence performance than the original DQN algorithm, and the corresponding policy outperforms the other reference policies by lower completion time with fewer requested resources.

In order to maximize detection precision rate as well as the recall rate, this paper proposes an in-vehicle multisource fusion scheme in Keyword Spotting (KWS) System for vehicle applications. Vehicle information, as a new source for the original system, is collected by an in-vehicle data acquisition platform while the user is driving. A Deep Neural Network (DNN) is trained to extract acoustic features and make a speech classification. Based on the posterior probabilities obtained from DNN, the vehicle information including the speed and direction of vehicle is applied to choose the suitable parameter from a pair of sensitivity values for the KWS system. The experimental results show that the KWS system with the proposed multi-source fusion scheme can achieve better performances in term of precision rate, recall rate, and mean square error compared to the system without it.

Content caching has brought huge potential for the provisioning of non-safety related infotainment services in future vehicular networks. Assisted by multiaccess edge computing, roadside units (RSUs) could become cache-capable and offer fast caching services to moving vehicles for content providers. On the other hand, deep learning makes it possible to accurately estimate the behavior of vehicles, which enables effective proactive caching strategies. However, caching services considering both the mobility of vehicles and storage could incur increased latency and considerable cost due to the cache size needed in RSUs. In this paper, we model such a problem using Markov decision processes, and propose a heuristic Q-learning solution together with vehicle movement predictions based on a long short-term memory network. The optimal caching strategy which minimizes the latency of caching services can be derived by our heuristic εn-greedy training processes. Numerical results demonstrate that our proposed strategy can achieve better performance compared with several baselines under different prediction accuracies.

The Internet of Things (IoT) connects a huge number of resource-constraint IoT devices to the Internet, which generate massive amount of data that can be offloaded to the cloud for computation. As some of the applications may require very low latency, the emerging mobile edge computing (MEC) architecture offers cloud services by deploying MEC servers at the mobile base stations (BSs). The IoT devices can transmit the offloaded data to the BS for computation at the MEC server. Narrowband-IoT (NB-IoT) is a new cellular technology for the transmission of IoT data to the BS. In this paper, we propose a joint computation offloading and multiuser scheduling algorithm in NB-IoT edge computing system that minimizes the long-term average weighted sum of delay and power consumption under stochastic traffic arrival. We formulate the dynamic optimization problem into an infinite-horizon average-reward continuous-time Markov decision process (CTMDP) model. In order to deal with the curse-of-dimensionality problem, we use the approximate dynamic programming techniques, i.e., the linear value-function approximation and temporal-difference learning with post-decision state and semi-gradient descent method, to derive a simple algorithm for the solution of the CTMDP model. The proposed algorithm is semi-distributed, where the offloading algorithm is performed locally at the IoT devices, while the scheduling algorithm is auction-based where the IoT devices submit bids to the BS to make the scheduling decision centrally. Simulation results show that the proposed algorithm provides significant performance improvement over the two baseline algorithms and the MUMTO algorithm which is designed based on the deterministic task model.

By leveraging the concept of mobile edge computing (MEC), massive amount of data generated by a large number of Internet of Things (IoT) devices could be offloaded to MEC server at the edge of wireless network for further computational intensive processing. However, due to the resource constraint of IoT devices and wireless network, both communications and computation resources need to be allocated and scheduled efficiently for better system performance. In this article, we propose a joint computation off-loading and multiuser scheduling algorithm for IoT edge computing system to minimize the long-term average weighted sum of delay and power consumption under stochastic traffic arrival. We formulate the dynamic optimization problem as an infinite-horizon average-reward continuous-time Markov decision process (CTMDP) model. One critical challenge in solving this MDP problem for the multiuser resource control is the curse-of-dimensionality problem, where the state space of the MDP model and the computation complexity increase exponentially with the growing number of users or IoT devices. In order to overcome this challenge, we use the deep reinforcement learning (RL) techniques and propose a neural network architecture to approximate the value functions for the post-decision system states. The designed algorithm to solve the CTMDP problem supports semi distributed auction-based implementation, where the IoT devices submit bids to the BS to make the resource control decisions centrally. The simulation results show that the proposed algorithm provides significant performance improvement over the baseline algorithms, and also outperforms the RL algorithms based on other neural network architectures.

Vehicular networks enable vehicles to generate and broadcast messages in order to improve traffic safety and efficiency. However, due to the nontrusted environments, it is difficult for vehicles to evaluate the credibilities of received messages. In this paper, we propose a decentralized trust management system in vehicular networks based on blockchain techniques. In this system, vehicles can validate the received messages from neighboring vehicles using Bayesian Inference Model. Based on the validation result, the vehicle will generate a rating for each message source vehicle. With the ratings uploaded from vehicles, roadside units (RSUs) calculate the trust value offsets of involved vehicles and pack these data into a “block.” Then, each RSU will try to add their “blocks” to the trust blockchain which is maintained by all the RSUs. By employing the joint proof-of-work (PoW) and proof-of-stake consensus mechanism, the more total value of offsets (stake) is in the block, the easier RSU can find the nonce for the hash function (PoW). In this way, all RSUs collaboratively maintain an updated, reliable, and consistent trust blockchain. Simulation results reveal that the proposed system is effective and feasible in collecting, calculating, and storing trust values in vehicular networks.

A Sybil attacker is able to obtain more than one identities and disguise as multiple vehicles in order to interfere the normal operations of the connected vehicle system (CVS). In this paper, we propose a novel classifier to detect Sybil attackers according to their mobility behaviors. Specifically, three levels of Sybil attackers are first defined according to their attack abilities. Through analyzing the mobility behaviors of vehicles, a learning-based model is used in the central server (CS) to extract mobility features and distinguish Sybil attackers from benign vehicles. Three classification algorithms are tested and compared, i.e., the naive Bayes, decision tree, and support vector machine. Furthermore, location certificates issued by base stations are used to resist location forgery by attackers. Based on the location certificates, the CS is able to evaluate the credibilities of uploaded locations using the subjective logic theory. In addition, we develop an edge betweenness-based community detection algorithm to handle the collusion among multiple Sybil attackers. Simulations are conducted based on a real-world vehicle trajectory dataset, which indicate that the proposed scheme is effective to resist Sybil attackers in CVS.

The IEEE 802.15.4k standard has defined the phys- ical and multiple media access (MAC) layer for low-energy critical infrastructure monitoring (LECIM) networks, which can be used to monitor infrastructure facilities including industrial metering. The main features of LECIM networks are minimal infrastructure with star topology, long range communication with high receiver sensitivity, very limited energy supplied devices. Based on IEEE 802.15.4k specifications, we have designed and developed the prototypes of end device (ED) and access point (AP) using software defined radio technology. The end device is implemented with an ARM-based MCU and a RF module, while the access point is realized by GNURadio and universal software radio peripheral (USRP). A novel parallel preamble and payload detection is applied at AP to acquire multiple packets from respective ED instead of collision avoidance. Furthermore, the field trails are conducted in urban area to demonstrate and evaluate the effectiveness of testbed design.

The communication delay of train control services has a great impact on the track utilization and speed profile of high-speed trains. This paper undertakes stochastic delay analysis of train control services over a high-speed railway fading channel using stochastic network calculus. The mobility model of high-speed railway communications system is formulated as a semi-Markov process. Accordingly, the instantaneous data rate of the wireless channel is characterized by a semi-Markov modulated process, which takes into account the channel variations due to both large- and small-scale fading effects. The stochastic service curve of high-speed railway communications system is derived based on the semi-Markov modulated process. Based on the analytical approach of stochastic network calculus, the stochastic upper delay bounds of train control services are derived with both the moment generating function method and the complementary cumulative distribution function method. The analytical results of the two methods are compared and validated by simulation.

For industrial wireless sensor networks, it is essential to reliably sense and deliver the environmental data on time to avoid system malfunction. While energy harvesting is a promising technique to extend the lifetime of sensor nodes, it also brings new challenges for system reliability due to the stochastic nature of the harvested energy. In this paper, we investigate the optimal energy management policy to minimize the weighted packet loss rate under the delay constraint, where the packet loss rate considers the lost packets, both during the sensing and delivering processes. We show that the above-mentioned energy management problem can be modeled as an infinite horizon average reward constraint Markov decision problem. In order to address the well-known curse of dimensionality problem and facilitate distributed implementation, we use the linear value approximation technique. Moreover, we apply stochastic online learning with a post-decision state to deal with the lack of the knowledge of the underlying stochastic processes. A distributed energy allocation algorithm with a water-filling structure and a scheduling algorithm by an auction mechanism are obtained. Experimental results show that the proposed algorithm achieves nearly the same performance as the optimal offline value iteration algorithm while requiring much less computation complexity and signaling overhead, and outperforms various existing baseline algorithms.

In this paper, the resource allocation problem is investigated for Device-to-Device (D2D) communications underlaying cellular networks with bursty traffic arrival. We formulate an infinite horizon average reward constraint Markov decision process (CMDP) that aims at minimizing the average delay under the dropping propability constraint. Then, we present a reduced-state Bellman's equation with linear value function approximation to deal with the curse of dimensionality problem in solving the CMDP. A distributed resource allocation algorithm is derived with low computation complexity and signaling overhead, which consists of a subchannel bidding mechanism to obtain the optimal control action, and a distributed online stochastic learning algorithm to estimate the value function and the optimal Lagrangian Multipliers (LMs). Simulation results show that the performance of our proposed algorithm is very close to that achieved by the offline value iteration algorithm, and is better than various baselines algorithms.

Millimeter wave (mmWave) communication is a promising solution for future fifth generation (5G) cellular networks to offer extremely high capacity. Because of the propagation characteristics of mmWave band, 5G users with high mobility in metropolitan areas could suffer frequent link outages resulting in challenges on video quality provisioning. In this paper, a playout buffer is used to regulate and maintain the video playout quality. We formulate the problem of using dynamically allocated bandwidth to charge the buffer as a Markov decision process (MDP), aiming to maximize video playout quality for all the users moving in the whole coverage area. Dynamic programming is adopted to solve the MDP problem with state aggregation considering the characteristics of mmWave network. Numerical results demonstrate that the resultant optimal policy of the MDP model can effectively maintain video playout quality for high-mobility users with intermittent mmWave connection.

The performance analysis of high-speed railway (HSR) network has attracted broad attention in recent years. The characteristics of HSR scenario, compared with traditional public network scenario, mainly lies in the special traffic between trains and stations, i.e., train control traffic, video monitoring traffic and passenger traffic in general. Since the performance analysis of traffic in HSR, especially train control traffic and video monitoring traffic, can better guarantee the safety and improvement of railway, thus this paper researches and analyzes the performance bounds of HSR railway uplink traffic. The work is carried out based on a group of actual measurement data gathered from Chinese Train Control System (CTCS) and an advanced performance analysis tool stochastic network calculus (SNC). Finally, by means of the measurement data in the uplink direction, the stochastic backlog and delay bounds of train control traffic and video monitoring traffic under different violation probabilities and bit rates are derived and compared.

In this study, the authors form a wireless channel model for orthogonal frequency‐division multiplexing (OFDM) systems with stochastic high‐level Petri net (SHLPN) formalism in order to simplify the cross‐layer performance analysis of modern wireless systems. Compared with existing finite state Markov channel model whose state space grows exponentially with the number of OFDM subchannels, the author's proposed SHLPN model uses state aggregation technique to deal with this problem. Closed‐form expressions to calculate the transition probabilities among the compound markings of the SHLPN model are provided. When applied to derive the performance measures for OFDM system in terms of the average throughput, average delay and packet dropping probability, the SHLPN model can accurately capture the correlated time‐varying nature of wireless channels. Simulation is performed to show that the numerical results offered by the proposed model are more accurate compared with other simplified channel models for avoiding state space complexity.

In this paper, we study the performance of mode selections in device-to-device (D2D) communications in terms of end-to-end average throughput, average delay, and dropping probability, considering dynamic data arrival with non-saturated buffers. We first introduce a general framework that includes three canonical routing modes, namely D2D mode, cellular mode, and hybrid mode, which can be combined with different resource allocation restrictions to represent the semi-static and dynamic selections of the three resource sharing modes. A queuing model is developed when the routing mode for every D2D connection is chosen, and an exact numerical analysis and an approximate decomposition and iteration approach are proposed. The performance measures are obtained from the decomposition approach and validated by means of simulation. We further introduce a mode selection scheme that adaptively chooses to semi-statically or dynamically select the resource sharing modes according to the estimated performance measures.

This paper studies the queuing performance of direct communications between user equipments in cellular networks. Both the fast fading effects of the wireless channel and the dynamic variation of interference from a link associated with its backlogged state are considered in the service process description, which complicates the queuing model and results in the size of the underlying Markov process growing exponentially with the link number. We use the model decomposition and iteration approach in Stochastic Petri Nets (SPN) to deal with the coupling between the service rates of the different links and thus reduce the state space of the Markov process. Simulations are performed to verify the accuracy of the analytical results.

In this paper, the energy efficiency performance of device-to-device (D2D) communications in orthogonal frequency division multiplexing cellular networks is studied. Different from previous work that was based on the static interference model, we assume a dynamic number of competing flows, such as continuous transfers of file transport protocol or web browsing sessions. The complex, dynamic interaction of the amount of backlogged traffic at the D2D and cellular links introduced by the strong impact of interference between them is captured by the coupled-processors server in the formulated model. The energy efficiency and spectral efficiency of the different resource-sharing strategies are analyzed using the semidefinite optimization approach. It is shown that the simulation results match well with the analytical bounds under different traffic loads and topologies.

In this paper, we study the performance of Device-to-Device (D2D) communications with dynamic interference. In specific, we analyze the performance of frequency reuse among D2D links with dynamic data arrival setting. We first consider the arrival and departure processes of packets in a non-saturated buffer, which result in varying interference on a link based on the change of its backlogged state. The packet-level system behavior is then represented by a coupled processor queuing model, where the service rate varies with time due to both the fast fading and the dynamic interference effects. In order to analyze the queuing model, we formulate it as a Discrete Time Markov Chain (DTMC) and compute its steady-state distribution. Since the state space of the DTMC grows exponentially with the number of D2D links, we use the model decomposition and some iteration techniques in Stochastic Petri Nets (SPNs) to derive its approximate steady state solution, which is used to obtain the approximate performance metrics of the D2D communications in terms of average queue length, mean throughput, average packet delay and packet dropping probability of each link. Simulations are performed to verify the analytical results under different traffic loads and interference conditions.

Wireless networks are expected to support a diverse range of quality of service requirements and traffic characteristics. This paper undertakes stochastic performance analysis of a wireless finite-state Markov channel (FSMC) by using stochastic network calculus. Particularly, delay and backlog upper bounds are derived directly based on the analytical principle behind stochastic network calculus. Both the single user and multi-user cases are considered. For the multi-user case, two channel sharing methods among eligible users are studied, i.e., the even sharing and exclusive use methods. In the former, the channel service rate is evenly divided among eligible users, whereas in the latter, it is exclusively used by a user randomly selected from the eligible users. When studying the exclusive use method, the problem that the state space increases exponentially with the user number is addressed using a novel approach. The essential idea of this approach is to construct a new Markov modulation process from the channel state process. In the new process, the multi-user effect is equivalently manifested by its transition and steady-state probabilities, and the state space size remains unchanged even with the increase of the user number. This significantly reduces the complexity in computing the derived backlog and delay bounds. The presented analysis is validated through comparison between analytical and simulation results.

Due to the dense and self-deployment of home eNodeBs (HeNBs) in femtocells, serious inter-cell interference may arise without good coordination. To deal with it, a distributed interference coordination scheme for femtocells networks with carrier aggregation (CA) is proposed in this paper. Firstly, each femtocell operates on all the component carriers (CCs) of the network and gathers the information needed via local measurements. Then, based on the measurements, decisions of maintaining or releasing the component carrier are individually made by each femtocell on each component carrier. Simulation results validate the effectiveness of our proposed scheme in term of the signal to interference and noise ratio (SINR) and throughput.

The theoretical analysis on quality-of-service (QoS) performances is required to provide the guides for the developments of the next-generation wireless networks. As a good analysis tool, the probabilistic network calculus with moment generating functions (MGFs) recently can be used for delay and backlog performance measures in wireless networks. Different from the existed studies which mostly focused on the single-hop networks with single-user under a two state Markov channel model, this study develops an analytical framework for wireless multi-hop relaying networks under the finite-state Markov channel by using probabilistic network calculus with MGFs. By using the concatenation character of network calculus, the authors regard a two-hop wireless relaying channel as a single server equivalently, which consisting of two dynamic servers in series. When the single-user model is straightforwardly extended and applied in multi-user scenarios, the state space of service process is increased exponentially with the number of users, which is only applicable in case of very small user number. Then, in order to avoid the limitation of user number, the authors propose to reflect the multi-user effects by using the equivalent data rate of the modified service process, whose transition and stationary probabilities are kept unchanged with those in single-user scenarios. Next, delay and backlog bounds of multi-hop wireless relaying networks are derived with the proposed analytical framework. Simulation results show that analytical bounds match simulation results, whose accuracy depends on the required violation probability. The effectiveness of the relaying techniques in improving the performances is also demonstrated.

Multihop relaying is one of the promising techniques in future generation wireless networks. The adaptive modulation and coding (AMC) mechanisms can be applied in order to increase the spectral efficiency of wireless multihop networks. However, most of these mechanisms concentrate on the physical layer without taking the queuing effects at the data link layer into account, whose performances are overestimated. Therefore the cross-layer analytical framework is presented in analysing the quality-of-service (QoS) performances of the decode-and-forward (DF) relaying wireless networks, where the AMC is employed at the physical layer under the conditions of unsaturated traffic and finite-length queue at the data link layer. Considering the characteristics of DF relaying protocol at the physical layer, the authors first propose modelling a two-hop DF relaying wireless channel with AMC as an equivalent Finite State Markov Chain (FSMC) in queuing analysis. Then, the performances in terms of queuing delay, packet loss rate and average throughput are derived. The numerical results show that the proposed analytical method can be efficiently applied for studying the issues including the relay deployment and the cross-layer design in the multihop relaying networks.

In this paper, performance of wireless opportunistic schedulers in multiuser systems is studied under a dynamic data arrival setting. Different from the previous studies which mostly focus on the network stability and the worst case scenarios, we emphasize on the average performance of wireless opportunistic schedulers. We first develop a framework based on Markov queueing model and then analyze it by applying decomposition and iteration techniques in the stochastic Petri nets (SPN). Since the size of the state space in our analytical model is small, the proposed framework shows an improved efficiency in computational complexity. Based on the established analytical model, performance of both opportunistic and non-opportunistic schedulers are studied and compared in terms of average queue length, mean throughput, average delay and dropping probability. Analytical results demonstrate that the multiuser diversity effect as observed in the infinite backlog scenario is only valid in the heavy traffic regime. The performance of the opportunistic schedulers in the light traffic regime is worse than that of the non-opportunistic round-robin scheduler, and becomes worse especially with the increase of the number of users. Simulations are also performed to verify the accuracy of the analytical results.

This paper studies the flow-level performance of a special family of weight-based opportunistic scheduler using discriminatory processor sharing (DPS) model. It is known that this family of schedulers can achieve any feasible long- term throughput vectors by the variation of its weights. The optimal weight setting problem under DPS model is studied by decomposing it into two subproblems. In each subproblem, a guideline as to how the weight should be chosen is derived. The usage of these two guidelines in analyzing the original problem is discussed via theoretical and simulation results.

In this paper, the performance of opportunistic scheduling in orthogonal frequency division multiplexing (OFDM) networks is studied. An analytical model is developed to extend the multi-class processor-sharing model in single-carrier networks to multi-carrier OFDM networks, where the total service rate depends on the total number of users. Based on the analytical model, the scheduling gains in both OFDM-TDMA (time division multiple access) and OFDMA (orthogonal frequency division multiple access) networks are evaluated. Different from previous works in this area, we focus on the scheduling performance at the flow level and consider a dynamic network setting with random finite-size service demands. Simulations are performed to verify the analytical results.

In this paper, the flow-level performance of opportunistic scheduling in orthogonal frequency division multiplexing (OFDM) networks is studied. The analysis accounts for the applications with a dynamic number of competing flows, such as continuous transfers of file transport protocol (FTP) or web browsing sessions. An analytical model is developed to extend the multi-class processor-sharing model in single-carrier networks to multi-carrier OFDM networks, where the total service rate varies with the number of flows. Based on the analytical model, the scheduling gains in both OFDM-TDMA (time division multiple access) and OFDMA (orthogonal frequency division multiple access) networks are evaluated for low and moderate signal-to-noise ratio (SNR). Different from previous works, we focus on the scheduling performance at the flow level and consider a dynamic network setting with random sized service demands. Furthermore, we use stochastic comparison techniques to examine the effects of physical-layer characteristics, such as fading speed and channel frequency selectivity, on flow-level performance. Simulations are performed to verify the analytical results.

In hierarchical networks, the topology and QoS parameters of a domain have to be first aggregated before being propagated to other domains. However, topology aggregation may distort useful information. This paper focuses on minimizing the distortion caused by reducing a full-mesh representation to a spanning tree. An optimization method of minimizing the distortion of additive parameters caused by spanning tree aggregation is presented. Based on this new method, two approximation algorithms are proposed. Simulation results show that both algorithms perform much better than the traditional way of decoding the spanning tree with upper or lower bounds.

This paper proposes a novel detection algorithm for multiple-input multiple-out (MIMO) code division multiplex-orthogonal frequency division multiplex access (CDM-OFDMA) system in the uplink. Vertical Bell Laboratories Layered Space-Time (V-BLAST) detection is processed only after despreading/combining at the receiver, which not only achieves good frequency diversity gain but also has low implementation complexity. Parallel interference cancellation (PIC) algorithm can also be applied to improve the performance of systems with heavy loads. Computer simulation demonstrates effectiveness of this detector and conclusion is followed.

OFDM-CDMA systems have been regarded as the most promising candidates for future mobile communication systems. This paper explores a novel OFDM-CDMA system with joint time-frequency spreading method proposed. The average bit error probability of this system using maximum-ratio combining (MRC) is derived in a frequency-selective fading channel. Numerical analysis and simulation results indicate in detail that the proposed system outperforms the conventional MC-CDMA system.