This paper addresses the challenges of unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) in high-density user mobility scenarios, a field that has not been extensively explored in current research. We introduce a novel deep reinforcement learning (DRL) framework, named “j-PPO+EN-ConvNTM”, specifically designed to optimize MEC performance in urban environments with user mobility. The framework integrates spatiotemporal data modeling and spatial transformer network (STN) through an enhance Convolution Neural Turing Machine (EN-ConvNTM) module, which includes a three-dimensional external memory. It also features a joint continuous and discrete action decision-making module, termed joint proximal policy optimization (j-PPO). This design enables effective handling of the dynamic and complex nature of urban mobility patterns. The proposed approach extends the PPO technique to accommodate joint continuous and discrete action decisions, thereby enhancing UAV adaptability and efficiency in providing MEC services. Extensive simulations demonstrate significant improvements over all baseline models, particularly in terms of equilibrium efficiency and service continuity in high-density scenarios. Our research addresses a critical gap in existing UAV-assisted MEC studies, which primarily focus on static or low-mobility user scenarios, and supports the development of more robust and efficient smart city applications, meeting the real-world demands of modern urban infrastructures.

The management of computing resources through the computing power network (CPN) has gradually become a focal point of research. With the development of the 6th generation (6G) mobile networks, some promising technologies such as satellite-terrestrial integrated network (STIN) and smart endogenous network driven by artificial intelligence (AI) are increasingly being applied in Industrial Internet of Things (IIoT). However, several issues in current studies are worthy of attention: 1) the large number of devices powered by battery in IIoT, 2) the complex environments of communication, 3) the finite computing resources for task data processing. To cope with these challenges, a satellite-terrestrial integrated computing power network (STICPN) framework is introduced in this article. Within this framework, a task offloading link selection scheme is proposed, which minimizes the delay and the consumption of energy. The task offloading optimization problem is modeled as a Markov decision process (MDP). Meanwhile, deep reinforcement learning (DRL) algorithm is employed to adapt to the dynamic states of environment. Specifically, a dueling double deep Q network (D3QN) is used to make optimal decisions and delay as well as energy consumption can be reduced significantly. Moreover, the D3QN-based scheme extends the usage time of IIoT devices. The simulation results indicate that the proposed scheme outperforms comparison schemes significantly.

The management of computing resources through the computing power network (CPN) has gradually become a focal point of research. With the development of the 6th generation (6G) mobile networks, some promising technologies, such as satellite-terrestrial integrated network (STIN) and smart endogenous network driven by artificial intelligence (AI) are increasingly being applied in Industrial Internet of Things (IIoT). However, several issues in current studies are worthy of attention: 1) the large number of devices powered by battery in IIoT; 2) the complex communication environments; and 3) the finite computing resources for task data processing. To cope with these challenges, a satellite-terrestrial integrated CPN (STICPN) framework is introduced in this article. Within this framework, a task offloading link selection scheme is proposed, which minimizes the delay and the consumption of energy. The task offloading optimization problem is modeled as a markov decision process (MDP). Meanwhile, deep reinforcement learning (DRL) algorithm is employed to adapt to the dynamic states of environment. Specifically, a Dueling Double Deep Q Network (D3QN) is used to make optimal decisions and delay as well as energy consumption can be reduced significantly. Moreover, the D3QN-based scheme extends the usage time of IIoT devices. The simulation results indicate that the proposed scheme outperforms the comparison schemes significantly.

A new concept of center processing unit (CPU)-integrated simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) is proposed, namely computing STAR-RIS. Computation-intensive and delay-sensitive tasks from mobile users can be partially processed at the computing STAR-RIS. We aim to minimize the energy consumption of users and the computing STAR-RIS, and formulate a joint task offloading and transmission resource allocation problem. The solution of this problem is affected by the offloading decision and the amplitude and phase-shift of the computing STAR-RIS. To solve the non-convex problem, we decompose it into two subproblems: 1) For the task offloading subproblem, the offloading decision is optimized utilizing the Karush-Kuhn- Tucker (KKT) conditions; and 2) For the transmission resource allocation subproblem, the transmission-reflection coefficient matrix are optimized via successive convex approximation (SCA). Simulation results show that our proposed algorithm can converge faster and have lower energy consumption than the conventional STAR-RIS.

With the advancement of mobile communication technology, there has been a marked increase in the demand for personalized and ubiquitous Internet of Things (IoT) services, raising the expectations for network Quality of Service (QoS) and Quality of Experience (QoE). Existing popularity-prediction-based content caching policies improve QoS and QoE by precaching contents at the network edge, but jointly optimizing multiple network metrics remains a challenge. To address this challenge, we propose a many-objective optimization-based popularity prediction for cooperative caching (MaOPPC-Caching) framework for cloud–edge–end collaborative IoT networks. This framework simultaneously optimizes prediction accuracy, delay, offloaded traffic, and load balance. We integrate three prediction algorithms to forecast content popularity and present a horizontal and vertical collaborative caching decision strategy to generate caching forms based on the predicted results. Then, the many-objective evolutionary algorithm (MaOEA) is employed to optimize the combined proportions to take full advantage of hidden preferences and popularity characteristics of both users and items. To promote the convergence of the framework, we present a knowledge mining-based MaOEA (KMaOEA) to incorporate knowledge mining into the optimization process. Simulation results show that the proposed MaOPPC-Caching framework outperforms existing prediction algorithms in terms of four evaluation indicators. Furthermore, KMaOEA shows a significant advantage over NSGA-III in load balance, as indicated by a Mann–Whitney rank sum test with a$p$-value of 0.040.

The development of intelligent transportation systems (ITS) has attracted significant attention to connected and autonomous vehicles (CAVs). It is urgent to investigate multi-CAV intelligent cruise control solutions in mixed traffic environments. In addition, the impact of platoon dynamics and time delays, induced by shared wireless communications, data processing, and actuation cannot be ignored. This article investigates the development of a multi-agent deep reinforcement learning (MADRL) controller tailored for CAVs operating within mixed and dynamic traffic scenarios that involve time delays. Firstly, the error dynamics in the discrete-time domain for each subplatoon is derived by considering the time-varying delays and leading vehicle states, and then the optimal CAV cruise control problem is formulated. Subsequently, the partially observable Markov game (POMG) is used to construct the multi-agent environment, and then a centralized training decentralized execution (CTDE) algorithm framework is proposed based on the multi-agent deep deterministic policy gradient (MADDPG) method. Finally, the computational complexity and the influence of delay are analyzed. The simulation results illustrate the effectiveness of the proposed intelligent algorithm.

A novel random access (RA) preamble format is proposed in this paper to support fifth generation new ratio (5G NR) enabled satellite system, which is a low earth orbiting (LEO) based non-terrestrial network (NTN). Considering a fact that traditional design of RA preamble can not meet the link budget due to a long distance between the satellite and terminal on the earth, and also will cause a wrong or failure detection of PRACH, or wrong timing estimation for uplink synchronization. The frequency offset under the large relative moving speed between the satellite and the terminal will also increase the failure detection of PRACH (physical random access channel). Therefore, a novel RA preamble format, i.e., a Zadoff-Chu (ZC) sequence with multiple lengths, are designed. To reduce the ambiguous estimation of RA preamble, a symmetric transmission of the proposed preamble is analyzed. Further, two detection algorithms (Algorithm 1 and Algorithm 2) are proposed to detect PRACH. Simulation results validate that the proposed RA preamble can meet the LEO based NTN performance requirements. According to simulation results, it can be proved that Algorithm 2 is more robust, considering timing error and frequency offset.

Driven by numerous emerging mobile devices and various quality of service requirements, mobile edge computing (MEC) has been recognized as a prospective paradigm to promote the computation capability of mobile devices, as well as reduce energy overhead and service latency of applications for the Internet of Things (IoT). However, there are still some open issues in the existing research works: 1) limited network and computing resource, 2) simple or non-intelligent resource management, 3) ignored security and reliability. In order to cope with these issues, in this article, 6G and blockchain technology are considered to improve network performance and ensure the authenticity of data sharing for the MEC-enabled IoT. Meanwhile, a novel intelligent optimization method named as collective reinforcement learning (CRL) is proposed and introduced, to realize intelligent resource allocation, meet distributed training results sharing and avoid excessive consumption of system resources. Based on the designed network model, a cloud-edge collaborative resource allocation framework is formulated. By joint optimizing the offloading decision, block interval and transmission power, it aims to minimize the consumption overheads of system energy and service latency. Then the formulated problem is designed as a Markov decision process, and the optimal strategy can be obtained by the CRL. Some evaluation results reveal that the system performance based on the proposed scheme outperforms other existing schemes obviously.

Nowadays, the rise of the Internet of Vehicles (IoV) has led to the rapid development of smart transportation. To increase the computing capacity of mobile vehicles and decrease the content delivery latency of suppliers, mobile edge computing (MEC) is considered as an indispensable solution. However, there are some essential issues to be considered: 1) security and privacy of data transmission, and 2) reasonable resource allocation for collaborative computing and caching. In this paper, to solve above issues, blockchain technology is adopted to ensure reliable transmission and interaction of data. Meanwhile, we develop an intelligent resource framework about computing and caching for blockchain-enabled MEC systems in IoV. Through jointly considering and optimizing offloading decision of computation task carried by vehicle, caching decision, the number of offloaded consensus nodes, block interval and block size, the energy consumption and computation overheads can be decreased, and the data throughput of the blockchain can be increased significantly. Moreover, the proposed optimization problem is modeled and formulated as a Markov decision process. Facing the complexity and dynamic of resource allocation, the asynchronous advantage actor-critic approach is considered and applied to solve the optimization problem. Experiment results demonstrate that the advantages of the proposed optimization scheme are obvious compared with other existing schemes.

Recently, electric vehicles (EVs) have been widely used under the call of green travel and environmental protection, and diverse requirements for charging are also increasing gradually. In order to ensure the authenticity and privacy of charging information interaction, blockchain technology is proposed and applied in charging station billing systems. However, there are some issues in blockchain itself, including lower computing efficiency of the nodes and higher energy consumption in the consensus process. To handle the above issues, in this paper, combining blockchain and mobile edge computing, we develop a reliable billing data transmission scheme to improve the computing capacity of nodes and reduce the energy consumption of the consensus process. By jointly optimizing the primary and replica nodes offloading decisions, block size and block interval, the transaction throughput of the blockchain system is maximized, as well as the consumption costs of latency and energy consumption is minimized. Moreover, we formulate the joint optimization problem as Markov decision process (MDP). To tackle this dynamic and continuity of the system state, the actor–critic reinforcement learning is introduced to solve the MDP problem. Finally, simulation results demonstrate that the performance improvement of the proposed scheme through comparison with other existing schemes.

In this paper, the optimal navigation control design for biomedical untethered microrobot is comprehensively investigated in discrete-time domain with stochastic network-induced delays. First, the error dynamics of the microrobot tracking location and velocity are analyzed based on the 3D-based microrobot navigation modeling. Then, the optimal navigation optimization problem is formulated to regulate the microrobot to achieve the target reference trajectory, and a two-step control algorithm is proposed by using a backward recursion method. In particular, for each sampling interval, the optimal control gain is iteratively derived off-line and the control strategy can be calculated on-line in a real-time fashion.

Wireless sensor actuator network (WSAN) emerges as a potential technology with the capacities of self-organizing communication and feedback control. In this paper, we present a novel collaborative optimization algorithm of plant control and system cost toward WSANs taking time delay into account. First, the WSAN model is formulated as a linear system with multipath network structure. In order to provide effective control and reduce the usage of system resource, the quadratic cost function is introduced as the collaborative optimization problem in discrete-time domain. Then, a two-phase design is proposed to derive the design of optimal control for each given path in terms of a backward recursion. In addition, the best transmission path selection is obtained depending on minimal system power consumption. Finally, numerical simulations are utilized to show the effectiveness of the proposed algorithm in both traditional control system and load frequency control in the power grid application.

With the rapid developments of wireless communication and increasing number of connected vehicles, Vehicular Ad Hoc Networks (VANETs) enable cyberinteractions in the physical transportation system. Future networks require real-time control capability to support delay-sensitive application such as connected autonomous vehicles. In recent years, fog computing becomes an emerging technology to deal with the insufficiency in traditional cloud computing. In this paper, a fog-based distributed network control design is proposed toward connected and automated vehicle application. The proposed architecture combines VANETs with the new fog paradigm to enhance the connectivity and collaboration among distributed vehicles. A case study of connected cruise control (CCC) is introduced to demonstrate the efficiency of the proposed architecture and control design. Finally, we discuss some future research directions and open issues to be addressed.

As mobile data rapidly grows, power efficiency problem becomes an increasing concern in wireless networks. To efficiently reduce energy consumption, nowadays researchers attempt to introduce the thought of "edge cache" into Internet. However, the power efficiency problem in the existing solutions is mainly researched under the background of the access networks and lack of in-depth analysis from the perspective of the whole network. Therefore, we design a new power minimization mechanism for content distribution applications by deploying edge caches in wireless network scenarios. Then, we theoretically analyze the optimal power efficiency problem to realize efficient content distribution by simultaneously taking into account the effects of edge cache size, popularity distribution of network contents, network topology, and the number of different contents. Simulation process indicates that the designed model can significantly reduce power consumption in comparison to traditional Internet solutions without deploying edge caches at the edge of wireless networks.

In data center networks based on SDN, mice-flows are latency-sensitive and packet loss sensitive. Meanwhile, they account for the majority of traffic in the network, most flow rules are installed to direct the forwarding of mice-flows. According to the characteristics mentioned above, this paper proposes a joint balancing flow table and reducing delay (BFTRTD) scheme for mice-flows in data center networks to efficiently utilize limited flow tables and minimize the delay for mice-flows. In this scheme, a novel evaluation index for table balance is proposed to balance flow tables, combining with the delay of the path to initialize routes. In addition, this paper also adopts the uptodate flow rules installation mechanism to further guarantee the transmission quality and delay of mice-flows. We evaluated the proposed BFTRTD in terms of average packet loss rate and average delay of mice-flows. Simulation results show that, compared with ECMP and DIFF- Mice, the proposed BFTRTD scheme reduces the average packet loss rate by an average of 4.5% and 5.9%, while decreases the average delay by an average of 4.1% and 4.7%, when the flow arrival rate is between 120 Mbit/min and 280 Mbit/min where network load goes from low to high.

Recent advances in Internet of Things (IoT) provide plenty of opportunities for various areas. Nevertheless, the machine-to-machine (M2M) communications-based IoT develops rapidly but suffers from extra energy consumption, large data transmission latency as well as overmuch network cost, because various of machine-type communication devices (MTCDs) are deployed in the network. To meet the requirements of energy efficient M2M communications, in this paper, we introduce a promising technology named as mobile edge computing (MEC), and propose a performance optimization framework with MEC for M2M communications network based on deep reinforcement learning (DRL). According to dynamic decision process by DRL, the appropriate access networks and the computing servers can be determined and selected with the minimum system cost, which includes lower network cost, time cost and energy consumption for data transmission and computing tasks execution. Extensive simulation results with different system parameters show that our proposed framework can effectively improve the system performance for M2M communications compared to the existing schemes.

In this paper, an optimal control algorithm for connected vehicle systems is proposed in order to ensure the vehicular platoon stable as well as reduce the transmission power consumptions in the presence of the network-induced delays. First, the vehicle dynamic modeling and power consumption analysis are addressed based on a typical 3-vehicle platoon. With the objective of minimizing the deviations of vehicle's headway and velocity as well as reducing power consumption, an optimization problem is formulated using a quadratic cost function. Then, the design of the optimal control strategy with minimum power consumption for connected vehicle systems can be divided into two steps: first the minimal hop routings for human- driven vehicles are obtained based on the network topology, and then the optimal control strategy is derived based on the determined transmission routing.

Wireless sensor and actuator networks (WSANs), as the promising technologies to realize efficient and energy-saving control, recently have been one of the main research focuses in academic fields as well as in control applications. In this paper, considering the network-induced delays, the joint design of optimal control strategy and power consumption for WSANs is addressed. First, the WSAN system with multiple transmission paths is modeled as a linear system and the joint optimization problem is formulated by using a quadratic cost function. Then, a two-step control scheme is presented to realize the joint design of control law and power consumption. In particular, the optimal control strategy for each given path is iteratively derived, and then the optimal transmission path is selected with the minimum system cost. Finally, numerical simulations in both generic control systems and power grid systems demonstrate the effectiveness of the proposed control scheme.

Considering the distributed controllers, this paper studies the optimal control law for the wireless sensor and actuator network (WSAN) with stochastic network-induced delays. First, the structure of the WSAN including multiple controllers is presented, and the network stochastic properties such as the network delay and plant noise are analyzed. Then, the optimization problem to minimize the total cost in order to keep the system stability is formulated, and the optimal state estimation control strategy is derived using the Kalman filter approach and the non-cooperative game. Finally, the proposed algorithm is validated by the simulation experiments of a stable control system and the load frequency control system.

This paper investigates the effect of imperfect channel state information (CSI) on the performance of radio resource management for the uplink OFDMA system. First, we prove that the imperfect CSI introduces a constellation-point dependent and error-floor symbol error rate (SER), which correspondingly yields the nonconvex and nondifferentiable throughput function in real communication systems. Then, in the uplink OFDMA system, a bisection algorithm and BA-based algorithm are proposed for single-user and multi-user cases, respectively, for the adaptive resource allocation to minimize the total transmit power.

This paper addresses an adaptive multi-user resource allocation for a distributed multi-carrier direct sequence-code division multiple access (MC DS-CDMA) network. The packet throughput, which turns out to be nonconvex and nondifferentiable, is considered to measure the system performance. A sub-optimal non-cooperative power control game is proposed to adaptively allocate the transmit power, available subchannels and alphabet size by minimizing the transmit power with a transmit power constraint and a packet throughput requirement. Also, the paper investigates the effect of channel estimation error on the adaptive resource allocation in a distributed MC DS-CDMA network, and shows that using the Gaussian approximation for the signal-dependent noise can lead to a nontrivial discrepancy in system performance.

In this paper, we consider an adaptive multi-user resource allocation for the downlink transmission of a multi-cluster tactical multicarrier DS CDMA network. The goal is to maximize the sum packet throughput, subject to transmit power constraints. Since the objective function turns out to be noncovex and nondifferentiable, we propose a simple iterative bisection algorithm. At each iteration, a closed-form expression is derived for the transmit power, subchannel, and modulation assignment, which significantly reduces the computational complexity. We also provide an optimization algorithm for the downlink transmission under the condition of imperfect channel knowledge, and investigate the effects of both channel estimation error and partial-band jamming.

In this paper, taking the channel estimation error (CEE) and noise into consideration, an analytical expression for envelope correlation coefficient (ECC) of the maximal ratio combining (MRC) output has been obtained in correlated Rician fading environments in the high signal-to-noise ratio (SNR) region, which provides a deep insight into the impacts of CEE, noise, frequency separation, Rician factor and channel attenuation, etc. Besides, we found that the ECC can be used to predict system performance such as average channel capacity and outage probability quantitatively in the identical statistical characteristic environment.

In this paper, a multi-user multiple input multiple output (MIMO) system using semi-orthogonal space division multiplexing (semi-OSDM) and single-user QR-triangular detection with Alamouti code is studied. In this proposed semi-OSDM with STBC system, the transmit signals for each user are grouped into pairs and separately coded by using the standard Alamouti space-time code, while QR decomposition is used to make the signaling streams triangular so that the serial interference cancellation (SIC) can be utilized to maintain an accurate signal detection at the receiver. The bit error rate (BER) performance of the proposed semi-OSDM with STBC system outperforms the orthogonal space division multiplexing (OSDM) system as well as the semi-OSDM system, while exploiting a reduced complexity and stability when the correlation of the channel changes. Moreover, it largely reduces the complexity of the receivers.

In this paper, we provide some exact expressions in a close form for central moment envelope correlation coefficient (ECC) and joint moment ECC of the maximal ratio combining (MRC) output in correlated Rayleigh and Rician fading environments. Besides, we deduce an analytical formula for average capacity with minor approximation. With these expressions, a simple relationship between joint moment ECC and the average capacity is presented, which means that only joint moment ECC can be used to predict system performance such as average channel capacity quantitatively. Furthermore, it has been demonstrated by Monte Carlo simulation that our analysis are matched with the numerical results excellently.

This work investigates a design of minimizing the system outage probability adaptive modulation algorithm in the downlink of the multi-user multiple-input multiple-output (MIMO) system. Based on a beamforming scheme that maximizes the signal-to-leakage ratio (SLR), this new algorithm is proposed for constant bit rate services to maximize the number of satisfied users and minimize the transmission power. Furthermore, this proposed algorithm has advantage in the system spectral efficiency.

In this paper, with minor approximation, we provide analytical expressions in a close form for the envelope correlation coefficient (ECC) and outage probability of the maximal ratio combining (MRC) output in correlated Rayleigh and Rician fading environments. Moreover, we find a simple relationship between ECC and the outage probability, which indicates that ECC can be used to predict system performance such as outage probability quantitatively. Besides, it has been demonstrated by Monte Carlo simulation that our analysis are matched with the numerical results excellently.