To support the intelligent evolution of Internet of Things (IoT) systems toward enhanced comprehensiveness and sophistication, this paper proposes a distributed training and inference oriented toward continual learning. By integrating federated continual learning with inference offloading, the system addresses key challenges in IoT scenarios, including large-scale data processing, catastrophic forgetting during incremental updates, and resource constraints. A joint optimization framework of training-inference is established to analyze model accuracy, latency, and energy consumption. The optimization objective and strategy are formulated as a Markov Decision Process (MDP) with a hybrid action space and customized reward mechanism. The Soft Actor-Critic (SAC) algorithm enables action grouping and the transformation between discrete and continuous actions, achieving unified optimization of training and inference. Simulation results show that compared to existing approaches, the proposed method improves node selection, resource allocation, and offloading strategy by jointly considering communication and computation costs.

The management of computing resources through the computing power network (CPN) has gradually become a focal point of research. With the development of the 6th generation (6G) mobile networks, some promising technologies such as satellite-terrestrial integrated network (STIN) and smart endogenous network driven by artificial intelligence (AI) are increasingly being applied in Industrial Internet of Things (IIoT). However, several issues in current studies are worthy of attention: 1) the large number of devices powered by battery in IIoT, 2) the complex environments of communication, 3) the finite computing resources for task data processing. To cope with these challenges, a satellite-terrestrial integrated computing power network (STICPN) framework is introduced in this article. Within this framework, a task offloading link selection scheme is proposed, which minimizes the delay and the consumption of energy. The task offloading optimization problem is modeled as a Markov decision process (MDP). Meanwhile, deep reinforcement learning (DRL) algorithm is employed to adapt to the dynamic states of environment. Specifically, a dueling double deep Q network (D3QN) is used to make optimal decisions and delay as well as energy consumption can be reduced significantly. Moreover, the D3QN-based scheme extends the usage time of IIoT devices. The simulation results indicate that the proposed scheme outperforms comparison schemes significantly.

Driven by numerous emerging mobile devices and various Quality-of-Service (QoS) requirements, mobile-edge computing (MEC) has been recognized as a prospective paradigm to promote the computation capability of mobile devices, as well as reduce energy overhead and service latency of applications for the Internet of Things (IoT). However, there are still some open issues in the existing research works: 1) limited network and computing resource; 2) simple or nonintelligent resource management; and 3) ignored security and reliability. In order to cope with these issues, in this article, 6G and blockchain technology are considered to improve network performance and ensure the authenticity of data sharing for the MEC-enabled IoT. Meanwhile, a novel intelligent optimization method named as collective reinforcement learning (CRL) is proposed and introduced, to realize intelligent resource allocation, meet distributed training results sharing, and avoid excessive consumption of system resources. Based on the designed network model, a cloud–edge collaborative resource allocation framework is formulated. By joint optimizing the offloading decision, block interval, and transmission power, it aims to minimize the consumption overheads of system energy and latency. Then, the formulated problem is designed as a Markov decision process, and the optimal strategy can be obtained by the CRL. Some evaluation results reveal that the system performance based on the proposed scheme outperforms other existing schemes obviously.

Industrial Internet of Things (IIoT) has emerged with the developments of various communication technologies. In order to guarantee the security and privacy of massive IIoT data, blockchain is widely considered as a promising technology and applied into IIoT. However, there are still several issues in the existing blockchain-enabled IIoT: 1) unbearable energy consumption for computation tasks; 2) poor efficiency of consensus mechanism in blockchain; and 3) serious computation overhead of network systems. To handle the above issues and challenges, in this article, we integrate mobile-edge computing (MEC) into blockchain-enabled IIoT systems to promote the computation capability of IIoT devices and improve the efficiency of the consensus process. Meanwhile, the weighted system cost, including the energy consumption and the computation overhead, are jointly considered. Moreover, we propose an optimization framework for blockchain-enabled IIoT systems to decrease consumption, and formulate the proposed problem as a Markov decision process (MDP). The master controller, offloading decision, block size, and computing server can be dynamically selected and adjusted to optimize the devices energy allocation and reduce the weighted system cost. Accordingly, due to the high-dynamic and large-dimensional characteristics, deep reinforcement learning (DRL) is introduced to solve the formulated problem. Simulation results demonstrate that our proposed scheme can improve system performance significantly compared to other existing schemes.

In this paper, the optimal navigation control design for biomedical untethered microrobot is comprehensively investigated in discrete-time domain with stochastic network-induced delays. First, the error dynamics of the microrobot tracking location and velocity are analyzed based on the 3D-based microrobot navigation modeling. Then, the optimal navigation optimization problem is formulated to regulate the microrobot to achieve the target reference trajectory, and a two-step control algorithm is proposed by using a backward recursion method. In particular, for each sampling interval, the optimal control gain is iteratively derived off-line and the control strategy can be calculated on-line in a real-time fashion.

As mobile data rapidly grows, power efficiency problem becomes an increasing concern in wireless networks. To efficiently reduce energy consumption, nowadays researchers attempt to introduce the thought of "edge cache" into Internet. However, the power efficiency problem in the existing solutions is mainly researched under the background of the access networks and lack of in-depth analysis from the perspective of the whole network. Therefore, we design a new power minimization mechanism for content distribution applications by deploying edge caches in wireless network scenarios. Then, we theoretically analyze the optimal power efficiency problem to realize efficient content distribution by simultaneously taking into account the effects of edge cache size, popularity distribution of network contents, network topology, and the number of different contents. Simulation process indicates that the designed model can significantly reduce power consumption in comparison to traditional Internet solutions without deploying edge caches at the edge of wireless networks.

In data center networks based on SDN, mice-flows are latency-sensitive and packet loss sensitive. Meanwhile, they account for the majority of traffic in the network, most flow rules are installed to direct the forwarding of mice-flows. According to the characteristics mentioned above, this paper proposes a joint balancing flow table and reducing delay (BFTRTD) scheme for mice-flows in data center networks to efficiently utilize limited flow tables and minimize the delay for mice-flows. In this scheme, a novel evaluation index for table balance is proposed to balance flow tables, combining with the delay of the path to initialize routes. In addition, this paper also adopts the uptodate flow rules installation mechanism to further guarantee the transmission quality and delay of mice-flows. We evaluated the proposed BFTRTD in terms of average packet loss rate and average delay of mice-flows. Simulation results show that, compared with ECMP and DIFF- Mice, the proposed BFTRTD scheme reduces the average packet loss rate by an average of 4.5% and 5.9%, while decreases the average delay by an average of 4.1% and 4.7%, when the flow arrival rate is between 120 Mbit/min and 280 Mbit/min where network load goes from low to high.

Wireless sensor and actuator networks (WSANs), as the promising technologies to realize efficient and energy-saving control, recently have been one of the main research focuses in academic fields as well as in control applications. In this paper, considering the network-induced delays, the joint design of optimal control strategy and power consumption for WSANs is addressed. First, the WSAN system with multiple transmission paths is modeled as a linear system and the joint optimization problem is formulated by using a quadratic cost function. Then, a two-step control scheme is presented to realize the joint design of control law and power consumption. In particular, the optimal control strategy for each given path is iteratively derived, and then the optimal transmission path is selected with the minimum system cost. Finally, numerical simulations in both generic control systems and power grid systems demonstrate the effectiveness of the proposed control scheme.

In most cases, the batteries of sensor nodes in the Internet of Things (IoT) are usually constrained by size and weight, and are difficult to recharge or replace. In traditional wireless sensor networks, data is transmitted in a multi-hop manner, which may cause the high data transmission delay and unbalanced traffic load. In this paper, an Unmanned Aerial Vehicle (UAV)-assisted IoT architecture is introduced, in which UAV is utilized to achieve low-latency and seamless-coverage acquisition of the sensing data. Furthermore, based on the recent advances on deep reinforcement learning algorithms, considering both data delay requirements and network energy consumption, a real-time flight path planning scheme of the UAV in the dynamic IoT sensor networks has been proposed based on dueling deep Q-network (DQN). Besides, the grid-based method is used to handle the network state modeling, which effectively reduces the complexity of the proposed scheme. Simulation results show that the proposed scheme significantly improves the network performance.

As an important part of the Internet-of-Things (IoT), machine-to-machine (M2M) communications have attracted great attention. In this paper, we introduce mobile edge computing (MEC) into virtualized cellular networks with M2M communications, to decrease the energy consumption and optimize the computing resource allocation as well as improve computing capability. Moreover, based on different functions and quality of service (QoS) requirements, the physical network can be virtualized into several virtual networks, and then each MTCD selects the corresponding virtual network to access. Meanwhile, the random access process of MTCDs is formulated as a partially observable Markov decision process (POMDP) to minimize the system cost, which consists of both the energy consumption and execution time of computing tasks. Furthermore, to facilitate the network architecture integration, software-defined networking (SDN) is introduced to deal with the diverse protocols and standards in the networks. Extensive simulation results with different system parameters reveal that the proposed scheme could significantly improve the system performance compared to the existing schemes.

Machine-to-machine (M2M) communications have attracted great attention from both academia and industry. In this paper, with recent advances in wireless network virtualization and software- defined networking (SDN), we propose a novel framework for M2M communications in software- defined cellular networks with wireless network virtualization. In the proposed framework, according to different functions and quality of service (QoS) requirements of machine-type communication devices (MTCDs), a hypervisor enables the virtualization of the physical M2M network, which is abstracted and sliced into multiple virtual M2M networks. Moreover, we formulate a decision-theoretic approach to optimize the random access process of M2M communications. In addition, we develop a feedback and control loop to dynamically adjust the number of resource blocks (RBs) that are used in the random access phase in a virtual M2M network by the SDN controller. Extensive simulation results with different system parameters are presented to show the performance of the proposed scheme.

This paper is concerned with the challenging problem of joint channel estimation and data detection for high mobility multiple-input multiple-output orthogonal frequency division multiplexing systems. We propose a new iterative channel estimation and detection scheme, which reduces the unexpected effects of both detection errors and channel estimation errors. Detection errors in channel estimation are analyzed and transformed as part of the noise. To filter this equivalent noise by Kalman estimator, we derive the covariance of both the channels and data errors in detection. Besides, we propose a new detection algorithm with an optimized weight to minimize the detection error caused by channel estimation errors. To obtain this optimized weight, the error covariance of the estimated channels is derived from the error estimate covariance matrix in Kalman estimator. Simulation results are presented to demonstrate the significant performance improvement in joint channel estimation and data detection with the proposed iterative scheme.

In cognitive radio (CR) networks, there is a common assumption that the secondary devices always obey the spectrum access rules and are under full control. However, this may become unrealistic for future CR networks composed of intelligent, complicated and autonomous devices. To solve this problem, the concept of “proactive-optimization” cognitive radio (POCR) is proposed in this paper, in which the highly-intelligent secondary users proactively optimize their own behavior decisions according to the available information including device state and network condition to maximize their long-term reward. Furthermore, we propose an optimal transmission behavior decision scheme for secondary users in POCR networks considering imperfect spectrum channel sensing results. Specifically, we formulate the system as a partially-observable Markov decision process (POMDP) problem. With this formulation, a low complexity dynamic programming framework is introduced to obtain the optimal behavior policy. Extensive simulation results are presented to illustrate the significant performance improvement of the proposed scheme compared with the existing one that ignores the secondary user behavior optimization.

High peak to average power ratio (PAPR) of the transmitted signal is a major drawback of multicarrier transmission such as orthogonal frequency division multiplexing (OFDM). This work considers the problem of PAPR reduction in a multi-symbol encapsulated OFDM (MSE-OFDM) system. This paper employs the discrete cosine transform (DCT) to the data sequence. Not only can this method have the same bandwidth efficiency as that of the CP-reduced MSE-OFDM system, but also reduce the PAPR in the MSE-OFDM. Simulation shows that the proposed scheme can significantly reduce the PAPR in MSE OFDM system without increasing the symbol error rate.