In this paper, we propose a semantic-twin-enabled bifurcated control architecture for multi-unmanned aerial vehicle (UAV) tasks in remote areas. To address the communication and computing burdens caused by high-fidelity reproduction of traditional digital twin (DT) in the remote interference environment, we propose the concept of semantic twin (ST). ST is a task-oriented system, which uses semantics for transmission, computing and decision-making, enhancing communication and computing efficiency. To achieve efficient remote control, we develop a bifurcated control architecture based on ST, in which satellites and the ground control station (GCS) function as edge controllers and the remote controller, respectively. For the satellite edge control, we employ the proximal policy optimization (PPO) algorithm to train a decision-making agent that generates action commands based on semantics from UAVs. Within the ST system of the GCS, we utilize the generative adversarial imitation learning (GAIL) algorithm to train an intelligent and interactive virtual target, creating a parallel environment for agent training. On this basis, we design a ST-enabled model-based offline reinforcement learning algorithm for lifelong learning. Compared to traditional reinforcement learning (RL) algorithms, we refine the weighted sample, model ensemble, and regularization methods, ensuring the reliability of the virtual environment and the efficacy of offline model training. Finally, we validate this framework by designing a multi-UAV tracking task and verify the significant advantages of the proposed control architecture in scenario reconstruction, model training and decision performance. Simulation results show that the ST-enabled bifurcated control architecture can counter the interference environment and accurately capture the motion features of the target, significantly improving the performance of remote UAV tasks.

This paper investigates a wireless remote control problem in cyber-physical system (CPS) with multiple processes. In the system, sensors collect the state information of each process and transmit it to the controller through wireless channels. The communication constraints, including transmission delays, packet loss, and bandwidth limitation, are taken into account. To evaluate control timeliness for each process, this paper adopts the concept of age of information (AoI) in the context of closed-loop control. Meanwhile, to strike a tradeoff between transmission delay and outage, this paper introduces an innovative cross-slot scheduling policy not covered in existing literature, which can freely allocate transmission time and occupancy bandwidth. We prove that the scheduling problem in bandwidth limited remote control is an NP-hard problem, and establish the optimization problem as a Markov decision process (MDP) problem. The Deep-Double-Dueling-Q-Learning (D3QN) algorithm is employed to approximate the optimal scheduling policy for the scenario where the channel information is unknown and the system is model-free. By extensive simulations, the proposed cross-time slot scheduling policy demonstrates superior effectiveness in allocating time-frequency resources and achieving outstanding results.

This article explores the model-free remote control problem in a wireless networked cyber-physical system (CPS) composed of spatially distributed sensors, controllers, and actuators. The sensors sample the states of the controlled system to generate control instructions at the remote controller, while the actuators maintain the system's stability by executing control commands. To realize the control under a model-free system, the deep deterministic policy gradient (DDPG) algorithm is adopted in the controller to enable model-free control. Unlike the traditional DDPG algorithm, which only takes the system state as input, this article incorporates historical action information as input to extract more information and achieve precise control in the case of communication latency. Additionally, in the experience replay mechanism of the DDPG algorithm, we incorporate the reward into the prioritized experience replay (PER) approach. According to the simulation results, the proposed sampling policy improves the convergence rate by determining the sampling probability of transitions based on the joint consideration of temporal difference (TD) error and reward.

This paper presents a Hybrid Automatic Repeat Request (HARQ) based scheduling scheme for a multi-loop Wireless Networked Control System (WNCS). For each single-loop system in the multi-loop system, it includes uplink transmission and downlink transmission. By considering a practical application scenario, we formulate a mathematical model wherein the downlink transmission can be assumed ideal, and the uplink transmission updates the new system status which is used to generate control commands. Due to the resource constraints, not all single-loop systems can update their status information in the same time slot. Meanwhile, using the HARQ mechanism can ensure a higher probability of successful transmission. To achieve the stability of the system, we propose a scheduling strategy to minimize the long-term average Mean Square Error (MSE) of the plant state. And we model the optimization problem as a Markov Decision Process (MDP) problem to obtain the optimal strategy. For the case that the channel error rates change rapidly, we propose the Lyapunov optimization strategy. And through further analysis, the Lyapunov optimization strategy is a suboptimal strategy, it can achieve the performance approach to the optimal strategy.

In this paper, we consider a cyber-physical system (CPS) with multiple Internet of Things (IoT) devices. There are multiple independent linear time-invariant processes in the system, which are sampled by sensors, scheduled by controllers and controlled by actuators. In the literature of wireless control CPS, commonly assume that the system just have one controller and ignore the processing time on server. In this work we employ the edge computing, the controllers are facilitated by edge server and cloud server. The processing time of status update depends on the characteristic of different processes and servers. By taking into account such conditions, we mainly investigate how to choose the destination of status updates (i.e., edge server or cloud server) to minimize the average Mean Square Error (MSE) of the entire system. To address this issue, we formulate a Markov Decision Process (MDP) problem and obtain the optimal scheduling policy. The threshold property of the optimal scheduling policy is proved, and a suboptimal policy is proposed to overcome the curse of dimensionality. The simulation results illustrate that the selection of controller is related to the timeliness of process and show the superiority of the proposed policies.

In this article, we consider a cyber–physical system (CPS) with multiple Internet of Things (IoT) devices. There are multiple independent linear time-invariant processes in the system, which are sampled by sensors, scheduled by controllers, and controlled by actuators. In the literature of wireless control CPS, commonly assume that the system just have one controller and ignore the processing time on server. In this work we employ the edge computing, the controllers are facilitated by edge server and cloud server. The processing time of status update depends on the characteristic of different servers and processes. By taking into account such conditions, we mainly investigate how to choose the destination of status updates (i.e., edge server or cloud server) to minimize the average mean square error (MSE) of the entire system. To address this issue, we formulate a Markov decision process (MDP) problem and obtain the optimal scheduling policy. The threshold property of the optimal scheduling policy is proved, and a suboptimal policy is proposed to overcome the curse of dimensionality. Furthermore, the processing preemption mechanism is considered to schedule the status updates more flexibly, and its consistency property is proved. The simulation results illustrate that the selection of controller is related to the timeliness of process and show the superiority of the proposed policies.