We consider semantics-aware remote estimation of a discrete-state Markov source with both normal (low-priority) and alarm (high-priority) states. Erroneously announcing a normal state at the destination when the source is actually in an alarm state (i.e., missed alarm) incurs a significantly higher cost than falsely announcing an alarm state when the source is in a normal state (i.e., false alarm). Moreover, consecutive estimation errors may cause significant lasting impacts, such as maintenance costs and misoperations. Motivated by this, we introduce two new metrics, the Age of Missed Alarm (AoMA) and the Age of False Alarm (AoFA), to capture the lasting impacts incurred by different estimation errors. Notably, these two age processes evolve interdependently and distinguish between different error types. Our goal is to design a transmission policy that achieves an optimized trade-off between lasting impact and communication cost. The problem is formulated as a countably infinite-state Markov decision process (MDP) with an unbounded cost function. We show the existence of a simple switching policy with distinct thresholds for each age process and derive closed-form expressions for its performance. For symmetric and non-prioritized sources, we show that the optimal policy reduces to a threshold policy with identical thresholds. For numerical tractability, we propose a finite-state approximate MDP and prove that it converges exponentially fast to the original MDP in the truncation size. Finally, we develop an efficient search algorithm to compute the optimal switching policy and validate our theoretical findings with numerical results.

This paper studies semantics-aware remote estimation of Markov sources. We leverage two complementary information attributes: the urgency of lasting impact, which quantifies thesignificanceof consecutive estimation error at the transmitter, and the age of information (AoI), which captures thepredictabilityof outdated information at the receiver. The objective is to minimize the long-run average lasting impact subject to a transmission frequency constraint. The problem is formulated as a constrained Markov decision process (CMDP) with potentially unbounded costs. We show the existence of an optimalsimple mixture policy, which randomizes between two neighboringswitching policiesat a common regeneration state. A closed-form expression for the optimal mixture coefficient is derived. Each switching policy triggers transmission only when the error holding time exceeds a threshold that depends on both the instantaneous estimation error and the AoI.We further derive sufficient conditions under which the thresholds are independent of the instantaneous error and the AoI. Finally, we propose a structure-aware algorithm, Insec-SPI, that computes the optimal policy with reduced computation overhead. Numerical results demonstrate that incorporating both the age and semantics of information significantly improves estimation performance compared to using either attribute alone.

The Age of Information (AoI) has emerged as a critical metric for quantifying information freshness; however, its interplay with channel estimation in partially observable wireless systems remains underexplored. This work considers a transmitter-receiver pair communicating over an unreliable channel with time-varying reliability levels. The transmitter observes the instantaneous link reliability through a channel state information acquisition procedure, during which the data transmission is interrupted. This leads to a fundamental trade-off between utilizing limited network resources for either data transmission or channel state information acquisition to combat the channel aging effect. Assuming the wireless channel is modeled as a finite-state Markovian channel, we formulate an optimization problem as a partially observable Markov decision process (POMDP), obtain the optimal policy through the relative value iteration algorithm, and demonstrate the efficiency of our solution through simulations. To the best of our knowledge, this is the first work to aim for an optimal scheduling policy for data transmissions while considering the effect of channel state information aging.

We study the semantics-aware remote state estimation of a Markov chain with prioritized states. The aim is to exploit the significance of information through the history of system realizations to determine the optimal timing of transmission, thereby reducing the amount of uninformative data transmitted in the network. To this end, we introduce the significanceaware Age of Consecutive Error (AoCE) that captures three semantic attributes: the significance of estimation error, the cost of consecutive error (or lasting impact, for short), and the urgency of lasting impact. We identify the optimal transmission problem as a countably infinite state Markov decision process (MDP) with unbounded costs. We give sufficient conditions under which an optimal policy exists to have bounded average costs. We show that the optimal policy exhibits a switching structure and, under certain conditions, degenerates into a simple threshold policy. A structured policy iteration (SPI) algorithm is proposed to compute an asymptotically optimal policy with reduced computation overhead. An important takeaway is that the more semantic attributes we utilize, the fewer transmissions are needed.

This paper studies the remote estimation of multiple Markov sources over a lossy and rate-constrained channel. Unlike most existing studies that treat all source states equally, we exploit thesemantics of informationand consider that the remote actuator has different tolerances for the estimation errors. We aim to find an optimal scheduling policy that minimizes the long-termstate-dependentcosts of estimation errors under a transmission frequency constraint. The optimal scheduling problem is formulated as aconstrained Markov decision process(CMDP). We show that the optimal Lagrangian cost follows a piece-wise linear and concave (PWLC) function, and the optimal policy is, at most, a randomized mixture of two simple deterministic policies. By exploiting the structural results, we develop a newintersection searchalgorithm that finds the optimal policy using only a few iterations. We further propose a reinforcement learning (RL) algorithm to compute the optimal policy without knowinga priorithe channel and source statistics. To avoid the “curse of dimensionality” in MDPs, we propose an online low-complexitydrift-plus-penalty(DPP) algorithm. Numerical results show that continuous transmission is inefficient, and remarkably, our semantic-aware policies can attain the optimum by strategically utilizing fewer transmissions by exploiting the timing of the important information.

This paper considers the semantics-aware remote state estimation of an asymmetric Markov chain withprioritizedstates. Due to resource constraints, the sensor needs to trade off estimation quality against communication cost. The aim is to exploit thesignificanceof information through the history of system realizations to determine the optimal timing of transmission, thereby reducing the amount of uninformative data transmitted in the network. To this end, we introduce a new metric, thesignificance-aware Age of Consecutive Error(AoCE), that captures three semantic attributes: thesignificance of estimation error, thecost of consecutive error(orlasting impact, for short), and theurgency of lasting impact. Different costs and non-linear age functions are assigned to different estimation errors to account for their relative importance to system performance. We identify the optimal transmission problem as a countably infinite state Markov decision process (MDP) with unbounded costs. We first give sufficient conditions on the age functions, source pattern, and channel reliability so that an optimal policy exists to have bounded average costs. We show that the optimal policy exhibits aswitching structure. That is, the sensor triggers a transmission only when the system has been trapped in an error for a certain number of consecutive time slots. We also provide sufficient conditions under which the switching policy degenerates into a simplethreshold policy, i.e., featuring identical thresholds for all estimation errors. Furthermore, we exploit the structural results and develop astructured policy iteration(SPI) algorithm that considerably reduces computation overhead. Numerical results show that the optimal policy outperforms the classic rule-, distortion- and age-based policies. An important takeaway is thatthe more semantic attributes we utilize, the fewer transmissions are needed.

We consider the remote estimation of a discrete-state Markov source with normal and alarm states. Data significance is revealed via two semantic attributes: 1) Erroneously announcing a normal state at the destination when the source is actually in an alarm state (i.e., missed alarm error) incurs a significantly higher cost than falsely announcing an alarm state when the source is in a normal state (i.e., false alarm error). 2) Successive reception of an estimation error may cause significant lasting impact, e.g., maintenance cost and wrong operations. Motivated by this, we assign different costs to different estimation errors and introduce two new age metrics, namely the Age of Missed Alarm (AoMA) and the Age of False Alarm (AoFA), to account for the lasting impact incurred by different estimation errors. We aim to achieve an optimal trade-off between the cost of estimation error, lasting impact, and communication utilization. The problem is formulated as an infinite-state Markov decision process (MDP). We show that the optimal policy exhibits a switching structure, i.e., triggering transmissions only when the AoMA or AoFA exceeds a threshold. Numerical results underscore that our approach significantly reduces the amount of less important information transmitted in the networks.

This paper investigates goal-oriented communication for remote estimation of multiple Markov sources in resource-constrained networks. An agent decides the updating times of the sources and transmits the packet to a remote destination over an unreliable channel with delay. The destination is tasked with source reconstruction for actuation. We utilize the metric cost of actuation error (CAE) to capture the state-dependent actuation costs. We aim for a sampling policy that minimizes the long-term average CAE subject to an average resource constraint. We formulate this problem as an average-cost constrained Markov Decision Process (CMDP) and relax it into an unconstrained problem by utilizing Lyapunov drift techniques. Then, we propose a low-complexity drift-plus-penalty (DPP) policy for systems with known source/channel statistics and a Lyapunov optimization-based deep reinforcement learning (LO-DRL) policy for unknown environments. Our policies significantly reduce the number of uninformative transmissions by exploiting the timing of the important information.

Since the traffic administration at road intersections determines the capacity bottleneck of modern transportation systems, intelligent cooperative coordination for connected autonomous vehicles (CAVs) has shown to be an effective solution. In this paper, we try to formulate a Bi-Level CAVs intersection coordination framework, where coordinators from High and Low levels are tightly coupled. In the High-Level coordinator where vehicles from multiple roads are involved, we take various metrics including throughput, safety, fairness and comfort into consideration. Motivated by the time consuming space-time resource allocation framework, we try to give a low complexity solution by transforming the complicated original problem into a sequential linear programming one. Based on the “feasible tunnels” (FT) generated from the high-Level coordinator, we then propose a rapid gradient-based trajectory optimization strategy in the low-level planner, to effectively avoid collisions beyond high-level considerations, such as the unexpected pedestrian or bicycles. Simulation results and laboratory experiments show that our proposed method outperforms existing strategies. Moreover, the most impressive advantage is that the proposed strategy can plan vehicle trajectory in milliseconds, which is promising in real-world deployments. A detailed description include the coordination framework and experiment demo could be found at the supplement materials, or online at https://youtu.be/MuhjhKfNIOg.

Cooperative coordination at unsignalized road intersections, which aims to improve the driving safety and traffic throughput for connected and automated vehicles (CAVs), has attracted increasing interests in recent years. However, most existing investigations either suffer from computational complexity or cannot harness the full potential of the road infrastructure. To this end, we first present a dedicated intersection coordination framework, where the involved vehicles hand over their control authorities and follow instructions from a centralized coordinator. Then a unified cooperative trajectory planning problem will be formulated to maximize the traffic throughput while ensuring driving safety. To address the key computational challenges in the real-world deployment, we reformulate this non-convex sequential decision-making problem into a model-free Markov Decision Process (MDP) and tackle it by devising a Twin Delayed Deep Deterministic Policy Gradient (TD3)-based strategy in the deep reinforcement learning (DRL) framework. Simulation and practical experiments show that the proposed strategy could achieve near-optimal performance in sub-static coordination scenarios and significantly improve the traffic throughput in the realistic continuous traffic flow. The most remarkable advantage is that our strategy could reduce the time complexity of computation to milliseconds, and is shown scalable when the road lanes increase.

Satellite communication is regarded as a promising technique for providing connectivity in remote areas, which creates opportunities for data collection and transmission in marine Internet-of-Remote-Things (IoRT) networks. Most existing investigations in the field of satellite access control focus on communication throughput and transmission delay. However, the freshness of information and the heterogeneous satellite networks are rarely considered. To this end, we first present a satellite-based marine IoRT system, where a GEO/LEO heterogeneous network is considered to harness the full potential of existing satellite systems, and the age-of-information (AoI) is introduced to characterize the freshness of the status update information generated by IoRT devices. Then, an optimal age-oriented access control problem is formulated to maintain the freshness of information in the long term. We transform this non-convex sequential decision problem into a model-free Markov Decision Process (MDP) problem and solve it by leveraging the deep reinforcement learning (DRL) framework. Simulation results show that the proposed strategy significantly outperforms the state-of-the-art ones in terms of long-term AoI performance. Moreover, the proposed strategy could make cooperative access decisions and obtain an excellent trade-off between satellites on different layers.

Recent investigations show that, the traffic throughput and safety could be effectively improved when vehicles follow coordination instructions from a centralized coordination center at road intersections. However, due to the inevitable noise of the vehicle control, sensing, and limited channel resources, the coordination instructions need to be “updated” periodically. Aiming at this issue, we define a “Yaw Risk” based re-planning strategy, which consists of a multi-vehicle re-planning selection and a multi-channel allocation scheme, to minimize the uncertainty of the entire coordination system. Variable safety redundancies of the collision-free tunnel are adopted to guarantee the tradeoff between safety and traffic throughput. Numerical results are provided and verify our analysis.

With the growing interest in the smart ocean, the satellite-based marine Internet of Remote Things (IoRT) network has been regarded as a promising architecture for sensory data collection and transmission in infrastructure-limited offshore areas. In this article, we investigate the access control problem in the context of GEO/LEO heterogeneous IoRT networks, where multiple gateways are deployed to collect data generated by IoRT devices and then forward them to the terrestrial data center via satellite links. However, most existing access control strategies shed light on the traditional network performance (i.e., transmission delay and communication throughput) in single-layer satellite networks (i.e., low-Earth orbit (LEO) layer or geosynchronous orbit (GEO) layer), whereas the interplay between LEO and GEO layers and the freshness of information are rarely considered. To this end, we first formulate an age-oriented access control problem to minimize the long-term peak Age of Information (AoI) and transform it into a model-free Markov decision process (MDP). Then, a Deep-Double-Dueling-$Q$-Learning (D3QN) policy is trained offline and can be deployed online to make decisions according to dynamic data arrivals and time-varying channels. Simulation results show that the proposed strategy significantly outperforms the state-of-the-art ones in terms of the long-term AoI performance. Furthermore, our strategy could make cooperative decisions for gateways and obtain a proper tradeoff between satellites on different layers.