In this paper, we investigate the problem of remote estimation of a discrete-time joint Markov process using multiple sensors. Each sensor observes a different component of the joint Markov process, and in each time slot, the monitor obtains a partial state value by sending a pull request to one of the sensors. The monitor chooses the sequence of sensors to observe with the goal of minimizing the mean of age of incorrect information (MAoII) by using the partial state observations obtained, which have different freshness levels. For instance, a monitor may be interested in tracking the location of an object by obtaining observations from two sensors, which observe the$x$and$y$coordinates of the object separately, in different time slots. The monitor, then, needs to decide which coordinate to observe in the next time slot given the history. In addition to this partial observability of the state of Markov process, there is an erasure channel with a fixed one-slot delay between each sensor and the monitor. First, we obtain a sufficient statistic, namely the belief, representing the joint distribution of the age of incorrect information (AoII) and the current state of the observed process by using the history of all pull requests and observations. Then, we formulate the problem with a continuous state-space Markov decision problem (MDP), namely belief MDP. To solve the problem, we propose two model predictive control (MPC) methods, namely MPC without terminal costs (MPC-WTC) and reinforcement learning MPC (RL-MPC), that have different advantages in implementation.

We develop a novel source coding strategy for sampling and monitoring of a Wiener process. For the encoding process, we employ a four level “quantization” scheme, which employs monotone function thresholds as opposed to fixed constant thresholds. Leveraging the hitting times of the Wiener process with these thresholds, we devise a sampling and encoding strategy which does not incur any quantization errors. We give analytical expressions for the mean squared error (MSE) and find the optimal source code lengths to minimize the MSE under this monotone function threshold scheme, subject to a sampling rate constraint.

We study push-based sampling and transmission policies for a status update system consisting of a general finite-state continuous-time Markov chain (CTMC) information source with known dynamics, with the goal of minimizing the average age of incorrect information (AoII) defined via a linear time penalty function. The problem setting we investigate involves an exponentially distributed delay channel for transmissions and a constraint on the average sampling rate. We first show that the optimum sampling and transmission policy is amulti-thresholdpolicy, where the thresholds depend on both the estimation value and the state of the original process, and sampling and transmission need to be initiated when the instantaneous AoII exceeds the corresponding threshold, called the estimation-and state-aware transmission (ESAT) policy. Subsequently, we formulate the problem of finding the thresholds as a constrained semi-Markov decision process (CSMDP) and the Lagrangian approach. Additionally, we propose two lower complexity sub-optimum policies, namely the estimation-aware transmission (EAT) policy, and the single-threshold (ST) policy, for which it is possible to obtain these thresholds for CTMCs with relatively larger number of states. The underlying CSMDP formulation relies on themulti-regime phase-type(MR-PH) distribution which is a generalization of the well-known phase-type distribution, which allows us to obtain the first two moments of time until absorption in a CTMC whose transition rates change with respect to time, in a piece-wise manner. The effectiveness of the proposed ESAT, EAT, and ST sampling and transmission policies are shown through numerical examples, along with comparisons with a baseline scheme that transmits packets according to a Poisson process in out-of-sync periods.

We consider a sensor that samples an$N-\mathbf{state}$continuous-time Markov chain (CTMC)-based information source process, and transmits the observed state of the source, to a remote monitor tasked with timely tracking of the source process. The mismatch between the source and monitor processes is quantified by age of incorrect information (AoII), which penalizes the mismatch as it stays longer, and our objective is to minimize the average AoII under an average sampling rate constraint. We assume a perfect reverse channel and hence the sensor has information of the estimate while initiating a transmission or preempting an ongoing transmission. First, by modeling the problem as an average cost constrained semi-Markov decision process (CSMDP), we show that the structure of the problem gives rise to an optimum threshold policy for which the sensor initiates a transmission once the AoII exceeds a threshold depending on the instantaneous values of both the source and monitor processes. However, due to the high complexity of obtaining the optimum policy in this general setting, we consider a relaxed problem where the thresholds are allowed to be dependent only on the estimate. We show that this relaxed problem can be solved with a novel CSMDP formulation based on the theory of absorbing MCs, with a computational complexity of$\mathcal{O}(N^{4})$, allowing one to obtain optimum policies for general CTMCs with over a hundred states.