Filtering refers to the methods for inferring time-varying parameters and is a crucial task in cyber-physical systems. An important category of filtering is distributed filtering, where sensor nodes transmit observations via communication links to inference nodes that estimate the unknown states. Distributed filtering is challenging in the sense that the communication constraint of the sensor nodes limits the amount of information available to the inference node, calling for the co-design of communication and computing. This paper establishes a theoretical framework for the co-design of communication and computing in distributed filtering, building on an information-theoretic view of the Kalman–Bucy filtering. In particular, this paper considers a networked system consisting of two nodes, where each node aims to infer its own time-varying state in continuous-time scenarios. The two nodes are connected by a Gaussian feedback channel. Via the feedback link, one of the nodes can obtain the sensor observations and received signals of the other node. This paper develops an optimal linear strategy, namely the information difference encoding strategy, for generating signals transmitted via the Gaussian feedback channel. This paper also presents an inequality that relates Shannon information with Fisher information in distributed filtering. The inference accuracy and power efficiency of the information difference encoding strategy are quantified via simulations.

Distributed learning is an important task in emerging applications such as localization and navigation, Internet-of-Things, and autonomous vehicles. This paper establishes a theoretical framework for learning states that evolve in real time over networks. Specifically, each agent node in the network aims to infer a time-varying state in a decentralized manner by using the node’s local observations and the messages received from other nodes within its communication range. As a result, the inference accuracy of a node is significantly affected by the quality of its received messages. This calls for carefully designed strategies for generating messages that are able to provide sufficient information for the receiver and are robust to channel impairments. This paper presents communication-efficient encoding strategies for generating transmitted messages and derives a sufficient condition for the boundedness of the distributed inference error of all the agent nodes over time. The findings of this paper provide guidelines for the design of communication-efficient distributed learning in complex networked systems.

Distributed learning is crucial for many applications such as localization and tracking, autonomy, and crowd sensing. This paper investigates communication-efficient distributed learning of time-varying states over networks. Specifically, the paper considers a network of nodes that infer their current states in a decentralized manner using observations obtained via local sensing and messages obtained via noisy inter-node communications. The paper derives a necessary condition in terms of the sensing and communication capabilities of the network for the boundedness of the learning error over time. The necessary condition is compared with the sufficient condition established in a companion paper and the gap between the two conditions is discussed. The paper provides guidelines for efficient management of the sensing and communication resources for distributed learning in complex networked systems.

This article presents considerations toward an information and control architecture for future electric energy systems driven by massive changes resulting from the societal goals of decarbonization and electrification. This article describes the new requirements and challenges of an extended information and control architecture that needs to be addressed for continued reliable delivery of electricity. It identifies several new actionable information and control loops, along with their spatial and temporal scales of operation, which can together meet the needs of future grids and enable deep decarbonization of the electricity sector. The present architecture of electric power grids designed in a different era is thereby extensible to allow the incorporation of increased renewables and other emerging electric loads.

In modern autonomous systems, there is typically a large number of connected components realizing complex functionalities. For example, in autonomous vehicles (AVs), there are tens of millions of lines of code implemented on hundreds of sensors, controllers, and actuators. AVs have been deployed, mostly in trials and restricted environments, showing that substantial progress has been made in functionality development. However, they are still faced with two major challenges: (i) performance guarantee of safety-critical functions under all possible scenarios; (ii) functionality implementation with limited resources. These two challenges are conflicting because safety guarantees necessitate a worst-case analysis that is often very pessimistic for complex hardware/software systems, and thus require more resources. To address this, we study an abstraction of a heterogeneous cyber-physical system architecture consisting of a mix of high- and low-quality resources, such as time- and event-triggered resources, or wired and wireless resources. We show that by properly managing such a mix of resources and formulating a formal verification (model checking) problem, it is possible to tightly dimension the high-quality resource to the minimum (50% in certain cases) while providing control performance guarantees.

Software-based control of physical systems is common in domains such as automotive, avionics, and industrial automation. Safety of such systems is determined by control-theoretic properties such as stability, settling time, and peak overshoot. These properties strongly depend on the software code generated from high-level controller models, and the implementation of such code on an embedded platform. To ensure safety, the semantics of the system model considered for controller design must be faithfully preserved in the platform implementation. However, traditionally, controller design and implementation platform design are carried out in isolation, followed by their integration, which often relies on simulations to estimate the behavior of the controllers. Thus, safety properties that were proven at the model level using control-theoretic tools can no longer be established in an actual implementation. This makes the design of embedded control systems costly, error prone, and hinders certification. In this paper, we review recent efforts in control-platform cosynthesis techniques toward addressing this problem. Here, the control and the embedded systems communities have come together to adopt a cyber-physical system (CPS)-oriented design paradigm. This cosynthesis paradigm integrates the design of control algorithms and platform parameters within a holistic optimization framework and accounts for relevant details from both sides. We survey the evolution of design approaches for such cosynthesis and show how-the originally disjoint-controller and the platform design methods are gradually converging.

Given the possibility of communication systems failing catastrophically, we investigate limits to communicating over channels that fail at random times. These channels are finite-state semi-Markov channels. We show that communication with arbitrarily small probability of error is not possible. Making use of results in finite blocklength channel coding, we determine sequences of blocklengths that optimize transmission volume communicated at fixed maximum message error probabilities. We provide a partial ordering of communication channels. A dynamic programming formulation is used to show the structural result that channel state feedback does not improve performance.

Experts (human or computer) are often required to assess the probability of uncertain events. When a collection of experts independently assess events that are structurally interrelated, the resulting assessment may violate fundamental laws of probability. Such an assessment is termed incoherent. In this work we investigate how the problem of incoherence may be affected by allowing experts to specify likelihood models and then update their assessments based on the realization of a globally-observable random sequence.

The hypothesis that high dimensional data tends to lie in the vicinity of a low dimensional manifold is the basis of a collection of methodologies termed Manifold Learning. In this paper, we study statistical aspects of the question of fitting a manifold with a nearly optimal least squared error. Given upper bounds on the dimension, volume, and curvature, we show that Empirical Risk Minimization can produce a nearly optimal manifold using a number of random samples that is {\it independent} of the ambient dimension of the space in which data lie. We obtain an upper bound on the required number of samples that depends polynomially on the curvature, exponentially on the intrinsic dimension, and linearly on the intrinsic volume. For constant error, we prove a matching minimax lower bound on the sample complexity that shows that this dependence on intrinsic dimension, volume and curvature is unavoidable. Whether the known lower bound of $O(\frac{k}{\eps^2} + \frac{\log \frac{1}{\de}}{\eps^2})$ for the sample complexity of Empirical Risk minimization on $k-$means applied to data in a unit ball of arbitrary dimension is tight, has been an open question since 1997 \cite{bart2}. Here $\eps$ is the desired bound on the error and $\de$ is a bound on the probability of failure. We improve the best currently known upper bound \cite{pontil} of $O(\frac{k^2}{\eps^2} + \frac{\log \frac{1}{\de}}{\eps^2})$ to $O\left(\frac{k}{\eps^2}\left(\min\left(k, \frac{\log^4 \frac{k}{\eps}}{\eps^2}\right)\right) + \frac{\log \frac{1}{\de}}{\eps^2}\right)$. Based on these results, we devise a simple algorithm for $k-$means and another that uses a family of convex programs to fit a piecewise linear curve of a specified length to high dimensional data, where the sample complexity is independent of the ambient dimension.

This paper provides a framework for generating high resolution time sequences of 3D images that show the dynamics of cerebral blood flow. These sequences have the potential to allow image feedback during medical procedures that facilitate the detection and observation of pathological abnormalities such as stenoses, aneurysms, and blood clots. The 3D time series is constructed by fusing a single static 3D model with two time sequences of 2D projections of the same imaged region. The fusion process utilizes a variational approach that constrains the volumes to have both smoothly varying regions separated by edges and sparse regions of nonzero support. The variational problem is solved using a modified version of the Gauss-Seidel algorithm that exploits the spatio-temporal structure of the angiography problem. The 3D time series results are visualized using time series of isosurfaces, synthetic X-rays from arbitrary perspectives or poses, and 3D surfaces that show arrival times of the contrasted blood front using color coding. The derived visualizations provide physicians with a previously unavailable wealth of information that can lead to safer procedures, including quicker localization of flow altering abnormalities such as blood clots, and lower procedural X-ray exposure. Quantitative SNR and other performance analysis of the algorithm on computational phantom data are also presented.

In this paper, we introduce a general framework for treating channels with memory and feedback. First, we prove a general feedback channel coding theorem based on Massey's concept of directed information. Second, we present coding results for Markov channels. This requires determining appropriate sufficient statistics at the encoder and decoder. We give a recursive characterization of these sufficient statistics. Third, a dynamic programming framework for computing the capacity of Markov channels is presented. Fourth, it is shown that the average cost optimality equation (ACOE) can be viewed as an implicit single-letter characterization of the capacity. Fifth, scenarios with simple sufficient statistics are described. Sixth, error exponents for channels with feedback are presented.

The 13 papers in this special issue focus on control and communications. The papers are summarized here.

We study in this paper randomized constructions of binary linear codes that are invariant under the action of some group on the bits of the codewords. We study a non-Abelian randomized construction corresponding to the action of the dihedral group on a single copy of itself as well as a randomized Abelian construction based on the action of an Abelian group on a number of disjoint copies of itself. Cyclic codes have been extensively studied over the last 40 years. However, it is still an open question as to whether there exist asymptotically good binary cyclic codes. We argue that by using a slightly more complex group than a cyclic group, namely, the dihedral group, the existence of asymptotically good codes that are invariant under the action of the group on itself can be guaranteed. In particular, we show that, for infinitely many block lengths, a random ideal in the binary group algebra of the dihedral group is an asymptotically good rate-half code with a high probability. We argue also that a random code that is invariant under the action of an Abelian group G of odd order on k disjoint copies of itself satisfies the binary Gilbert-Varshamov (GV) bound with a high probability for rate 1/k under a condition on the family of groups. The underlying condition is in terms of the growth of the smallest dimension of a nontrivial F/sub 2/-representation of the group and is satisfied by roughly most Abelian groups of odd order, and specifically by almost all cyclic groups of prime order.

In this paper, we review how Shannon's classical notion of capacity is not enough to characterize a noisy communication channel if the channel is intended to be used as part of a feedback loop to stabilize an unstable scalar linear system. While classical capacity is not enough, another sense of capacity (parametrized by reliability) called "anytime capacity" is necessary for the stabilization of an unstable process. The required rate is given by the log of the unstable system gain and the required reliability comes from the sense of stability desired. A consequence of this necessity result is a sequential generalization of the Schalkwijk-Kailath scheme for communication over the additive white Gaussian noise (AWGN) channel with feedback. In cases of sufficiently rich information patterns between the encoder and decoder, adequate anytime capacity is also shown to be sufficient for there to exist a stabilizing controller. These sufficiency results are then generalized to cases with noisy observations, delayed control actions, and without any explicit feedback between the observer and the controller. Both necessary and sufficient conditions are extended to continuous time systems as well. We close with comments discussing a hierarchy of difficulty for communication problems and how these results establish where stabilization problems sit in that hierarchy

A bound on the minimum distance of a binary error-correcting code is established given constraints on the computational time-space complexity of its encoder where the encoder is modeled as a branching program. The bound obtained asserts that if the encoder uses linear time and sublinear memory in the most general sense, then the minimum distance of the code cannot grow linearly with the block length when the rate is nonvanishing, that is, the minimum relative distance of the code tends to zero in such a setting. The setting is general enough to include nonserially concatenated turbo-like codes and various generalizations. Our argument is based on branching program techniques introduced by Ajtai. The case of constant-depth AND-OR circuit encoders with unbounded fanins are also considered.

We present a novel representation of shape for closed planar contours explicitly designed to possess a linear structure. This greatly simplifies linear operations such as averaging, principal component analysis or differentiation in the space of shapes. The representation relies upon embedding the contour on a subset of the space of harmonic functions of which the original contour is the zero level set.

The problem of matching two planar sets of points in the presence of geometric uncertainty has important applications in pattern recognition, image understanding, and robotics. The first set of points corresponds to the "template." The other set corresponds to the "image" that-possibly-contains one or more deformed versions of the "template" embedded in a cluttered image. Significant progress has been made on this problem and various polynomial-time algorithms have been proposed. We show how to sample the "image" in linear time, reducing the number of foreground points n by a factor of two-six (for commonly occurring images) without degrading the quality of the matching results. The direct consequence is a time-saving by a factor of 2/sup p/-6/sup p/ for an O(n/sup p/) matching algorithm. Our result applies to a fairly large class of available matching algorithms.

A large portion of image contours is characterized by local properties such as sharp variations of the image intensity across the contour. The integration of local image descriptors estimated by using these local properties into curvilinear descriptors is a difficult problem from a theoretical viewpoint because of the combinatorially large number of possible curvilinear descriptors. To deal with this difficulty, the notion of compressible graphs is introduced and a contour data model is defined leading to an efficient linear-time algorithm which provably recovers contours with an upper bound on the approximation error.

In this paper, we discuss the problem of recognizing single-dimensional, real-valued, functions in the presence of domain noise (i.e., noise that affects the domain rather than the amplitude). This problem is inspired by the field of on-line character recognition where it is more natural to view the hand as deforming the domain of the character rather than adding noise to its amplitude. The results obtained illustrate the difficulties one faces when dealing with both domain and amplitude deformation of waveforms or images. Our major result is a set of sufficient conditions that a recognition metric has to satisfy. Examples of metrics that satisfy these conditions, and hence are appropriate for recognition when the deformation affects the domain rather than the amplitude, include the supnorm metric and the total variation metric. Furthermore, we extend the results to the case when a waveform is corrupted by both amplitude and domain deformation.

Efficient edge detection algorithms such as Canny's (1986) fail near curve singularities. Moreover, the standard linking algorithms used on top of these detectors often fail because of instabilities in the tracking process (due to multiple responses to the same edge and interference of nearby edges). We propose a hierarchical approach to edge detection based on a graph stabilization method that allows bifurcation resolution in stages. Curve singularities are recovered at the last stage by using "top-down" feedback to select the best curve connections.

A reversible, ergodic, Markov process taking values in the space of polygonally segmented images is constructed. The stationary distribution of this process can be made to correspond to a Gibbs-type distribution for polygonal random fields as introduced by Arak and Surgailis (1989) and a few variants thereof, such as those arising in Bayesian analysis of random fields. Extensions to generalized polygonal random fields are presented where the segmentation boundaries are not necessarily straight line segments.

Considers the problem of computing the length of a curve from digitized versions of the curve using parallel computation. The authors' aim is to study the inherent parallel computational complexity of this problem as a function of the digitization level. Precise formulations for the digitization, the parallel computation, and notions of local and nonlocal computations are given. It is shown that length cannot be computed locally from digitizations on rectangular tessellations. However, for a random tessellation and appropriate deterministic ones, the authors show that the length of straight line segments can be computed locally. Implications of the authors' results for a method for image segmentation and a number of open problems are discussed.>

An extension of the standard probably approximately correct (PAC) learning model that allows the use of generalized samples is introduced. A generalized sample is viewed as a pair consisting of a functional on the concept class together with the value obtained by the functional operating on the unknown concept. It appears that this model can be applied to a number of problems in signal processing and geometric reconstruction to provide sample size bounds under a PAC criterion. A specific application of the generalized model to a problem of curve reconstruction is considered, and some connections with a result from stochastic geometry are discussed.>

In this paper, we introduce an extension of the standard PAC learning model which allows the use of generalized samples. We view a generalized sample as a pair consisting of a functional on the concept class together with the value obtained by the functional operating on the unknown concept. It appears that this model can be applied to a number of problems in signal processing and geometric reconstruction to provide sample size bounds under a PAC criterion. We consider a specific application of the model to a problem of curve reconstruction, and discuss some connections with a result from stochastic geometry.