Complex sequential decision-making planning problems, covering infinite states’ space have been shown to be solvable by AlphaZero type of algorithms. Such an approach that trains a neural model while simulating projection of futures with a Monte Carlo Tree Search algorithm were shown to be applicable to real life planning problems. As such, engineers and users interacting with the resulting policy of behavior might benefit from obtaining automated explanations about these planners’ decisions offline or online. This paper focuses on the information within the Monte Carlo Tree Search data structure. Given its construction, this information contains much of the reasoning of the sequential decision-making algorithm and is essential for its explainability. We show novel methods using information theoretic tools for the simplification and reduction of the Monte Carlo Tree Search and the extraction of information. Such information can be directly used for the construction of human understandable explanations. We show that basic explainability quantities can be calculated with limited additional computational cost, as an integrated part of the Monte Carlo Tree Search construction process. We focus on the theoretical and algorithmic aspects and provide examples of how the methods presented here can be used in the construction of human understandable explanations.

Humans, interacting with automated machines, expect certain behaviors: either because they have experienced this behavior (e.g., driving) or because they build such expectations from the machine (e.g., a user would expect from an AI based personal assistant to recognize all the sentences they might tell in any accent). In reality, these advanced AI systems might not behave perfectly or their optimal decisions might also differ from the subjective optimal decisions a human user might expect. This becomes a challenging problem when considering AI decision making algorithms, controlling the complex behaviors of autonomous vehicles, affected by their uncertain environments and their own sensing suites. This paper presents results from two large, on-line user studies, run in simulated autonomous driving scenarios. Our goal was to assess users’ trust in the automated behaviors, presented with different explanations and HMI solutions. We found that specific explanations, considering the risk of a driving scenario and what the vehicle is planning to do can reduce discomfort and increase understanding of an automated driving maneuver. We also present a data-driven solution to infer an explanation automatically and probabilistically that is the most suitable for a driving context and user’s group according to the data analysis and trust measures examined.

The method of types presented by Csiszár and Körner is a central tool used to develop and analyze the basic properties and constraints on sequences of data over finite alphabets. A central problem considered using these tools is that of data compression, and specifically lossy data compression. In this work we consider this very problem, however, instead of sequences of data we consider directed graphs. We show that given a more natural distortion measure, fitting the data structure of a directed graph, the method of types cannot be applied. The suggested distortion measure aims to preserves the local structure of a directed graph. We build on the recent work of Barvinok and extend the method of types to the two dimensional setting of directed graphs. We see that the extension is quite natural in many ways. Given this extension we provide a lower and upper bound on the rate-distortion problem of lossy compression given the suggested distortion measure.

A renowned information-theoretic formula by Shannon expresses the mutual information rate of a white Gaussian channel with a stationary Gaussian input as an integral of a simple function of the power spectral density of the channel input. We give in this paper a rigorous yet elementary proof of this classical formula. As opposed to all the conventional approaches, which either rely on heavy mathematical machineries or have to resort to some "external" results, our proof, which hinges on a recently proven sampling theorem, is elementary and self- contained, only using some well-known facts from basic calculus and matrix theory.

This paper studies optimization of the minimum mean square error (MMSE) in order to characterize the structure of the least favorable prior distributions. In the first part, the paper characterizes the local behavior of the MMSE in terms of the input distribution and finds the directional derivative of the MMSE at the distribution$P_{\mathbf{X}}$in the direction of the distribution$Q_{\mathbf{X}}$. In the second part of the paper, the directional derivative together with the theory of convex optimization is used to characterize the structure of least favorable distributions. In particular, under mild regularity conditions, it is shown that the support of the least favorable distributions must necessarily be very small and is contained in a nowhere dense set of Lebesgue measure zero. The results of this paper produce both sufficient and necessary conditions for optimality, do not rely on Gaussian statistics assumptions, and are not sensitive to the dimensionality of random vectors. The results are evaluated for the univariate and multivariate random Gaussian cases, and the Poisson case. Finally, as one of the applications, it is shown how the results can be used to characterize the capacity of Gaussian MIMO channels with an amplitude constraint.

This paper considers a Gaussian channel with one transmitter and two receivers. The goal is to maximize the communication rate at the intended/primary receiver subject to a disturbance constraint at the unintended/secondary receiver. The disturbance is measured in terms of the minimum mean square error (MMSE) of the interference that the transmission to the primary receiver inflicts on the secondary receiver. This paper presents a new upper bound for the problem of maximizing the mutual information subject to an MMSE constraint. The new bound holds for vector inputs of any length and recovers a previously known limiting (when the length of the vector input tends to infinity) expression from the work of Bustin et al. The key technical novelty is a new upper bound on the MMSE. This bound allows one to bound the MMSE for all signal-to-noise ratio (SNR) values below a certain SNR at which the MMSE is known (which corresponds to the disturbance constraint). The bound also complements the “single-crossing point property” of the MMSE that upper bounds the MMSE for all SNR values above a certain value at which the MMSE value is known. The MMSE upper bound provides a refined characterization of the phase-transition phenomenon, which manifests, in the limit as the length of the vector input goes to infinity, as a discontinuity of the MMSE for the problem at hand. For vector inputs of size n = 1, a matching lower bound, to within an additive gap of order O(log log(1/MMSE)) (where MMSE is the disturbance constraint), is shown by means of the mixed inputs technique recently introduced by Dytso et al.

We consider lossy compression of random binary matrices under distortion constraints that strive to preserve the structure of the matrix. Specifically, we assume that matrix elements are statistically independent (but not necessarily identically distributed), and that the worst case row/column average distortion is to be controlled. We discuss a natural notion of matrix types termed (R, c)-type, and provide various results concerning its probability and cardinality, as well as a “Sanov-type” result, in the spirit of the method-of-types. We then derive bounds on the associated matrix ratedistortion function via a suitable matrix version of the covering lemma.

This paper considers a problem of communication over an additive noise channel where the noise is distributed according to a Generalized Gaussian (GG) distribution. In the first part of the paper, a number of properties of the family of GG distributions are derived which are of independent interest. For example, considerable attention is given to the properties of the characteristic function of the GG distribution. In the second part of the paper, the capacity of an additive noise channel with GG noise is considered under p-th absolute moment constraints. It is shown that, even though Shannon's upper bound is achievable in some instances, in general such achievability is not possible. Moreover, it is shown that discrete inputs can achieve capacity within a constant gap or full degree of freedom for any p-th absolute moment constraint. Following the seminal work of Smith, the paper also gives a condition under which discrete inputs are exactly optimal.

The problem of estimating an arbitrary random vector from its observation corrupted by additive white Gaussian noise, where the cost function is taken to be the minimum mean pth error (MMPE), is considered. The classical minimum mean square error (MMSE) is a special case of the MMPE. Several bounds, properties, and applications of the MMPE are derived and discussed. The optimal MMPE estimator is found for Gaussian and binary input distributions. Properties of the MMPE as a function of the input distribution, signal-to-noiseratio (SNR) and order p are derived. The “single-crossing-point property” (SCPP) which provides an upper bound on the MMSE, and which together with the mutual information-MMSE relationship is a powerful tool in deriving converse proofs in multiuser information theory, is extended to the MMPE. Moreover, a complementary bound to the SCPP is derived. As a first application of the MMPE, a bound on the conditional differential entropy in terms of the MMPE is provided, which then yields a generalization of the Ozarow-Wyner lower bound on the mutual information achieved by a discrete input on a Gaussian noise channel. As a second application, the MMPE is shown to improve on previous characterizations of the phase transition phenomenon that manifests, in the limit as the length of the capacity achieving code goes to infinity, as a discontinuity of the MMSE as a function of SNR. As a final application, the MMPE is used to show new bounds on the second derivative of mutual information, or the first derivative of the MMSE.

This paper considers the minimum mean p-th error (MMPE) estimation problem: estimating a random vector in the presence of additive white Gaussian noise (AWGN) in order to minimize an Lpnorm of the estimation error. The MMPE generalizes the classical minimum mean square error (MMSE) estimation problem. This paper derives basic properties of the optimal MMPE estimator and MMPE functional. Optimal estimators are found for several inputs of interests, such as Gaussian and binary symbols. Under an appropriate p-th moment constraint, the Gaussian input is shown to be asymptotically the hardest to estimate for any p ≥ 1. By using a conditional version of the MMPE, the famous “MMSE single-crossing point” bound is shown to hold for the MMPE too for all p ≥ 1, up to a multiplicative constant. Finally, the paper develops connections between the conditional differential entropy and the MMPE, which leads to a tighter version of the Ozarow-Wyner lower bound on the rate achieved by discrete inputs on AWGN channels.

This paper considers the signal-to-noise ratio (SNR)-evolution, meaning the behavior as a function of the SNR, of the minimum mean-square error (MMSE) function of code sequences in several multi-user settings in the additive white Gaussian noise regime. The settings investigated in this context include the Gaussian wiretap channel, the Gaussian broadcast channel (BC), and the Gaussian BC with confidential messages (BCC). This paper shows that the specific properties of the SNR-evolution of the MMSE and conditional MMSE functions are necessary and sufficient conditions for capacity or equivocation achieving code sequences. In some cases, the complete SNR-evolution of a family of code sequences can be determined, providing significant insight into the disturbance (in terms of MMSE) such codes have on unintended receivers at other SNRs. Moreover, the effects of an additional MMSE constraint on the capacity region and on the SNR-evolution of code sequences are considered in the BC and BCC settings. Such an analysis emphasizes the tradeoff between rates and limited disturbance on unintended receivers.

This work examines the properties of code sequences for the scalar Gaussian broadcast channel (BC). Specifically, the behavior in terms of the mutual information and minimum mean-square error (MMSE) functions for all signal-to-noise ratios (SNRs) is explored. It is shown that “good”, capacity achieving, code sequences must follow the behavior of a capacity achieving superposition code sequence, even if they use a different encoding-decoding scheme (such as “Dirty Paper Coding”). Necessary and sufficient conditions for reliable decoding in general and specifically for “good” code sequences for the scalar Gaussian BC, in terms of the MMSE and conditional MMSE functions, are derived. Finally, “bad” code sequences, that do not obtain the capacity of the scalar Gaussian BC, are examined. These codes are defined by an additional MMSE constraint at some other SNR. This constraint limits the amount of disturbance these codes may have on some unintended receiver at that SNR. The capacity region, given this constraint, is fully depicted.

This work examines the properties of “good” codes for the scalar Gaussian wiretap channel that achieve the maximum level of equivocation. Specifically, the minimum mean-square error (MMSE) behavior of these codes is explored as a function of the signal-to-noise ratio (SNR). It is first shown that reliable decoding of the codeword at the legitimate receiver and at the eavesdropper, conditioned on the transmitted message, is a necessary and sufficient condition for an optimally secure code sequence. Moreover, it is observed that a stochastic encoder is required for any code sequence with rate below the channel point-to-point capacity. Then, for code sequences attaining the maximum level of equivocation, it is shown that their codebook sequences must resemble “good” point-to-point, capacity achieving, code sequences. Finally, it is shown that the mapping over such “good” codebook sequences that produces a maximum equivocation code must saturate the eavesdropper. These results support several “rules of thumb” in the design of capacity achieving codes for the Gaussian wiretap.

It is shown that, for a subset of input distributions, the effect of maximum rate transmission on an additional transmitted message, over the additive Gaussian noise channel, is, effectively, that of an additional additive Gaussian noise. Meaning, that the behavior of the mutual information and minimum mean-square error are as if additional additive Gaussian noise were transmitted. Such an observation provides corner points of the two-user Gaussian interference channel, for this subset.

The scalar additive Gaussian noise channel has the “single crossing point” property between the minimum mean square error (MMSE) in the estimation of the input given the channel output, assuming a Gaussian input to the channel, and the MMSE assuming an arbitrary input. This paper extends the result to the parallel vector additive Gaussian channel in three phases. 1) The channel matrix is the identity matrix, and we limit the Gaussian input to a vector of Gaussian i.i.d. elements. The “single crossing point” property is with respect to the signal-to-noise ratio (as in the scalar case). 2) The channel matrix is arbitrary, and the Gaussian input is limited to an independent Gaussian input. A “single crossing point” property is derived for each diagonal element of the MMSE matrix. 3) The Gaussian input is allowed to be an arbitrary Gaussian random vector. A “single crossing point” property is derived for each eigenvalue of the difference matrix between the two MMSE matrices. These three extensions are then translated to new information theoretic properties on the mutual information, using the I-MMSE relationship, a fundamental relationship between estimation theory and information theory revealed by Guo and coworkers. The results of the last phase are also translated to a new property of Fisher information. Finally, the applicability of all three extensions on information theoretic problems is demonstrated through a proof of a special case of Shannon's vector entropy power inequality, a converse proof of the capacity region of the parallel degraded broadcast channel (BC) under an input per-antenna power constraint and under an input covariance constraint, and a converse proof of the capacity region of the compound parallel degraded BC under an input covariance constraint.

We examine codes, over the additive Gaussian noise channel, designed for reliable communication at some specific signal-to-noise ratio (SNR) and constrained by the permitted minimum mean-square error (MMSE) at lower SNRs. The maximum possible rate is below point-to-point capacity, and hence, these are nonoptimal codes (alternatively referred to as “bad” codes). We show that the maximum possible rate is the one attained by superposition codebooks. Moreover, the MMSE and mutual information behavior as a function of SNR, for any code attaining the maximum rate under the MMSE constraint, is known for all SNR. We also provide a lower bound on the MMSE for finite length codes, as a function of the error probability of the code.

We examine codes, over the additive Gaussian noise channel, designed for reliable communication at some specific signal-to-noise ratio (snr) and constrained by the permitted MMSE at K lower snrs. Extending the result of the single MMSE constrained code, we show that K-layers superposition codes attain the constrained capacity. Moreover, we prove that given a reliable code attaining the multi-MMSE constrained capacity, its MMSE and mutual information, as functions of snr, are completely defined. Thus, no other multi-MMSE constrained capacity achieving code attains better MMSE performance at unconstrained snrs.

This paper extends the “single crossing point” property of the scalar MMSE function, derived by Guo, Shamai and Verdú (first presented in ISIT 2008), to the parallel degraded MIMO scenario. It is shown that the matrix Q(t), which is the difference between the MMSE assuming a Gaussian input and the MMSE assuming an arbitrary input, has, at most, a single crossing point for each of its eigenvalues. Together with the I-MMSE relationship, a fundamental connection between Information Theory and Estimation Theory, this new property is employed to derive results in Information Theory. As a simple application of this property we provide an alternative converse proof for the broadcast channel (BC) capacity region under covariance constraint in this specific setting.

This paper provides a closed-form expression for the secrecy capacity of the multiple-input multiple-output (MIMO) Gaussian wiretap channel, under a power-covariance constraint. Furthermore, the paper specifies the input covariance matrix required in order to attain the capacity. The proof uses the fundamental relationship between information theory and estimation theory in the Gaussian channel, relating the derivative of the mutual information to the minimum mean-square error (MMSE). The proof provides the missing intuition regarding the existence and construction of an enhanced degraded channel that does not increase the secrecy capacity. The concept of enhancement has been used in a previous proof of the problem. Furthermore, the proof presents methods that can be used in proving other MIMO problems, using this fundamental relationship.