The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been developed for settings with dependence, including by Takano, Johnson, and Rioul. We extend these works by developing a quantitative version of the entropy power inequality for dependent random vectors. A notable consequence is that an entropy power inequality stated using conditional entropies holds for random vectors whose joint density is log-supermodular.

We propose a privacy auditing framework using minimum mean-squared error (MMSE) estimation and linear auditing models. Our approach provides theoretical lower bounds on the true MMSE of inferring sensitive features from noisy observations of other correlated features. The bounds are in terms of the empirical MMSE under a restricted hypothesis class and a decomposable error term capturing finite sample and approximation effects. For linear auditing models, we derive order-optimal closed-form bounds for classes of relationships between the private and non-private features, including linear mappings, binary symmetric channels, and class-conditional Gaussian models. Through empirical evaluation, we demonstrate that our linear model-based auditing framework serves as a powerful yet tractable tool for MMSE-based privacy auditing that balances theoretical guarantees with practical efficiency.

Abstract. A remarkable conjecture of Feige [ SIAM J. Comput., 35 (2006), pp. 964–984] asserts that for any collection of [Formula: see text] independent nonnegative random variables [Formula: see text], each with expectation at most 1, [Formula: see text], where [Formula: see text]. In this paper, we investigate this conjecture for the class of discrete log-concave probability distributions, and we prove a strengthened version. More specifically, we show that the conjectured bound [Formula: see text] holds when [Formula: see text]’s are independent discrete log-concave with arbitrary expectation.

Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of mixture distributions is of fundamental importance in many of these applications. This article provides sharp bounds on the entropy concavity deficit, which is the difference between the differential entropy of the mixture and the weighted sum of differential entropies of constituent components. Toward establishing lower and upper bounds on the concavity deficit, results that are of importance in their own right are obtained. In order to obtain nontrivial upper bounds, properties of the skew-divergence are developed and notions of “skew”$f$-divergences are introduced; a reverse Pinsker inequality and a bound on Jensen-Shannon divergence are obtained along the way. Complementary lower bounds are derived with special attention paid to the case that corresponds to independent summation of a continuous and a discrete random variable. Several applications of the bounds are delineated, including to mutual information of additive noise channels, thermodynamics of computation, and functional inequalities.

We establish a discrete analog of the Rényi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log e of the usual Shannon entropy. Additionally we investigate the entropic Rogers-Shephard inequality studied by Madiman and Kontoyannis, and establish a sharp Rényi version for certain parameters in both the continuous and discrete cases.

We establish a discrete analog of the Rényi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log2e of the usual Shannon entropy. With the additional assumption that the variable is monotone we obtain a sharp bound of loge.

We establish a central limit theorem for Rényi entropies when the Rényi parameters belong to (0, 1) for a large class of random vectors. This complements a celebrated result of Barron (1986). As an application, we show that a general Rényi entropy power inequality fails when the Rényi parameter is in (0, 1).

In this paper, we investigate the role of convexity in entropy power inequalities. We establish Rényi entropy power inequalities of order r ∈ (0, 1) for a large class of densities, the so-called s-concave densities. This extends recent works on Rényi entropy power inequalities.

Using a sharp version of the reverse Young inequality, and a Rényi entropy comparison result due to Fradelizi, Madiman, and Wang (2016), the authors derive Rényi entropy power inequalities for log-concave random vectors when Rényi parameters belong to [0, 1]. Furthermore, the estimates are shown to be sharp up to absolute constants.

Cover and Zhang proved a certain reversal of the Entropy Power Inequality for the sum of (possibly dependent) random variables possessing the same log-concave density, and what is more that log-concave densities were the only densities that satisfied such an inequality. In this work the authors consider the analogous reversal of recent Renyi Entropy Power Inequalities for random vectors and again show that not only do they hold for s-concave densities, but that s-concave densities are characterized by satisfying said inequalities.

Using a sharp version of the reverse Young inequality, and a Renyi entropy comparison result due to Fradelizi, Madiman, and Wang, the authors derive a Renyi entropy power inequality for log-concave random vectors when Renyi parameters belong to [0, 1]. A discussion of symmetric decreasing rearrangements of random variables strengthens the inequality and guides the exploration as to its sharpness.

Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and the other discrete. In the case where the continuous random variable is Gaussian, we derive strong quantitative bounds on the deficit in the inequality. More explicitly it is shown that the decay of the deficit is sub-Gaussian with respect to the reciprocal of the standard deviation of the Gaussian variable. What is more, up to rational terms these results are shown to be sharp.

We develop a general notion of rearrangement for certain metric groups, and prove a Hardy-Littlewood type inequality. Combining this with a characterization of the extreme points of the set of probability measures with bounded densities with respect to a reference measure, we establish a general min-entropy inequality for convolutions. Special attention is paid to the integers where a min-entropy power inequality is conjectured and a partial result proved.

An optimal ∞-Rényi entropy power inequality is derived for d-dimensional random vectors. In fact, the authors establish a matrix ∞-EPI analogous to the generalization of the classical EPI established by Zamir and Feder. The result is achieved by demonstrating uniform distributions as extremizers of a certain class of ∞-Rényi entropy inequalities, and then putting forth a new rearrangement inequality for the ∞-Rényi entropy. Quantitative results are then derived as consequences of a new geometric inequality for uniform distributions on Euclidean balls.

We explore conditions under which a reverse Rényi entropy power inequality holds for random vectors with s-concave densities, and also discuss connections with Convex Geometry.