We study an information-theoretic privacy mechanism design problem, where an agent observes useful data $Y$ that is arbitrarily correlated with sensitive data $X$, and design disclosed data $U$ generated from $Y$ (the agent has no direct access to $X$). We introduce \emph{sparse point-wise privacy leakage}, a worst-case privacy criterion that enforces two simultaneous constraints for every disclosed symbol $u\in\mathcal{U}$: (i) $u$ may be correlated with at most $N$ realizations of $X$, and (ii) the total leakage toward those realizations is bounded. In the high-privacy regime, we use concepts from information geometry to obtain a local quadratic approximation of mutual information which measures utility between $U$ and $Y$. When the leakage matrix $P_{X|Y}$ is invertible, this approximation reduces the design problem to a sparse quadratic maximization, known as the Rayleigh-quotient problem, with an $\ell_0$ constraint. We further show that, for the approximated problem, one can without loss of optimality restrict attention to a binary released variable $U$ with a uniform distribution. For small alphabet sizes, the exact sparsity-constrained optimum can be computed via combinatorial support enumeration, which quickly becomes intractable as the dimension grows. For general dimensions, the resulting sparse Rayleigh-quotient maximization is NP-hard and closely related to sparse principal component analysis (PCA). We propose a convex semidefinite programming (SDP) relaxation that is solvable in polynomial time and provides a tractable surrogate for the NP-hard design, together with a simple rounding procedure to recover a feasible leakage direction. We also identify a sparsity threshold beyond which the sparse optimum saturates at the unconstrained spectral value and the SDP relaxation becomes tight.

An information-theoretic privacy mechanism design is studied, where an agent observes useful data $Y$ which is correlated with the private data $X$. The agent wants to reveal the information to a user, hence, the agent utilizes a privacy mechanism to produce disclosed data $U$ that can be revealed. We assume that the agent has no direct access to $X$, i.e., the private data is hidden. We study privacy mechanism design that maximizes the disclosed information about $Y$, measured by the mutual information between $Y$ and $U$, while satisfying a point-wise constraint with different privacy leakage budgets. We introduce a new measure, called the \emph{multi-level point-wise leakage}, which allows us to impose different leakage levels for different realizations of $U$. In contrast to previous studies on point-wise measures, which use the same leakage level for each realization, we consider a more general scenario in which each data point can leak information up to a different threshold. As a result, this concept also covers cases in which some data points should not leak any information about the private data, i.e., they must satisfy perfect privacy. In other words, a combination of perfect privacy and non-zero leakage can be considered. When the leakage is sufficiently small, concepts from information geometry allow us to locally approximate the mutual information. We show that when the leakage matrix $P_{X|Y}$ is invertible, utilizing this approximation leads to a quadratic optimization problem that has closed-form solution under some constraints. In particular, we show that it is sufficient to consider only binary $U$ to attain the optimal utility. This leads to simple privacy designs with low complexity which are based on finding the maximum singular value and singular vector of a matrix.

A cache-aided compression problem with perfect privacy is studied, where a server has access to a database ofNfiles, (Y1, ...,YN), each of sizeFbits. The server is connected toKusers through a shared link, where each user has access to a local cache of sizeMFbits. In the placement phase, the server fills the users’ caches without prior knowledge of their future demands, while the delivery phase takes place after the users send their demands to the server. We assume that each fileYiis arbitrarily correlated with a private attributeX, and an adversary is assumed to have access to the shared link. The users and the server have access to a shared secret keyW. The goal is to design the cache contents and the delivered messageCsuch that the average length ofCis minimized, while satisfying: i. The responseCdoes not disclose any information aboutX, i.e.,XandCare statistically independent yieldingI(X;C) = 0, which corresponds to the perfect privacy constraint; ii. Useriis able to decode its demand,Ydi, by using its local cacheZi, delivered messageC, and the shared secret keyW. Due to the correlation of database with the private attribute, existing codes for cache-aided delivery do not fulfill the perfect privacy constraint. Indeed, in this work, we propose a lossless variable-length coding scheme that combines privacy-aware compression with coded caching techniques. In particular, we use two-part code construction and Functional Representation Lemma. Furthermore, we propose an alternative coding scheme based on the minimum entropy coupling concept and a greedy entropy-based algorithm. We show that the proposed scheme improves the previous results obtained by Functional Representation Lemma. Considering two special cases we improve both coding schemes using the common information concept. Finally, we compare the proposed schemes in numerical examples and provide an application considering an encoder with limited buffer size.

In this paper, we study the fundamental limits in the design of fair representations under different demographic (statistical) parity constraints through the lens of information theory. We first consider the design problem achieving perfect demographic parity. More specifically, an agent uses some useful dataXto solve a taskT. Since bothXandTare correlated with some sensitive attribute or secretS, the agent designs a representationYthat has no information about the sensitive attributeS, i.e., satisfying perfect demographic parity constraint, implying thatI(Y ; S)= 0. We then relax the perfect demographic parity and consider a bounded-parity constraint, implying thatI(Y ; S)≤ ϵ. Under perfect demographic parity, we consider two scenarios. First, we consider a design problem where we want to maximize the informationI(Y ; T)that the representation contains about the task, while constraining the level of compression (or encoding rate), that is, ensuring thatI(Y ;X)≤r. Second, inspired by the Conditional Fairness Bottleneck problem, we consider a design problem where we want to maximize the informationI(Y ; T|S)that the representation contains about the task which is not shared by the sensitive attribute, while constraining the amount of irrelevant information, that is, ensuring thatI(Y ;X|T, S)≤r. Under bounded demographic parity, we designYthat maximizes the mutual informationI(Y ; T)about the task while satisfying a bounded compression (or encoding rate) constraint, that is, ensuring thatI(Y ;X)≤r. Simultaneously,Ysatisfies the bounded demographic parity constraintI(Y ; S)≤ ϵ. To designY, we use extended versions of the Functional Representation Lemma and the Strong Functional Representation Lemma which are based on randomization techniques and study the tightness of the obtained bounds in special cases. Every design problem studied in this paper can also be interpreted as a code design problem with either perfect privacy or a bounded leakage constraint and a limited rate, considering the sensitive attribute as a secret.

We study two semantic communication problems with privacy constraints considering single-task and multi-task scenarios. In both scenarios, an encoder has access to an information source arbitrarily correlated with some latent private information. In the single-task scenario, a user has a task, and the encoder designs a message to be revealed, which is called the semantic of the information source. Due to the privacy constraints, the semantic cannot be disclosed directly, so the encoder adds noise to produce data that can be disclosed. The goal is to design the disclosed message that maximizes the utility attained by the user while satisfying a privacy constraint. In the multi-task scenario, the user hasLtasks with priorities. Similarly, the encoder designs the disclosed message by adding noise to the semantic, which is optimized for the intended tasks. The goal is to design a mechanism to produce the disclosed message that maximizes the weighted sum of the utilities achieved by the user while satisfying a privacy constraint on the private data. In this work, we first consider the single-task scenario and design the added noise utilizing various methods, including the extended versions of the Functional Representation Lemma, Strong Functional Representation Lemma, and the separation technique. By designing the added noise, we obtain lower bounds with constructive proofs. We then study the multi-task scenario and derive a simple privacy mechanism design considering the source semantics. We show that in the multi-task scenario the main problem can be divided into multiple parallel single-task problems. In both scenarios, the obtained lower and upper bounds are studied considering different cases to study their tightness. We show that under some assumptions our proposed designs are optimal. We provide a few numerical experiments based on the MNIST dataset and medical applications to illustrate the designs and evaluate the bounds, considering both single and multi-task scenarios. Finally, we study an application where a semantic communication with two separate blind encoders is considered.

Integrated sensing and communications (ISAC) is a promising component of 6G networks, fusing communication and radar technologies to facilitate new services. Additionally, the use of extremely large-scale antenna arrays (ELAA) at the ISAC common receiver not only facilitates terahertz-rate communication links but also significantly enhances the accuracy of target detection in radar applications. In practical scenarios, communication scatterers and radar targets often reside in close proximity to the ISAC receiver. This, combined with the use of ELAA, fundamentally alters the electromagnetic characteristics of wireless and radar channels, shifting from far-field planar-wave propagation to near-field spherical wave propagation. Under the far-field planar-wave model, the phase of the array response vector varies linearly with the antenna index. In contrast, in the near-field spherical wave model, this phase relationship becomes nonlinear. This shift presents a fundamental challenge: the widely-used Fourier analysis can no longer be directly applied for target detection and communication channel estimation at the ISAC common receiver. In this work, we propose a feasible solution to address this fundamental issue. Specifically, we demonstrate that there exists a high-dimensional space in which the phase nonlinearity can be expressed as linear. Leveraging this insight, we develop a lifted super-resolution framework that simultaneously performs communication channel estimation and extracts target parameters with high precision.

We study a semantic communication problem with a privacy constraint where an encoder consists of two separate parts, e.g., encoder 1 and encoder 2. The first encoder has access to information source X = (X1,…,XN) which is arbitrarily correlated with private data S. The private data is not accessible by encoder 1, however, the second encoder has access to it and the output of encoder 1. A user asks for a task h(X) and the first encoder designs the semantic of the information source f(X) to disclose. Due to the privacy constraints f(X) can not be revealed directly to the user and the second encoder applies a statistical privacy mechanism to produce disclosed data U. Here, we assume that encoder 2 has no access to the task and the design of the disclosed data is based on the semantic and the private data.In this work, we propose a novel approach where U is produced by solving a privacy-utility trade-off based on the semantic and the private data. We design U utilizing different methods such as using extended versions of the Functional Representation Lemma and the Strong Functional Representation Lemma. We evaluate our design by computing the utility attained by the user. Finally, we study and compare the obtained bounds in a numerical example.

We study an information-theoretic privacy mechanism design, where an agent observes useful data Y and wants to reveal the information to a user. Since the useful data is correlated with the private data X, the agent uses a privacy mechanism to produce disclosed data U that can be released. We assume that the agent observes Y and has no direct access to X, i.e., the private data is hidden. We study the privacy mechanism design that maximizes the revealed information about Y while satisfying a bounded Local Information Privacy (LIP) criterion. When the leakage is sufficiently small, concepts from information geometry allow us to locally approximate the mutual information. By utilizing this approximation the main privacy-utility trade-off problem can be rewritten as a quadratic optimization problem that has closed-form solution under some constraints. For the cases where the closed-form solution is not obtained we provide lower bounds on it. In contrast to the previous works that have complexity issues, here, we provide simple privacy designs with low complexity which are based on finding the maximum singular value and singular vector of a matrix. To do so, we follow two approaches where in the first one we find a lower bound on the main problem and then approximate it, however, in the second approach we approximate the main problem directly.In this work, we present geometrical studies of the proposed methods and in a numerical example we compare our results considering both approaches with the optimal solution and the previous methods. Finally, we discuss how the proposed methods can be applied to deal with differential privacy.

A multi-user private data compression problem is studied. A server has access to a database of$N$files,$(Y_{1},\ldots,\ Y_{N})$, each of size$F$bits and is connected to an encoder. The encoder is connected through an unsecured link to a user. We assume that each file$Y_{i}$is arbitrarily correlated with a private attribute$X$, which is assumed to be accessible by the encoder. Moreover, an adversary is assumed to have access to the link. The users and the encoder have access to a shared secret key$W$. We assume that at each time the user asks for a file$Y_{d_{i}}$, where$(d_{1},\ \ldots,\ d_{K})$corresponds to the demand vector. The goal is to design the delivered message$\mathcal{C}=(\mathcal{C}_{1},\ \ldots,\mathcal{C}_{K})$after the user send his demands to the encoder such that the average length of$\mathcal{C}$is minimized, while satisfying:$\mathbf{i}$. The message$\mathcal{C}$does not reveal any information about$X$, i.e.,$X$and$\mathcal{C}$are independent, which corresponds to the perfect privacy constraint; ii. The user is able to decode its demands,$Y_{d_{i}}$, by using$\mathcal{C}$, and the shared key$W$. Here, the encoder sequentially encode each demand$Y_{d_{i}}$at time$i$, using the shared key and previous encoded messages. We propose a variable-length coding scheme that uses privacy-aware compression techniques. We study proposed upper and lower bounds on the average length of$\mathcal{C}$in an example. Finally, we study an application considering cache-aided networks.

In this paper, we study an information-theoretic problem of designing a fair representation that attains bounded statistical (demographic) parity. More specifically, an agent uses some useful data$X$to solve a task$T$. Since both$X$and$T$are correlated with some sensitive attribute or secret$S$, the agent designs a representation$Y$that satisfies a bounded statistical parity and/or privacy leakage constraint, that is, such that$I(Y; S) \leq \epsilon$. Here, we relax the perfect demographic (statistical) parity and consider a bounded-parity constraint. In this work, we design the representation$Y$that maximizes the mutual information$I(Y; T)$about the task while satisfying a bounded compression (or encoding rate) constraint, that is, ensuring that$I(Y; X) \leq r$. Simultaneously,$Y$satisfies the bounded statistical parity constraint$I(Y; S) \leq \epsilon$. To design$Y$, we use extended versions of the Functional Representation Lemma and the Strong Functional Representation Lemma which are based on randomization techniques and study the tightness of the obtained bounds in special cases. The main idea to derive the lower bounds is to use randomization over useful data$X$or sensitive data$S$. Considering perfect demographic parity, i.e.,$\epsilon=0$, we improve the existing results (lower bounds) by using a tighter version of the Strong Functional Representation Lemma and propose new upper bounds. We then propose upper and lower bounds for the main problem and show that allowing non-zero leakage can improve the attained utility. Finally, we study the bounds and compare them in a numerical example. The problem studied in this paper can also be interpreted as one of code design with bounded leakage and bounded rate privacy considering the sensitive attribute as a secret.

We study a multi-task private semantic communication problem, in which an encoder has access to an information source arbitrarily correlated with some latent private data. A user has$L$tasks with priorities. The encoder designs a message to be revealed which is called the semantic of the information source. Due to the privacy constraints the semantic can not be disclosed directly and the encoder adds noise to produce disclosed data. The goal is to design the disclosed data that maximizes the weighted sum of the utilities achieved by the user while satisfying a privacy constraint on the private data. In this work, we first consider a single-task scenario and design the added noise utilizing various methods including the extended versions of the Functional Representation Lemma, Strong Functional Representation Lemma, and separation technique. We then study the multi-task scenario and derive a simple design of the source semantics. We show that in the multi-task scenario the main problem can be divided into multiple parallel single-task problems.

In this article, we study the fundamental limits in the design of fair and/or private representations achieving perfect demographic parity and/or perfect privacy through the lens of information theory. More precisely, given some useful data$X$that we wish to employ to solve a task$T$, we consider the design of a representation$Y$that has no information of some sensitive attribute or secret$s$, that is, such that$I(Y;S)=0$. We consider two scenarios. First, we consider a design desiderata where we want to maximize the information$I(Y;T)$that the representation contains about the task, while constraining the level of compression (or encoding rate), that is, ensuring that$I(Y;X)\leq r$. Second, inspired by the Conditional Fairness Bottleneck problem, we consider a design desiderata where we want to maximize the information$I(Y,\ T\vert S)$that the representation contains about the task which is not shared by the sensitive attribute or secret, while constraining the amount of irrelevant information, that is, ensuring that$I(Y;X\vert T,\ S)\leq r$. In both cases, we employ extended versions of the Functional Representation Lemma and the Strong Functional Representation Lemma and study the tightness of the obtained bounds. Every result here can also be interpreted as a coding with perfect privacy problem by considering the sensitive attribute as a secret.

We study an information theoretic privacy mechanism design problem for two scenarios where the private data is either observable or hidden. In the hidden private data scenario, an agent observes useful dataYthat is correlated with private dataX, and generate disclosed dataUwhich maximizes the revealed information aboutYwhile satisfying a bounded privacy leakage constraint. Considering the other scenario, the agent has additional access toX. To design the privacy mechanism, we first extend the Functional Representation Lemma and Strong Functional Representation Lemma by relaxing the independence condition and thereby allowing a certain leakage. We then find lower and upper bounds on the privacy-utility trade-offs in both scenarios. In particular, for the case where no leakage is allowed andXis observable, our upper and lower bounds improve previous bounds. Considering bounded mutual information as privacy constraint and the observable private data scenario we show that if the common information and mutual information betweenXandYare equal, then the attained upper bound is tight. Finally, the privacy-utility trade-off with prioritized private data is studied where part ofXis more private than the remaining part.

A privacy mechanism design problem is studied through the lens of information theory. In this work, an agent observes useful data Y = (Y1,…,YN) that is correlated with private data X = (X1,…,XN) which is assumed to be also accessible by the agent. Here, we consider K users where user i demands a sub-vector of Y, denoted by Ci. The agent wishes to disclose Cito user i. A privacy mechanism is designed to generate disclosed data U which maximizes a linear combinations of the users utilities while satisfying a bounded privacy constraint in terms of mutual information. In a similar work it has been assumed that Xiis a deterministic function of Yi, however in this work we let Xiand Yibe arbitrarily correlated.First, an upper bound on the privacy-utility trade-off is obtained by using a specific transformation, Functional Representation Lemma and Strong Functional Representation Lemma, then we show that the upper bound can be decomposed into N parallel problems. Next, lower bounds on privacy-utility tradeoff are derived using Functional Representation Lemma and Strong Functional Representation Lemma. The upper bound is tight within a constant and the lower bounds assert that the disclosed data is independent of all $\left\{ {{X_j}} \right\}_{i = 1}^N$ except one which we allocate the maximum allowed leakage to it. Finally, the obtained bounds are studied in special cases.

A private cache-aided compression problem is studied, where a server has access to a database of$N$files,$(Y_{1},\ldots,Y_{N})$, each of size$F$bits and is connected through a shared link to$K$users, each equipped with a local cache of size$MF$bits. In the placement phase, the server fills the users' caches without knowing their demands, while the delivery phase takes place after the users send their demands to the server. We assume that each file$Y_{i}$is arbitrarily correlated with a private attribute$X$, and an adversary is assumed to have access to the shared link. The users and the server have access to a shared key$W$. The goal is to design the cache contents and the delivered message$\mathcal{C}$such that the average length of$\mathcal{C}$is minimized, while satisfying:$\mathbf{i}$. The response$\mathcal{C}$does not reveal any information about$X$, i.e.,$X$and$\mathcal{C}$are independent, which corresponds to the perfect privacy constraint;$\mathbf{ii}$. User$i$is able to decode its demand,$Y_{d_{i}}$, by using$\mathcal{C}$, its local cache$Z_{i}$, and the shared key$W$. Since the database is correlated with$X$, existing codes for cache-aided delivery do not satisfy the perfect privacy condition. Indeed, we propose a variable-length coding scheme that combines privacy-aware compression with coded caching techniques. In particular, we use two-part code construction and Functional Representation Lemma. Finally, we extend the results to the case, where$X$and$\mathcal{C}$can be correlated, i.e., non-zero leakage is allowed.

The design of privacy mechanisms for two scenarios is studied where the private data is hidden or observable. In the first scenario, an agent observes useful data Y , which is correlated with private data X, and wants to disclose the useful information to a user. A privacy mechanism is employed to generate data U that maximizes the revealed information about Y while satisfying a privacy criterion. In the second scenario, the agent has additionally access to the private data. To this end, the Functional Representation Lemma and Strong Functional Representation Lemma are extended relaxing the independence condition and thereby allowing a certain leakage. Lower bounds on privacy-utility trade-off are derived for the second scenario as well as upper bounds for both scenarios. In particular, for the case where no leakage is allowed, our upper and lower bounds improve previous bounds.

An information theoretic privacy mechanism design problem for two scenarios is studied where the private data is either hidden or observable. In each scenario, privacy leakage constraints are considered using two different measures. In these scenarios the private data is hidden or observable. In the first scenario, an agent observes useful data Y that is correlated with private data X, and wishes to disclose the useful information to a user. A privacy mechanism is designed to generate disclosed data U which maximizes the revealed information about Y while satisfying a per-letter privacy constraint. In the second scenario, the agent has additionally access to the private data. First, the Functional Representation Lemma and Strong Functional Representation Lemma are extended by relaxing the independence condition to find a lower bound considering the second scenario. Next, lower bounds as well as upper bounds on privacy-utility trade-off are derived for both scenarios. In particular, for the case where X is deterministic function of Y, we show that our upper and lower bounds are asymptotically optimal considering the first scenario.

We study a statistical signal processing privacy problem, where an agent observes useful data$Y$and wants to reveal the information to a user. Since the useful data is correlated with the private data$X$, the agent employs a privacy mechanism to generate data$U$that can be released. We study the privacy mechanism design that maximizes the revealed information about$Y$while satisfying a strong$\ell _{1}$-privacy criterion. When a sufficiently small leakage is allowed, we show that the optimizer distributions of the privacy mechanism design problem have a specific geometry, i.e., they are perturbations of fixed vector distributions. This geometrical structure allows us to use a local approximation of the conditional entropy. By using this approximation the original optimization problem can be reduced to a linear program so that an approximate solution for the optimal privacy mechanism can be easily obtained. The main contribution of this work is to consider a non-invertible leakage matrix with non-zero leakage. In our first example, inspired by a watermark application, we first demonstrate the accuracy of the approximation. Then, we employ different measures for utility and privacy leakage to compare the privacy-utility trade-off using our approach with other methods. In particular, we show that by allowing small leakage, significant utility can be achieved using our method compared to the case where no leakage is allowed. In the second and third examples which are based on the MNIST data set and medical applications, we illustrate the suggested design for disclosed data$U$. It has been shown that the letters of$Y$which are disclosing more information about$X$are combined (randomized) to produce a new letter of$U$.

In this paper, we study a stochastic disclosure control problem using information-theoretic methods. The useful data to be disclosed depend on private data that should be protected. Thus, we design a privacy mechanism to produce new data which maximizes the disclosed information about the useful data under a strong χ2-privacy criterion. For sufficiently small leakage, the privacy mechanism design problem can be geometrically studied in the space of probability distributions by a local approximation of the mutual information. By using methods from Euclidean information geometry, the original highly challenging optimization problem can be reduced to a problem of finding the principal right-singular vector of a matrix, which characterizes the optimal privacy mechanism. In two extensions we first consider a scenario where an adversary receives a noisy version of the user's message and then we look for a mechanism which finds U based on observing X, maximizing the mutual information between U and Y while satisfying the privacy criterion on U and Z under the Markov chain (Z, Y)-X-U.

We study an information-theoretic privacy problem, where an agent observes useful data Y and wants to reveal the information to a user. Since the useful data is correlated with sensitive data X, the agent employs a privacy mechanism to produce data U that can be disclosed. Thus, we study the privacy mechanism design that maximizes the revealed information about Y while satisfying an ℓ1-privacy criterion under the Markov chain X-Y -U. When a sufficiently small leakage is allowed, we show that the optimizer of the design problem has a specific structure which allows us to use a local approximation of mutual information. More specifically, we show that the optimizer vectors are perturbations of fixed distributions. By using this approximation the original optimization problem can be reduced to a linear programming problem and an approximate solution for privacy mechanism design can be obtained.