For $\tilde f(t) = \exp \left( {\frac{{\alpha - 1}}{\alpha }t} \right)$, this paper shows that the Sibson mutual information is an α-leakage averaged over the adversary’s $\tilde f$ -mean relative information gain (on the secret) at elementary event of channel output Y as well as the joint occurrence of elementary channel input X and output Y . This interpretation is used to derive a sufficient condition that achieves a δ-approximation of ϵ-upper bounded α-leakage. A Y -elementary α-leakage is proposed, extending the existing pointwise maximal leakage to the overall Rényi order range α ∈ [0,∞). Maximizing this Y -elementary leakage over all attributes U of channel input X gives the Rényi divergence. Further, the Rényi capacity is interpreted as the maximal $\tilde f$-mean information leakage over both the adversary’s malicious inference decision and the channel input X (represents the adversary’s prior belief). This suggests an alternating max-max implementation of the existing generalized Blahut-Arimoto method.

We develop a unified Data Processing Inequality PAC-Bayesian framework—abbreviated DPI-PAC-Bayesian—for deriving the generalization error bounds in the supervised learning setting. By embedding the Data Processing Inequality (DPI) into the change-of-measure technique, we obtain explicit bounds on the binary Kullback-Leibler generalization gap for both Rényi divergence and any f-divergence measured between a data-independent prior distribution and an algorithm-dependent posterior distribution. We present three bounds derived under our framework using Rényi, Hellinger p and Chi-Squared divergences. Additionally, our framework also demonstrates a close connection with other well-known bounds. When the prior distribution is chosen to be uniform, our bounds recover to the classical Occam's Razor bound and, crucially, eliminate the extraneous $\log (2\sqrt n )/n$ slack present in the PAC-Bayes bound, thereby achieving tighter bounds. The framework thus bridges data-processing and PAC-Bayesian perspectives, providing a flexible, information-theoretic tool to construct generalization guarantees.

In this article, we define noiseless privacy, as a non-stochastic rival to differential privacy, requiring that the outputs of a mechanism (i.e., function composition of a privacy-preserving mapping and a query) attain only a few values while varying the data of an individual (the logarithm of the number of the distinct values is bounded by the privacy budget). Therefore, the output of the mechanism is not fully informative of the data of the individuals in the dataset. We prove several guarantees for noiselessly-private mechanisms. The information content of the output about the data of an individual, even if an adversary knows all the other entries of the private dataset, is bounded by the privacy budget. The zero-error capacity of memory-less channels using noiselessly private mechanisms for transmission is upper bounded by the privacy budget. The performance of a non-stochastic hypothesis-testing adversary is bounded again by the privacy budget. Assuming that an adversary has access to a stochastic prior on the dataset, we prove that the estimation error of the adversary for individual entries of the dataset is lower bounded by a decreasing function of the privacy budget. In this case, we also show that the maximal leakage is bounded by the privacy budget. In addition to privacy guarantees, we prove that noiselessly-private mechanisms admit composition theorem and post-processing does not weaken their privacy guarantees. We prove that quantization or binning can ensure noiseless privacy if the number of quantization levels is appropriately selected based on the sensitivity of the query and the privacy budget. Finally, we illustrate the privacy merits of noiseless privacy using multiple datasets in energy, transport, and finance.

This article studies the zero-error feedback capacity ofcausaldiscrete channels with memory. First, by extending the classical zero-error feedback capacity concept, a new notion ofuniform zero-error feedback capacity$C_{0f} $for such channels is introduced. Using this notion a tight condition for bounded stabilization of unstable noisy linear systems via causal channels is obtained, assuming no channel state information at either end of the channel. Furthermore, the zero-error feedback capacity of a class of additive noise channels is investigated. It is known that for a discrete channel with correlated additive noise, the ordinary capacity with or without feedback is equal$\log q-\mathcal {H}_{ch} $, where$\mathcal {H}_{ch} $is the entropy rate of the noise process and$q $is the input alphabet size. In this paper, for a class of finite-state additive noise channels (FSANCs), it is shown that the zero-error feedback capacity is either zero or$C_{0f} =\log q -h_{ch} $, where$h_{ch} $is thetopological entropyof the noise process. A condition is given to determine when the zero-error capacity with or without feedback is zero. This, in conjunction with the stabilization result, leads to a “Small-Entropy Theorem”, stating that stabilization over FSANCs can be achieved if the sum of the topological entropies of the linear system and the channel is smaller than$\log q$.

We investigate the extent to which statistical predictive models leak information about their training data. More specifically, based on the use case of household (electrical) energy consumption, we evaluate whether white-box access to auto-regressive (AR) models trained on such data together with background information, such as household energy data aggregates (e.g., monthly billing information) and publicly-available weather data, can lead to inferring fine-grained energy data of any particular household. We construct two adversarial models aiming to infer fine-grained energy consumption patterns. Both threat models use monthly billing information of target households. The second adversary has access to the AR model for a cluster of households containing the target household. Using two real-world energy datasets, we demonstrate that this adversary can apply maximuma posterioriestimation to reconstruct daily consumption of target households with significantly lower error than the first adversary, which serves as a baseline. Such fine-grained data can essentially expose private information, such as occupancy levels. Finally, we use differential privacy (DP) to alleviate the privacy concerns of the adversary in dis-aggregating energy data. Our evaluations show that differentially private model parameters offer strong privacy protection against the adversary with moderate utility, captured in terms of model fitness to the cluster.

This paper considers the problem of publishing data$X$while protecting the correlated sensitive information$S$. We propose a linear method to generate the sanitized data$Y$with the same alphabet$\mathcal{Y}=\mathcal{X}$that attains local differential privacy (LDP) and log-lift at the same time. It is revealed that both LDP and log-lift are inversely proportional to the statistical distance between conditional probability$P_{Y\vert S}(x\vert s)$and marginal probability$P_{Y}(x)$: the closer the two probabilities are, the more private$Y$is. Specifying$P_{Y\vert S}(x\vert s)$that linearly reduces this distance$\vert P_{Y\vert S}(x\vert s)-P_{Y}(x)\vert =(1-\alpha)\vert P_{X\vert S}(x\vert s)-P_{X}(x)\vert, \forall s, x$for some$\alpha\in(0,1]$, we study the problem of how to generate$\mathrm{Y}$from the original data$S$and$X$. The Markov randomization/sanitization scheme$P_{Y\vert X}(x\vert x^{\prime})=P_{Y\vert S,X}(x\vert s,x^{\prime})$is obtained by solving linear equations. The optimal non-Markov sanitization, the transition probability$P_{Y\vert S,X}(x\vert s,x^{\prime})$that depends on$S$,, can be determined by maximizing the data utility subject to linear equality constraints on data privacy. We compute the solution for two linear utility function: the expected distance and total variance distance. It is shown that the non-Markov randomization significantly improves data utility and the marginal probability$P_{X}(x)$remains the same after the linear sanitization method:$P_{Y}(x)=P_{X}(x),\forall x\in \mathcal{X}$.

We consider the problem of publicly releasing a dataset for support vector machine classification while not infringing on the privacy of data subjects (i.e., individuals whose private information is stored in the dataset). The dataset is systematically obfuscated using an additive noise for privacy protection. Motivated by the Cramér-Rao bound, inverse of the trace of the Fisher information matrix is used as a measure of the privacy. Conditions are established for ensuring that the classifier extracted from the original dataset and the obfuscated one are close to each other (capturing the utility). The optimal noise distribution is determined by maximizing a weighted sum of the measures of privacy and utility. The optimal privacy-preserving noise is proved to achieve local differential privacy. The results are generalized to a broader class of optimization-based supervised machine learning algorithms. Applicability of the methodology is demonstrated on multiple datasets.

We address the problem of maximizing privacy of stochastic dynamical systems whose state information is released through quantized sensor data. In particular, we consider the setting where information about the system state is obtained using noisy sensor measurements. This data is quantized and transmitted to a (possibly untrustworthy) remote station through a public/unsecured communication network. We aim at keeping (part of) the state of the system private; however, because the network (and/or the remote station) might be unsecure, adversaries might have access to sensor data, which can be used to estimate the system state. To prevent such adversaries from obtaining an accurate state estimate, before transmission, we randomize quantized sensor data using additive random vectors, and send the corrupted data to the remote station instead. We design the joint probability distribution of these additive vectors (over a time window) to minimize the mutual information (our privacy metric) between some linear function of the system state (a desired private output) and the randomized sensor data for a desired level of distortion-how different quantized sensor measurements and distorted data are allowed to be. We pose the problem of synthesising the joint probability distribution of the additive vectors as a convex program subject to linear constraints. Simulation experiments are presented to illustrate our privacy scheme.

It is known that for a discrete channel with correlated additive noise, the ordinary capacity with or without feedback both equal log q-H(Z), where H(Z) is the entropy rate of the noise process Z and q is the alphabet size. In this paper, a class of finite-state additive noise channels is introduced. It is shown that the zero-error feedback capacity of such channels is either zero or C0f= log q - h(Z), where h(Z) is the topological entropy of the noise process. Moreover, the zero-error capacity without feedback is lower-bounded by log q-2h(Z). We explicitly compute the zero-error feedback capacity for several examples, including channels with isolated errors and a Gilbert-Elliot channel.

We propose an operational measure of information leakage in a non-stochastic setting to formalize privacy against a brute-force guessing adversary. We use uncertain variables, non-probabilistic counterparts of random variables, to construct a guessing framework in which an adversary is interested in determining private information based on uncertain reports. We consider brute-force trial-and-error guessing in which an adversary can potentially check all the possibilities of the private information that are compatible with the available outputs to find the actual private realization. The ratio of the worst-case number of guesses for the adversary in the presence of the output and in the absence of it captures the reduction in the adversary’s guessing complexity and is thus used as a measure of private information leakage. We investigate the relationship between the newly-developed measure of information leakage with maximin information and stochastic maximal leakage that are shown to arise in one-shot guessing.

We consider private function evaluation to provide query responses based on private data of multiple untrusted entities in such a way that each cannot learn something substantially new about the data of others. First, we introduce perfect non-stochastic privacy in a two-party scenario. Perfect privacy amounts to conditional unrelatedness of the query response and the private uncertain variable of other individuals conditioned on the uncertain variable of a given entity. We show that perfect privacy can be achieved for queries that are functions of the common uncertain variable, a generalization of the common random variable. We compute the closest approximation of the queries that do not take this form. To provide a trade-off between privacy and utility, we relax the notion of perfect privacy. We define almost perfect privacy and show that this new definition equates to using conditional disassociation instead of conditional unrelatedness in the definition of perfect privacy. Then, we generalize the definitions to multi-party function evaluation (more than two data entities). We prove that uniform quantization of query responses, where the quantization resolution is a function of privacy budget and sensitivity of the query (cf., differential privacy), achieves function evaluation privacy.

In this paper, we apply machine learning to distributed private data owned by multiple data owners, entities with access to non-overlapping training datasets. We use noisy, differentially-private gradients to minimize the fitness cost of the machine learning model using stochastic gradient descent. We quantify the quality of the trained model, using the fitness cost, as a function of privacy budget and size of the distributed datasets to capture the trade-off between privacy and utility in machine learning. This way, we can predict the outcome of collaboration among privacy-aware data owners prior to executing potentially computationally-expensive machine learning algorithms. Particularly, we show that the difference between the fitness of the trained machine learning model using differentially-private gradient queries and the fitness of the trained machine model in the absence of any privacy concerns is inversely proportional to the size of the training datasets squared and the privacy budget squared. We successfully validate the performance prediction with the actual performance of the proposed privacy-aware learning algorithms, applied to: financial datasets for determining interest rates of loans using regression; and detecting credit card frauds using support vector machines.

In this study, I consider privacy against hypothesis testing adversaries within a non‐stochastic framework. He developed a theory of non‐stochastic hypothesis testing by borrowing the notion of uncertain variables from non‐stochastic information theory. I define tests as binary‐valued mappings on uncertain variables and proved a fundamental bound on the best performance of the tests in non‐stochastic hypothesis testing. I provide parallels between stochastic and non‐stochastic hypothesis‐testing frameworks. I use the performance bound in non‐stochastic hypothesis testing to develop a measure of privacy. I then construct the reporting policies with the prescribed privacy and utility guarantees. The utility of a reporting policy is measured by the distance between the reported and original values. Finally, I present the notion of indistinguishability as a measure of privacy by extending the identifiability from the privacy literature to the non‐stochastic framework. I prove that the linear quantisers can indeed achieve identifiability for responding to linear queries on private datasets.

Noisy (stochastic) gradient descent is used to develop privacy-preserving algorithms for solving constrained quadratic optimization problems. The variance of the error of an adversary's estimate of the parameters of the quadratic cost function based on iterates of the algorithm is related to the Fisher information of the noise using the Cramér-Rao bound. This motivates using the Fisher information as a measure of privacy. Noting that the performance degradation in noisy gradient descent is proportional to the variance of the noise, a measure of utility is defined to be equal to the variance of the noise. Trade-off between privacy and utility is balanced by minimizing the Fisher information subject to a constraint on the variance of the noise. The optimal privacy-preserving noise is proved to be Gaussian, which implies that the developed privacy-preserving optimization algorithm also guarantees differential privacy.

We consider a non-stochastic privacy-preserving problem in which an adversary aims to infer sensitive information S from publicly accessible data X without using statistics. We consider the problem of generating and releasing a quantization X̂ of X to minimize the privacy leakage of S to X̂ while maintaining a certain level of utility (or, inversely, the quantization loss). The variables S and X are treated as bounded and non-probabilistic, but are otherwise general. We consider two existing non-stochastic privacy measures, namely the maximum uncertainty reduction L0(S → X̂) and the refined information X̂) (also called the maximin information) of S. For each I (S; privacy measure, we propose a corresponding agglomerative clustering algorithm that converges to a locally optimal quantization solution X̂ by iteratively merging elements in the alphabet of X. To instantiate the solution to this problem, we consider two specific utility measures, the worst-case resolution of Xby observing X̂ and the maximal distortion of the released data X̂. We show that the value of the maximin information I (S; X̂) can be determined by dividing the confusability graph into connected subgraphs. Hence, I (S; X̂) can be reduced by merging nodes connecting subgraphs. The relation to the probabilistic information-theoretic privacy is also studied by noting that the Gács-Körner common information is the stochastic version of I and indicates the attainability of statistical indistinguishability.

Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving clients’ private data from being exposed to adversaries. Nevertheless, private information can still be divulged by analyzing uploaded parameters from clients, e.g., weights trained in deep neural networks. In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noise is added to parameters at the clients’ side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noise. Then we develop a theoretical convergence bound on the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) there is a tradeoff between convergence performance and privacy protection levels, i.e., better convergence performance leads to a lower protection level; 2) given a fixed privacy protection level, increasing the number$N$of overall clients participating in FL can improve the convergence performance; and 3) there is an optimal number aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a$K$-client random scheduling strategy, where$K$($1\leq K< N$) clients are randomly selected from the$N$overall clients to participate in each aggregation. We also develop a corresponding convergence bound for the loss function in this case and the$K$-client random scheduling strategy also retains the above three properties. Moreover, we find that there is an optimal$K$that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the design of various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.

Worst-case models of erasure and symmetric channels are investigated, in which the number of channel errors occurring in each sliding window of a given length is bounded. Upper and lower bounds on their zero-error capacities are derived, with the lower bounds revealing a connection with the topological entropy of the channel dynamics. Necessary and sufficient conditions for linear state estimation with bounded estimation errors via such channels are then obtained, by extending previous results for non-stochastic memoryless channels to those with finite memory. These estimation conditions involve the topological entropies of the linear system and the channel.

A deterministic privacy metric using non-stochastic information theory is developed. Particularly, maximin information is used to construct a measure of information leakage, which is inversely proportional to the measure of privacy. Anyone can submit a query to a trusted agent with access to a non-stochastic uncertain private dataset. Optimal deterministic privacy-preserving policies for responding to the submitted query are computed by maximizing the measure of privacy subject to a constraint on the worst-case quality of the response (i.e., the worst-case difference between the response by the agent and the output of the query computed on the private dataset). The optimal privacy-preserving policy is proved to be a piecewise constant function in the form of a quantization operator applied on the output of the submitted query. The measure of privacy is also used to analyze k -anonymity (a popular deterministic mechanism for privacy-preserving release of datasets using suppression and generalization techniques), proving that it is in fact not privacy preserving.

The development of sustainable transportation infrastructure for people and goods, using new technology and business models, can prove beneficial or detrimental for mobility, depending on its design and use. The focus of this paper is on the increasing impact new mobility services have on traffic patterns and transportation efficiency in general. Over the last decade, the rise of the mobile internet and the usage of mobile devices have enabled ubiquitous traffic information. With the increased adoption of specific smartphone applications, the number of users of routing applications has become large enough to disrupt traffic flow patterns in a significant manner. Similarly, but at a slightly slower pace, novel services for freight transportation and city logistics improve the efficiency of goods transportation and change the use of road infrastructure. This paper provides a general four-layer framework for modeling these new trends. The main motivation behind the development is to provide a unifying formal system description that can at the same time encompass system physics (flow and motion of vehicles) as well as coordination strategies under various information and cooperation structures. To showcase the framework, we apply it to the specific challenge of modeling and analyzing the integration of routing applications in today's transportation systems. In this framework, at the lowest layer (flow dynamics), we distinguish routed users from nonrouted users. A distributed parameter model based on a nonlocal partial differential equation is introduced and analyzed. The second layer incorporates connected services (e.g., routing) and other applications used to optimize the local performance of the system. As inputs to those applications, we propose a third layer introducing the incentive design and global objectives, which are typically varying over the day depending on road and weather conditions, external events, etc. The high-level planning is handled on the fourth layer taking social longterm objectives into account. We illustrate the framework by considering its ability to model at two different levels. Specific to vehicular traffic, numerical examples enable us to demonstrate the links between the traffic network layer and the routing decision layer. With a second example on optimized freight transport, we then discuss the links between the cooperative control layer and the lower layers. The congestion pricing in Stockholm is used to illustrate how also the social planning layer can be incorporated in future mobility services.

In this letter, the interplay between a class of nonlinear estimators and strategic sensors is studied in several participatory-sensing scenarios. It is shown that for the class of estimators, if the strategic sensors have access to noiseless measurements of the to-be-estimated-variable, truth-telling is an equilibrium of the game that models the interplay between the sensors and the estimator. Furthermore, performance of the proposed estimators is examined in the case that the strategic sensors form coalitions and in the presence of noise.

We introduce an atomic congestion game with two types of agents, namely, cars and trucks, to model the traffic flow on a road over various time intervals of the day. Cars maximize their utility by finding a tradeoff between the time they choose to use the road, the average velocity of the flow at that time, and the dynamic congestion tax that they pay for using the road. In addition to these terms, the trucks have an incentive for using the road at the same time as their peers because they have platooning capabilities, which allow them to save fuel. The dynamics and equilibria of this game-theoretic model for the interaction between car traffic and truck platooning incentives are investigated. We use traffic data from Stockholm, Sweden, to validate parts of the modeling assumptions and extract reasonable parameters for the simulations. We use joint strategy fictitious play and average strategy fictitious play to learn a pure strategy Nash equilibrium of this game. We perform a comprehensive simulation study to understand the influence of various factors, such as the drivers' value of time and the percentage of the trucks that are equipped with platooning devices, on the properties of the Nash equilibrium.