Generative Adversarial Networks (GANs) have been widely used for generating synthetic data for cases where there is a limited size real-world data set or when data holders are unwilling to share their data samples. Recent works showed that GANs, due to overfitting and memorization, might leak information regarding their training data samples. This makes GANs vulnerable to Membership Inference Attacks (MIAs). Several defense strategies have been proposed in the literature to mitigate this privacy issue. Unfortunately, defense strategies based on differential privacy are proven to reduce extensively the quality of the synthetic data points. On the other hand, more recent frameworks such as PrivGAN and PAR-GAN are not suitable for small-size training data sets. In the present work, the overfitting in GANs is studied in terms of the discriminator, and a more general measure of overfitting based on the Bhattacharyya coefficient is defined. Then, inspired by Fano’s inequality, our first defense mechanism against MIAs is proposed. This framework, which requires only a simple modification in the loss function of GANs, is referred to as the maximum entropy GAN or MEGAN and significantly improves the robustness of GANs to MIAs. As a second defense strategy, a more heuristic model based on minimizing the information leaked from the generated samples about the training data points is presented. This approach is referred to as mutual information minimization GAN (MIMGAN) and uses a variational representation of the mutual information to minimize the information that a synthetic sample might leak about the whole training data set. Applying the proposed frameworks to some commonly used data sets against state-of-the-art MIAs reveals that the proposed methods can reduce the accuracy of the adversaries to the level of random guessing accuracy with a small reduction in the quality of the synthetic data samples.

Adversarial robustness has become a topic of growing interest in machine learning since it was observed that neural networks tend to be brittle. We propose an information-geometric formulation of adversarial defense and introduce Fire, a new Fisher-Rao regularization for the categorical cross-entropy loss, which is based on the geodesic distance between the softmax outputs corresponding to natural and perturbed input features. Based on the information-geometric properties of the class of softmax distributions, we derive an explicit characterization of the Fisher-Rao Distance (FRD) for the binary and multiclass cases, and draw some interesting properties as well as connections with standard regularization metrics. Furthermore, we verify on a simple linear and Gaussian model, that all Pareto-optimal points in the accuracy-robustness region can be reached by Fire while other state-of-the-art methods fail. Empirically, we evaluate the performance of various classifiers trained with the proposed loss on standard datasets, showing up to a simultaneous 1% of improvement in terms of clean and robust performances while reducing the training time by 20% over the best-performing methods.

The rapid development and expansion of the Internet of Things (IoT) paradigm has drastically increased the collection and exchange of data between sensors and systems, a phenomenon that raises serious privacy concerns in some domains. In particular, smart meters (SMs) share fine-grained electricity consumption of households with utility providers that can potentially violate users’ privacy as sensitive information is leaked through the data. In order to enhance privacy, electricity consumers can exploit the availability of physical resources such as a rechargeable battery (RB) to shape their power demand as dictated by a privacy-cost management unit (PCMU). In this article, we present a novel method to learn the PCMU policy using deep reinforcement learning (DRL). We adopt the mutual information (MI) between the user’s demand load and the masked load seen by the power grid as a reliable and general privacy measure. Unlike previous studies, we model the whole temporal correlation in the data to learn the MI in its general form and use a neural network to estimate the MI-based reward signal to guide the PCMU learning process. This approach is combined with a model-free DRL algorithm known as the deep double${Q}$-learning (DDQL) method. The performance of the complete DDQL-MI algorithm is assessed empirically using an actual SMs data set and compared with simpler privacy measures. Our results show significant improvements over state-of-the-art privacy-aware demand shaping methods.