Nowadays, the leakage of personal information and privacy becomes a major concern in various fields, such as social sciences, genomics, and medicine. To combat this problem, differential privacy has been proposed and soon be widely studied and applied. Meanwhile, the compositional nature of differential privacy has motivated the design and implementation of differentially private machine learning mechanisms. However, as these mechanisms are gradually deployed in practice, a serious problem appears: they find it hard to choose a meaningful ε value or understand the meaning of a chosen ε value in practice. To this end, we propose a novel and efficient approach that would allow users to choose ε according to their utility or accuracy requirement. Specifically, we can efficiently obtain the expected ε value that would generate a private model satisfying the expected empirical loss or expected accuracy, through at least one-round training and some calculations. As product requirements often impose hard accuracy constraints, our approach allows users to focus on their own benefits without paying too much attention to things they don’t understand or care. Both theoretical analysis and experimental results demonstrate high accuracy and broad applicability of our mechanism in practical applications.

Traditional implementations of differential privacy implicitly assume that users have homogeneous privacy requirement for all attributes of data. They provide a uniform level of privacy guarantee for all attributes by using a single privacy budget,$\varepsilon$. However, this brings trouble to users in practice applications, where privacy requirements are often heterogeneous. In this case, users have to make a choice: either setting the privacy level high enough to satisfy even the privacy fundamentalists, which often results in poor model utility, or sacrificing the privacy of some attributes for practical model. Both are hard to be accepted by users. In this paper, we offer users what they probably want, an option that could provide a satisfactory privacy guarantee for important attributes with minimal utility loss. Our method, called heterogeneous differentially private ERM (HDP-ERM), allows the private learning algorithms to guarantee heterogeneous privacy for each attribute of training data. The noise injected in each parameter is adaptive according to its individual privacy budget, so that we can control the privacy-utility trade-off more finely. Experimental results show that our method is able to reach a satisfactory utility when only the important attributes need strong privacy guarantee, while the traditional DP-ERM would only get a useless model (one with low test accuracy) when providing the same level of privacy guarantee.