Local differential privacy (LDP) is increasingly employed in privacy-preserving machine learning to protect user data before sharing it with an untrusted aggregator. Most LDP methods assume that users possess only a single data record, which is a significant limitation since users often gather extensive datasets (e.g., images, text, time-series data) and frequently have access to public datasets. To address this limitation, we propose a locally private sampling framework that leverages both the private and public datasets of each user. Specifically, we assume each user has two distributions: $p$ and $q$ that represent their private and public datasets, respectively. The objective is to design a mechanism that generates a private sample approximating $p$ while simultaneously preserving $q$. We frame this objective as a minimax optimization problem using $f$-divergence as the utility measure. We fully characterize the minimax optimal mechanisms for general $f$-divergences provided that $p$ and $q$ are discrete distributions. Remarkably, we demonstrate that this optimal mechanism is universal across all $f$-divergences. Experiments validate the effectiveness of our minimax optimal mechanism compared to the state-of-the-art private sampler.

We investigate the mixing times of Markov kernels under$\mathsf{E}_{\gamma}-\mathbf{divergence}$. We demonstrate that the zero-error$\mathsf{E}_{\gamma}- \mathbf{mixing}$time, for any$\gamma > 1$, of irreducible and aperiodic Markov chains, is bounded, a property that is not shared by the TV-mixing time. We further obtain upper bounds on the$\mathsf{E}_{\gamma}-\mathbf{mixing}$times for a broad family of contractive Markov kernels via a new non-linear strong data processing inequality for the$\mathsf{E}_{\gamma}- \mathbf{divergence}$. We apply our results to derive new bounds for the local differential privacy guarantees offered by the sequential application of a privacy mechanism to data.

We study the course allocation problem, where universities assign course schedules to students. The current state-of-the-art mechanism, Course Match, has one major shortcoming: students make significant mistakes when reporting their preferences, which negatively affects welfare and fairness. To address this issue, we introduce a new mechanism, Machine Learning-powered Course Match (MLCM). At the core of MLCM is a machine learning-powered preference elicitation module that iteratively asks personalized pairwise comparison queries to alleviate students' reporting mistakes. Extensive computational experiments, grounded in real-world data, demonstrate that MLCM, with only ten comparison queries, significantly increases both average and minimum student utility by 7%--11% and 17%--29%, respectively. Finally, we highlight MLCM's robustness to changes in the environment and show how our design minimizes the risk of upgrading to MLCM while making the upgrade process simple for universities and seamless for their students.

We investigate the strong data processing inequalities of locally differentially private mechanisms under a specific f -divergence, namely the Eγ-divergence. More specifically, we characterize an upper bound on the Eγ-divergence between PK and QK, the output distributions of an ε-LDP mechanism K, in terms of the Eγ-divergence between the corresponding input distributions P and Q. Interestingly, the tightest such upper bound in the binary case turns out to have a non-multiplicative form. We then extend our results to derive a tight upper bound for general f-divergences. As an application of our main findings, we derive a lower bound on the locally private Bayesian estimation risk that is tighter than the available divergence-based bound in the literature.

Most stochastic gradient descent algorithms can optimize neural networks that are sub-differentiable in their parameters; however, this implies that the neural network’s activation function must exhibit a degree of continuity which limits the neural network model’s uniform approximation capacity to continuous functions. This paper focuses on the case where the discontinuities arise from distinct sub-patterns, each defined on different parts of the input space. We propose a new discontinuous deep neural network model trainable via a decoupled two-step procedure that avoids passing gradient updates through the network’s only and strategically placed, discontinuous unit. We provide approximation guarantees for our architecture in the space of bounded continuous functions and universal approximation guarantees in the space of piecewise continuous functions which we introduced herein. We present a novel semi-supervised two-step training procedure for our discontinuous deep learning model, tailored to its structure, and we provide theoretical support for its effectiveness. The performance of our model and trained with the propose procedure is evaluated experimentally on both real-world financial datasets and synthetic datasets.