We study personalized heterogeneous mean estimation under user-level local differential privacy (LDP), which protects the privacy of local datasets with multiple samples. We consider a distributed environment with$n$users, each associated with one of$k$clusters. Our goal is to estimate the mean of each cluster while preserving the privacy of users' local datasets. Focusing on the scalar case, we propose algorithms that handle scenarios with (unknown) equal and unequal variance proxies (spans of the clusters), and even the number of clusters. Our methods identify the clusters via private frequency estimation and subsequently perform private mean estimation for each cluster. Theoretical guarantees on the trade-off between user-level local differential privacy guarantees and performance are provided.

This paper studies the Multi-Input-Multi-Output (MIMO) interference networks with arbitrary number of transmitters and receivers, where both the transmitters and receivers are equipped with caches. The main goal is to design content placement and delivery schemes that minimize the worst case normalized delivery time (NDT). First, we propose a delivery scheme for the cache-aided Single-Input-Multiple-Output (SIMO) interference networks. Then, we obtain the achievable NDT of the cache-aided MIMO interference networks using the decomposition property of splitting each multi-antenna transmitter into multiple single antenna transmitters. Furthermore, we derive an information-theoretic bound on the optimal NDT of the cache-aided MIMO interference network. Analytical results show that the proposed scheme is within a multiplicative gap of 2 from the derived lower bound independent of all system parameters for any uncoded cache placement scheme. We also derive a novel delivery scheme for the cache-aided Multi-Input-Single-Output (MISO) interference network outperforming our proposed scheme for the cache-aided MIMO interference network. The numerical results show the superiority of our proposed scheme over the state-of-the-art schemes in the literature. Our results show that the coded caching gain has a more significant contribution in reducing the transmission latency than the spatial multiplexing gain. Our results indicate that the receive-antennas become more effective in reducing the NDT than the transmit-antennas in the presence of caches at the receiver-side. In addition, we show that increasing the number of transmit-antennas has a higher gain in reducing the NDT than adding more transmitters in the cache-aided MISO interference network.

Traditionally, an item-level differential privacy framework has been studied for applications in distributed learning. However, when a client has multiple data samples, and might want to also hide its potential participation, a more appropriate notion is that of user-level privacy [1]. In this paper, we develop a distributed private optimization framework that studies the trade-off between user-level local differential privacy guarantees and performance. This is enabled by a novel distributed user-level private mean estimation algorithm using distributed private heavy-hitter estimation. We use this result to develop the privacy-performance trade-off for distributed optimization.

We consider a distributed empirical risk minimization (ERM) optimization problem with communication efficiency and privacy requirements, motivated by the federated learning (FL) framework. We propose a distributed communication-efficient and local differentially private stochastic gradient descent (CLDP-SGD) algorithm and analyze its communication, privacy, and convergence trade-offs. Since each iteration of the CLDP-SGD aggregates the client-side local gradients, we develop (optimal) communication-efficient schemes for mean estimation for several $\ell_p$ spaces under local differential privacy (LDP). To overcome performance limitation of LDP, CLDP-SGD takes advantage of the inherent privacy amplification provided by client subsampling and data subsampling at each selected client (through SGD) as well as the recently developed shuffled model of privacy. For convex loss functions, we prove that the proposed CLDP-SGD algorithm matches the known lower bounds on the \textit{centralized} private ERM while using a finite number of bits per iteration for each client, \emph{i.e.,} effectively getting communication efficiency for “free”. We also provide preliminary experimental results supporting the theory.

The central question studied in this paper is Rényi Differential Privacy (RDP) guarantees for general discrete local randomizers in the shuffle privacy model. In the shuffle model, each of the n clients randomizes its response using a local differentially private (LDP) mechanism and the untrusted server only receives a random permutation (shuffle) of the client responses without association to each client. The principal result in this paper is the first direct RDP bounds for general discrete local randomization in the shuffle privacy model, and we develop new analysis techniques for deriving our results which could be of independent interest. In applications, such an RDP guarantee is most useful when we use it for composing several private interactions. We numerically demonstrate that, for important regimes, with composition our bound yields an improvement in privacy guarantee by a factor of $8\times$ over the state-of-the-art approximate Differential Privacy (DP) guarantee (with standard composition) for shuffle models. Moreover, combining with Poisson subsampling, our result leads to at least $10\times$ improvement over subsampled approximate DP with standard composition.

This paper studies a distributed optimization problem in the federated learning (FL) framework under differential privacy constraints, whereby a set of clients having local samples are connected to an untrusted server, who wants to learn a global model while preserving the privacy of clients' local datasets. We propose a new client sampling called self-sampling that reflects the random availability of clients in the learning process in FL. We analyze the differential privacy of the SGD with client self-sampling by composing amplification by sub-sampling along with amplification by shuffling. Furthermore, we analyze the convergence of the proposed SGD algorithm showing that we can get a reasonable learning performance while preserving the privacy of clients' data even with client self-sampling.

We study privacy in a distributed learning framework, where clients collaboratively build a learning model iteratively throughinteractions with a server from whom we need privacy. Motivated by stochastic optimization and the federated learning (FL) paradigm, we focus on the case where a small fraction of data samples are randomly sub-sampled in each round to participate in the learning process, which also enables privacy amplification. To obtain even stronger local privacy guarantees, we study this in the shuffle privacy model, where each client randomizes its response using a local differentially private (LDP) mechanism and the server only receives a random permutation (shuffle) of the clients' responses without theirassociation to each client. The principal result of this paper is a privacy-optimization performance trade-off for discrete randomization mechanisms in this sub-sampled shuffle privacy model. This is enabledthrough a new theoretical technique to analyze the Renyi Differential Privacy (RDP) of the sub-sampled shuffle model. We numerically demonstrate that, for important regimes, with composition our boundyields significant improvement in privacy guarantee over the state-of-the-art approximate Differential Privacy (DP) guarantee (with strong composition) for sub-sampled shuffled models. We also demonstrate numerically significant improvement in privacy-learning performance operating point using real data sets. Despite these advances, an open question is to bridge the gap between lower and upper privacy bounds in our RDP analysis.

In this paper, we study an estimation problem under the local differential privacy (LDP) framework: There is an ordered list of d values (e.g., real numbers); a set of n users, where each user observes an element from this list and each value in the list is observed by at least one user; and an untrusted server, who wants to estimate the values that the users possess, without learning (in the sense of LDP) the actual value that each user has and its corresponding index in the list. Towards this, we propose two LDP estimation schemes: The first one is under the assumption that the server knows the number of users that observe each value; and the second one is for the general scenario, in which the server does not have this prior information. We show that the minimax risk decreases with the total number of users under a very mild condition on the number of users observing each value.

This paper studies the Multi-Input-Multi-Output (MIMO) interference networks with arbitrary number of transmitters and receivers, where both the transmitters and receivers are equipped with caches. Our objective is to propose content placement and delivery schemes that minimize the worst case normalized delivery time (NDT). First, we design a delivery scheme for the cache-aided Single-Input-Multiple-Output (SIMO) interference networks. Then, we obtain the achievable NDT of the cache-aided MIMO interference networks by using the decomposition property. The numerical results show the superiority of our proposed scheme over the state-of-the-art schemes in the literature. Furthermore, we show that increasing the receiver-cache sizes achieves a higher gain than increasing the number of receive-antennas. In other words, the coded caching gain has a more significant contribution in reducing the transmission latency than the spatial multiplexing gain.

We consider a fog radio access network (F-RAN) with multiple transmitters and receivers, where each transmitter is connected to the cloud via a fronthaul link. Each network node has a finite cache, where it fills its cache with portions of the library files in the off-peak hours. In the delivery phase, receivers request each library files according to an arbitrary popularity distribution. The cloud and the transmitters are responsible for satisfying the requests. This paper aims to design content placement and coded delivery schemes for minimizing both the expected normalized delivery time (NDT) and the peak NDT which measures the transmission latency. We propose achievable transmission policies, and derive an information-theoretic bound on the expected NDT under uniform popularity distribution. The analytical results show that the proposed scheme is within a gap of 2.58 from the derived bound for both the expected NDT under uniform popularity distribution and the peak NDT. Next, we investigate the expected NDT under an arbitrary popularity distribution for an F-RAN with transmitter-side caches only. The achievable and information-theoretic bounds on the expected NDT are derived, where we analytically prove that our proposed scheme is optimal within a gap of two independent of the popularity distribution.

This paper studies the decentralized coded caching for a Fog Radio Access Network (F-RAN), whereby two edge-nodes (ENs) connected to a cloud server via fronthaul links with limited capacity are serving the requests of K r users. We consider all ENs and users are equipped with caches. A decentralized content placement is proposed to independently store contents at each network node during the off-peak hours. After that, we design a coded delivery scheme in order to deliver the user demands during the peak-hours under the objective of minimizing the normalized delivery time (NDT), which refers to the worst case delivery latency. An information-theoretic lower bound on the minimum NDT is derived for arbitrary number of ENs and users. We evaluate numerically the performance of the decentralized scheme. Additionally, we prove the approximate optimality of the decentralized scheme for a special case when the caches are only available at the ENs.

In this paper, we consider a multiple-access fading channel where N users transmit to a single base station (BS) within a limited number of time slots. We assume that each user has a fixed amount of energy available to be consumed over the transmission window. We derive the optimal energy allocation policy for each user that maximizes the total system throughput under two different assumptions on the channel state information. First, we consider the offline allocation problem where the channel states are known a priori before transmission. We solve a convex optimization problem to maximize the sum-throughput under energy and delay constraints. Next, we consider the online allocation problem, where the channels are causally known to the BS and obtain the optimal energy allocation via dynamic programming when the number of users is small. We also develop a suboptimal resource allocation algorithm whose performance is close to the optimal one. Numerical results are presented showing the superiority of the proposed algorithms over baseline algorithms in various scenarios.

In this paper, proactive resource allocation based on user location for point-to-point communication over fading channels is introduced, whereby the source must transmit a packet when the user requests it within a deadline of a single time slot. We introduce a prediction model in which the source predicts the request arrival Tpslots ahead, where Tpdenotes the prediction window (PW) size. The source allocates energy to transmit some bits proactively for each time slot of the PW with the objective of reducing the transmission energy over the non-predictive case. The requests are predicted based on the user location utilizing the prior statistics about the user requests at each location. We also assume that the prediction is not perfect. We propose proactive scheduling policies to minimize the expected energy consumption required to transmit the requested packets under two different assumptions on the channel state information at the source. In the first scenario, offline scheduling, we assume the channel states are known a-priori at the source at the beginning of the PW. In the second scenario, online scheduling, it is assumed that the source has causal knowledge of the channel state. Numerical results are presented showing the gains achieved by using proactive scheduling policies compared with classical (reactive) networks. Simulation results also show that increasing the PW size leads to a significant reduction in the consumed transmission energy even with imperfect prediction.