Ranking a given set of items is a fundamental problem with widespread applications in information retrieval, recommendation systems, and beyond. Crowdsensing data trading (CDT) systems provide an effective means to collect opinions from distributed workers for ranking tasks. However, these systems face several challenges, including preserving privacy, maintaining ranking accuracy, optimizing worker participation, and ensuring incentive compatibility for all stakeholders. To address these challenges, we propose OPAR-IC, an integrated framework designed to enhance efficiency and privacy in CDT systems. The proposed framework integrates a multi-armed bandit (MAB) based approach to dynamically adjust ranking granularity, a hybrid privacy-preserving mechanism combining logarithmic and binary methods to safeguard worker data, and a combinatorial MAB model for efficient worker recruitment. To enhance worker participation and ensure incentive compatibility among all stakeholders, we integrate a hierarchical Stackelberg game into the framework, balancing competing incentives and achieving equilibrium. Our approach is thoroughly validated through theoretical analysis, offering theoretical guarantees for privacy, ranking accuracy, worker participation, and incentive compatibility. Extensive experiments on real-world datasets demonstrate significant improvements in adaptive ranking accuracy, privacy preservation, worker recruitment, and three-party incentive compatibility.

Big data and strong computing power have promoted artificial intelligence to the era of big models. In particular, ChatGPT’s debut heralded the vigorous development of large models. It is an urgent problem to train large models with trillion-level parameters efficiently. Traditional single-machine training stores all data and model parameters in memory. However, due to the limitation of memory and communication resources, when the amount of data or model parameters increases, the problem of memory shortage and communication blocking often occurs. Therefore, distributed training is the most effective ways to solve the above problems and improve training efficiency. In this paper, we propose the algorithmDL-DP, which can achieve an asymptotically optimal convergence rate$O(1/{\sqrt{TK\Gamma^*}})$while satisfyingε-differential privacy, whereTis the local epoch number,Kis the global maximum iteration number and$\Gamma^*$is the minimum covering index. In particular, when${\Gamma ^*} = N$, DL-DP achieves a convergence rate of$O(1/{\sqrt{TKN}})$, which is equivalent to the best-known FedAvg approach implemented by training the full model at each client. When${\Gamma ^*} = 1$, DL-DP achieves a convergence rate of$O(1/{\sqrt{TK}})$, which is comparable to OAP that assumes all parameters need to be trained at least once in each iteration. Finally, our algorithm has been demonstrated to converge through extensive experiments.

We investigate over-the-air federated learning (OTA-FL) that exploits over-the-air computing (AirComp) to integrate communication and computation seamlessly for FL. Privacy presents a serious obstacle for OTA-FL, as it can be compromised by maliciously manipulating channel state information (CSI). Moreover, the limited band at edge hinders OTA-FL from training large-scale models. It remains open how to enable a multitude of devices with constrained resources and sensitive data to collaboratively train a global model at band-limited edge. To tackle this, we design a novel algorithmPROBEbuilding upon a lightweight over-the-air gradients aggregation rulePB-O-GAR. Specifically,PB-O-GARcombines a random sparsification-like dimension reduction with Gaussian perturbation to provide rigorous privacy and band-adapted communication. It elaborately calibrates the transmission signal according to devices’ perceived CSI for heterogeneous power constraints accommodation and CSI attack resilience. We show that by utilizing the common randomness, which deviates from the conventional FL, random sparsification-like dimension reduction can augment privacy in addition to the intrinsic privacy amplification effect of AirComp. We establish near-optimal convergence rates and explicit trade-offs among privacy, communication and utility forPROBE. Finally, extensive experiments on benchmark datasets are conducted to validate our theoretical findings and showcase the superiority ofPROBEin realistic settings.

In this paper, we propose De-RPOTA, a novel algorithm designed for decentralized learning, equipped with mechanisms for resource adaptation and privacy protection through over-the-air computation. We theoretically analyze the combined effects of limited resources and lossy communication on decentralized learning, showing it converges towards a contraction region defined by a scaled errors version. Remarkably, De-RPOTA achieves a convergence rate of$\mathcal {O}\left ({{\frac {1}{\sqrt {nT}}}}\right)$in scenarios devoid of errors, matching the state-of-the-arts. Additionally, we tackle a power control challenge, breaking it down into transmitter and receiver sub-problems to hasten the De-RPOTA algorithm’s convergence. We also offer a quantifiable privacy assurance for our over-the-air computation methodology. Intriguingly, our findings suggest that network noise can actually strengthen the privacy of aggregated information, with over-the-air computation providing extra security for individual updates. Comprehensive experimental validation confirms De-RPOTA’s efficacy in communication resources limited environments. Specifically, the results on the CIFAR-10 dataset reveal nearly 30% reduction in communication costs compared to the state-of-the-arts, all while maintaining similar levels of learning accuracy, even under resource restrictions.

In this paper, we propose a novel decentralized learning algorithm, namely DLLR-OA, for resource-constrained over-the-air computation with formal privacy guarantee. Theoretically, we characterize how the limited resources induced model-components selection error and compound communication errors jointly impact decentralized learning, making the iterates of DLLR-OA converge to a contraction region centered around a scaled version of the errors. In particular, the convergence rate of the DLLR-OA algorithm in the error-free case [EQUATION] achieves the state-of-the-arts. Besides, we formulate a power control problem and decouple it into two sub-problems of transmitter and receiver to accelerate the convergence of the DLLR-OA algorithm. Furthermore, we provide quantitative privacy guarantee for the proposed over-the-air computation approach. Interestingly, we show that network noise can indeed enhance privacy of aggregated updates while over-the-air computation can further protect individual updates. Finally, the extensive experiments demonstrate that DLLR-OA performs well in the communication resources constrained setting. In particular, numerical results on CIFAR-10 dataset shows nearly 30% communication cost reduction over state-of-the-art baselines with comparable learning accuracy even in resource constrained settings.

Edge intelligence is an emerging technology that enables artificial intelligence on connected systems and devices in close proximity to the data sources. Decentralized Collaborative Learning (DCL) is a novel edge intelligence technique that allows distributed clients to cooperatively train a global learning model without revealing their data. DCL has a wide range of applications in various domains, such as smart city and autonomous driving. However, DCL faces significant challenges in ensuring its trustworthiness, as data isolation and privacy issues make DCL systems vulnerable to adversarial attacks that aim to breach system confidentiality, undermine learning reliability or violate data privacy. Therefore, it is crucial to design DCL in a trustworthy manner, with a focus on security, robustness, and privacy. In this survey, we present a comprehensive review of existing efforts for designing trustworthy DCL systems from the three key aformentioned aspects: security, robustness, and privacy. We analyze the threats that affect the trustworthiness of DCL across different scenarios and assess specific technical solutions for achieving each aspect of Trustworthy DCL (TDCL). Finally, we highlight open challenges and future directions for advancing TDCL research and practice.

Decentralized Federated Learning (DeFL) plays a critical role in improving effectiveness of training and has been proved to give great scope to the development of edge computing. However, on the one hand, inaccessibility of private data and excessively exploiting the data throughout the learning process have become a public concern, and on the other hand the connections between server-less edge devices are always varying due to the mobility of edge intelligent devices. To address the above issues, we propose aPrivacy-Enhanced -Dynamic -Decentralized -Federated -Learning algorithm called PED$ ^{2}$FL in a dynamic edge environment. We design the PED$ ^{2}$FL under the analog transmission scheme, where mobile edge devices transmit privacy preserving data simultaneously and accomplish efficient information aggregation with doubly-stochastic adjacent matrices. With thorough analysis, it can be demonstrated that PED$ ^{2}$FL satisfies$(\epsilon,\delta)$-differential privacy while the per-device privacy budget decays exponentially with the number of the neighbors, which greatly improved the data utility compared to the fixed budget in the orthogonal transmission strategy. PED$ ^{2}$FL has the same convergence rate$\mathcal {O}(\sqrt{\frac{1}{KN}})$as the non-private decentralized learning algorithm D-PSGD without enhanced privacy protection, where$K$and$N$are the total iterations and the number of nodes, respectively. Extensive experiments show that algorithm PED$ ^{2}$FL also performs well with real-world settings.

In this article, we study a distributed privacy-preserving learning problem in social networks with general topology. The agents can communicate with each other over the network, which may result in privacy disclosure, since the trustworthiness of the agents cannot be guaranteed. Given a set of options which yield unknown stochastic rewards, each agent is required to learn the best one, aiming at maximizing the resulting expected average cumulative reward. To serve the above goal, we propose a four-staged distributed algorithm which efficiently exploits the collaboration among the agents while preserving the local privacy for each of them. In particular, our algorithm proceeds iteratively, and in every round, each agent i) randomly perturbs its adoption for the privacy-preserving purpose, ii) disseminates the perturbed adoption over the social network in a nearly uniform manner through random walking, iii) selects an option by referring to the perturbed suggestions received from its peers, and iv) decides whether or not to adopt the selected option as preference according to its latest reward feedback. Through solid theoretical analysis, we quantify the trade-off among the number of agents (or communication overhead), privacy preserving and learning utility. We also perform extensive simulations to verify the efficacy of our proposed social learning algorithm.

This paper studies the privacy-preserving adaptive ranking problem for multi-armed-bandit crowdsourcing, where according to the crowdsourced data, the arms are required to be ranked with a tunable granularity by the untrustworthy third-party platform. Any online worker can provide its data by arm pulls but requires its privacy preserved, which will increase the ranking cost greatly. To improve the quality of the ranking service, we propose a Privacy- Preserving Adaptive Ranking algorithm called PPAR, which can solve the problem with a high probability while differential privacy can be ensured. The total cost of the proposed algorithm is ${\mathcal{O}}(K\ln K)$, which is near optimal compared with the trivial lower bound Ω(K), where K is the number of arms. Our proposed algorithm can also be used to solve the well-studied fully ranking problem and the best arm identification problem, by proper setting the granularity parameter. For the fully ranking problem, PPAR attains the same order of computation complexity with the best-known results without privacy preservation. The efficacy of our algorithm is also verified by extensive experiments on public datasets.

This article studies decentralized federated learning algorithms in wireless IoT networks. The traditional parameter server architecture for federated learning faces some problems such as low fault tolerance, large communication overhead and inaccessibility of private data. To solve these problems, we propose a decentralized wireless federated learning algorithm called DWFL. The algorithm works in a system where the workers are organized in a peer-to-peer and server-less manner, and the workers exchange their privacy preserving data with the analog transmission scheme over wireless channels in parallel. With rigorous analysis, we show that DWFL satisfies$(\epsilon,\delta)$-differential privacy and the privacy budget per worker scales as$\mathcal {O}(\frac{1}{\sqrt{N}})$, in contrast with the constant budget in the orthogonal transmission approach. Furthermore, DWFL converges at the same rate of$\mathcal {O}(\sqrt{\frac{1}{TN}})$as the best known centralized algorithm with a central parameter server. Extensive experiments demonstrate that our algorithm DWFL also performs well in real settings.

This article studies privacy-preserving collaborative learning in decentralized Internet-of-Things (IoT) networks, where the agents exchange information constantly to improve the learnability, and meanwhile make the privacy of agents protected during communications. However, the harsh constraints in IoT make executing collaborative learning much more difficult than well-connected systems composed by servers with strong computation power, due to the weak capacity of devices, limited bandwidth for exchanging information, the asynchronous communication environment, and the necessity of privacy preserving. We show that even if with the harsh constraints in IoT, it still can devise efficient privacy-preserving collaborative learning algorithms, by proposing the first known decentralized collaborative learning algorithm for the fundamental multiarmed bandits problem under the framework of local differential privacy. Rigorous analysis shows that the proposed learning algorithm can make every agent learn the best arm with a high probability and keep the privacy preserved meanwhile. Extensive experiments illustrate that our learning algorithm performs well in real settings.