Graph federated learning enables the collaborative extraction of high-order information from distributed subgraphs while preserving the privacy of raw data. However, graph data often exhibits overlap among different clients. Previous research has demonstrated certain benefits of overlapping data in mitigating data heterogeneity. However, the negative effects have not been explored, particularly in cases where the overlaps are imbalanced across clients. In this paper, we uncover the unfairness issue arising from imbalanced overlapping subgraphs through both empirical observations and theoretical reasoning. To address this issue, we propose FairGFL (FAIRness-aware subGraph Federated Learning), a novel algorithm that enhances cross-client fairness while maintaining model utility in a privacy-preserving manner. Specifically, FairGFL incorporates an interpretable weighted aggregation approach to enhance fairness across clients, leveraging privacy-preserving estimation of their overlapping ratios. Furthermore, FairGFL improves the tradeoff between model utility and fairness by integrating a carefully crafted regularizer into the federated composite loss function. Through extensive experiments on four benchmark graph datasets, we demonstrate that FairGFL outperforms four representative baseline algorithms in terms of both model utility and fairness.

Federated Learning (FL) succeeds in collaborative and privacy-preserving ML model training among multiple distributed data owners. To maintain a healthy FL ecosystem, it is crucial to estimate the contributions of all participants fairly. Due to provable fairness, Shapley value (SV) is widely used for contribution estimation in FL. However, current studies focus on static scenarios with fixed participants and neglect the dynamic settings with the random joining or leaving of participants in practice. This paper fills the gap by proposing FedDSV, a novel contribution estimation framework for FL with dynamic participation. FedDSV supports flexible weighting mechanisms and is compatible with the SV fairness properties in dynamic scenarios. To reduce the computational complexity, we propose a Monte Carlo variant sampling method (SMC), which can adapt well to dynamic scenarios and approximate the true SVs. To evaluate the effectiveness and efficiency of our proposed approaches, extensive experiments under different settings (e.g., frequency switching, low-quality detection, etc.) are conducted on both i.i.d and non-i.i.d. distributions. Experimental results demonstrate that FedDSV can reflect the real utility contribution of data sources for dynamic FL, and SMC can approximate the exact dynamic SVs with larger similarities in a much shorter time than the state-of-the-art methods.

Data synthesis is a promising solution to share data for various downstream analytic tasks without exposing raw data. However, without a theoretical privacy guarantee, a synthetic dataset would still leak some sensitive information in raw data. As a countermeasure, differential privacy is widely adopted to safeguard data synthesis by strictly limiting the released information. This technique is advantageous yet presents significant challenges in the vertical federated setting, where data attributes are distributed among different data parties. The main challenge lies in maintaining privacy while efficiently and precisely reconstructing the correlation between attributes. In this paper, we propose a novel algorithm called VertiMRF, designed explicitly for generating synthetic data in the vertical setting and providing differential privacy protection for all information shared from data parties. We introduce techniques based on the Flajolet-Martin (FM) sketch for encoding local data satisfying differential privacy and estimating cross-party marginals. We provide theoretical privacy and utility proof for encoding in this multi-attribute data. Collecting the locally generated private Markov Random Field (MRF) and the sketches, a central server can reconstruct a global MRF, maintaining the most useful information. Two critical techniques introduced in our VertiMRF are dimension reduction and consistency enforcement, preventing the noise of FM sketch from overwhelming the information of attributes with large domain sizes when building the global MRF. These two techniques allow flexible and inconsistent binning strategies of local private MRF and the data sketching module, which can preserve information to the greatest extent. We conduct extensive experiments on four real-world datasets to evaluate the effectiveness of VertiMRF. End-to-end comparisons demonstrate the superiority of VertiMRF.

The development of cloud and edge computing has enabled the easy access of artificial intelligence (AI) services for massive heterogeneous and resource-constrained devices. Particularly, computation-intensive AI services can be orchestrated and deployed in the cloud or edge according to varying performance and cost requirements. Nonetheless, the improved accessibility of deep learning (DL) model variants and the evolving of computational intelligence paradigms pose great challenges for orchestrating large-scale DL inference services in the cloud-edge continuum. Focusing on cloud or edge-based deployment, existing work on multi-variant service orchestration often has a limited solution space of deployment plans. To address this limitation, we first propose a novel multi-paradigm deployment model (MPDM) for service orchestration, which not only considers the model variants but also allows the co-existence of multiple paradigms for large-scale inference service deployment. The service deployment in the MPDM model is then formulated as a multiobjective optimization problem of seeking a better tradeoff among the system accuracy, service scale, and deployment cost. To solve the multiobjective optimization, we further propose a weighted metric-based constructive heuristic algorithm (WCH), which can efficiently obtain an approximately optimal Pareto frontier. Extensive experimental results have validated the effectiveness and efficiency of WCH, and revealed the impacts of both multi-paradigm deployment and edge-cloud collaborative intelligence (ECCI) paradigm on large-scale DL serving systems.

Impervious surface extraction from remote sensing images has become a promising technology to measure the urban ecological environment and monitor human activity. However, due to the complex characteristics of impervious landscapes, most researches on impervious surface extraction hardly identify the scattered and small objects especially in difficult areas, which severely affect the accuracy of mapping impervious surface. In this work, we propose a channel and spatial attention mechanism (CSAM) to extract impervious surface in difficult areas, which includes a channel attention module to learn the relationship in the multi-channel remote sensing images and a spatial attention module to capture the features of the inconspicuous objects. Experiments with the Sentinel-2 dataset in South Africa demonstrate that CSAM can outperform the state-of-the-art methods.

Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for hidden semantic discovery of text data and serves as a fundamental tool for text analysis in various applications. However, the LDA model as well as the training process of LDA may expose the text information in the training data, thus bringing significant privacy concerns. To address the privacy issue in LDA, we systematically investigate the privacy protection of the main-stream LDA training algorithm based on Collapsed Gibbs Sampling (CGS) and propose several differentially private LDA algorithms for typical training scenarios. In particular, we present the first theoretical analysis on the inherent differential privacy guarantee of CGS based LDA training and further propose a centralized privacy-preserving algorithm (HDP-LDA) that can prevent data inference from the intermediate statistics in the CGS training. Also, we propose a locally private LDA training algorithm (LP-LDA) on crowdsourced data to provide local differential privacy for individual data contributors. Furthermore, we extend LP-LDA to an online version as OLP-LDA to achieve LDA training on locally private mini-batches in a streaming setting. Extensive analysis and experiment results validate both the effectiveness and efficiency of our proposed privacy-preserving LDA training algorithms.

Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for discovery of hidden semantic architecture of text datasets, and plays a fundamental role in many machine learning applications. However, like many other machine learning algorithms, the process of training a LDA model may leak the sensitive information of the training datasets and bring significant privacy risks. To mitigate the privacy issues in LDA, we focus on studying privacy-preserving algorithms of LDA model training in this paper. In particular, we first develop a privacy monitoring algorithm to investigate the privacy guarantee obtained from the inherent randomness of the Collapsed Gibbs Sampling (CGS) process in a typical LDA training algorithm on centralized curated datasets. Then, we further propose a locally private LDA training algorithm on crowdsourced data to provide local differential privacy for individual data contributors. The experimental results on real-world datasets demonstrate the effectiveness of our proposed algorithms.