Federated learning (FL) commonly assumes that the server or some clients have labeled data, which is often impractical due to annotation costs and privacy concerns. Addressing this problem, we focus on a source-free domain adaptation task, where (1) the server holds a pre-trained model on labeled source domain data, (2) clients possess only unlabeled data from various target domains, and (3) the server and clients cannot access the source data in the adaptation phase. This task is known as Federated source-Free Domain Adaptation (FFREEDA). Specifically, we focus on classification tasks, while the previous work solely studies semantic segmentation. Our contribution is the novel Federated learning with Weighted Cluster Aggregation (FedWCA) method, designed to mitigate both domain shifts and privacy concerns with only unlabeled data. Fed-WCA comprises three phases: private and parameter-free clustering of clients to obtain domain-specific global models on the server, weighted aggregation of the global models for the clustered clients, and local domain adaptation with pseudo-labeling. Experimental results show that Fed-WCA surpasses several existing methods and baselines in FFREEDA, establishing its effectiveness and practicality.

We propose a Trojan horse-type attribute inference attack (AlA) against the gradient boosting decision trees (GBDT) in the federated learning setting. Our Trojan AlA consists of a Trojan tree creation and an attribute inference. Both algorithms leverage the characteristics of the federated learning protocol for the GBDT training. First, the adversary creates a decision tree, a Trojan tree, that isolates a target data record from other data records. The adversary sends the Trojan tree to the server through the federated learning protocol at their round. Trojan tree forces the victim's tree to “memorize” a target attribute value of target data record that the adversary wants to know. The adversary can recover the target attribute value by observing the tree submitted by the victim if the victim uses the target data record for training the tree. For the regression task, we derive sufficient conditions for a successful attack. According to our theorem, if the target data record is distinct in the victim's dataset, the proposed attack is always successful. Experiments on multiple datasets and settings show results that align with the above theoretical analysis. Even if some conditions for theoretical analysis are relaxed, the proposed attack outperforms baseline attacks. To the best of our knowledge, this is the first study of an attribute inference attack against the GBDT in the federated learning setting.

The need for studies on the privacy risks of machine learning models has been increasing as using sensitive data in training them has become prevalent in real-world applications. Decision tree and random forest models have been used for data mining for several decades. Yet, there are not enough studies on the privacy risks of these models. In this paper, we present two novel attribute inference attacks, i.e., CTAIA and RFAIA, for the decision tree and random forest classifiers, respectively. CTAIA is a black-box attack, and RFAIA is a white-box attack. Our attacks utilize the confidence score information from the model outputs in a novel way. Specifically, our attacks use the zero values in confidence scores of the decision tree classifiers. A zero confidence score for a specific class means that there is no training data sample for that class. This fact, the embedding of the information about the number of the train data samples in the confidence score, is used to exclude the candidate values of the target attribute. We define the train data samples, which an attacker of an attribute inference attack can infer the values of the sensitive attribute with 100% confidence, as "high-risk" data records. For the decision tree classifiers or classification trees, CTAIA selects some data records and infers the values of the target attribute of them with 100% accuracy, making them "high-risk" data records. Similarly, for the random forest classifiers, RFAIA selects some data records. Depending on whether the bootstrap sampling is used in training the classifiers or not, the RFAIA has 100% or near 100% attack accuracy for the selected data records. Therefore, in the case of random forest classifiers, for simplicity, we loosen the above-mentioned definition of the "high-risk" data records and call the data records selected by RFAIA the "high-risk" data in this paper. We have experimentally shown the effectiveness of our attack using three public datasets.

Heterogeneous unsupervised domain adaptation (HUDA) is the most challenging domain adaptation setting where the feature spaces of source and target domains are heterogeneous, and the target domain has only unlabeled data. Existing HUDA methods assume that both positive and negative examples are available in the source domain, which may not be satisfied in some real applications. This paper addresses a new challenging setting called positive and unlabeled heterogeneous unsupervised domain adaptation (PU-HUDA), a HUDA setting where the source domain only has positives. PU-HUDA can also be viewed as an extension of PU learning where the positive and unlabeled examples are sampled from different domains. A naive combination of existing HUDA and PU learning methods is ineffective in PU-HUDA due to the gap in label distribution between the source and target domains. To overcome this issue, we propose a novel method, predictive adversarial domain adaptation (PADA), which can predict likely positive examples from the unlabeled target data and simultaneously align the feature spaces to reduce the distribution divergence between the whole source data and the likely positive target data. PADA achieves this by a unified adversarial training framework for learning a classifier to predict positive examples and a feature transformer to transform the target feature space to that of the source. Specifically, they are both trained to fool a common discriminator that determines whether the likely positive examples are from the target or source domain. We experimentally show that PADA outperforms several baseline methods, such as the naive combination of HUDA and PU learning.

Due to the regulations to protect user data privacy and concerns about trade secrets, industrial organizations do not share user data with others. Federated learning makes it possible for multiple organizations to train a global model without revealing their data. Since, in real life, data distributions differ between organizations, it is necessary to personalize the model to have better performance for the data of a single participant. In this paper, we present the first personalized federated learning method for Gradient Boosting Decision Trees (GBDT) focusing on classification tasks, i.e., Personalized Gradient Boosting Forest (pGBF). Our method extends the existing federated learning method, Gradient Boosting Forest (GBF). Our experi-ments on three public datasets show that pGBF has better or similar performance to the existing methods, GBDT and GBF, in non-IID settings. Specifically, we find that our method has higher performance than GBDT when the data of the personalization target participant is small enough for GBDT model performance to be low. Moreover, pGBF has better performance than GBF when the data distributions among the participants are non-IID.

Federated learning (FL) is a decentralized machine learning technique that enables multiple clients to collaboratively train models without requiring clients to reveal their raw data to each other. Although traditional FL trains a single global model with average performance among clients, statistical data heterogeneity across clients has resulted in the development of personalized FL (PFL), which trains personalized models with good performance on each client's data. A key challenge with PFL is how to facilitate clients with similar data to collaborate more in a situation where each client has data from complex distribution and cannot determine one another's distribution. In this paper, we propose a new PFL method (pFedMB) using multi-branch architecture, which achieves personalization by splitting each layer of a neural network into multiple branches and assigning client-specific weights to each branch. We also design an aggregation method to improve the communication efficiency and the model performance, with which each branch is globally updated with weighted averaging by client-specific weights assigned to the branch. pFedMB is simple but effective in facilitating each client to share knowledge with similar clients by adjusting the weights assigned to each branch. We experimentally show that pFedMB performs better than the state-of-the-art PFL methods using the CIFAR10 and CIFAR100 datasets.

Federated learning is a promising machine learning technique that enables multiple clients to collaboratively build a model without revealing the raw data to each other. Among various types of federated learning methods, horizontal federated learning (HFL) is the best-studied category and handles homogeneous feature spaces. However, in the case of heterogeneous feature spaces, HFL uses only common features and leaves client-specific features unutilized. In this paper, we propose a HFL method using neural networks named continual horizontal federated learning (CHFL), a continual learning approach to improve the performance of HFL by taking advantage of unique features of each client. CHFL splits the network into two columns corresponding to common features and unique features, respectively. It jointly trains the first column by using common features through vanilla HFL and locally trains the second column by using unique features and leveraging the knowledge of the first one via lateral connections without interfering with the federated training of it. We conduct experiments on various real world datasets and show that CHFL greatly outperforms vanilla HFL that only uses common features and local learning that uses all features that each client has.

Abstract A membership inference attack (MIA) poses privacy risks for the training data of a machine learning model. With an MIA, an attacker guesses if the target data are a member of the training dataset. The state-of-the-art defense against MIAs, distillation for membership privacy (DMP), requires not only private data for protection but a large amount of unlabeled public data. However, in certain privacy-sensitive domains, such as medicine and finance, the availability of public data is not guaranteed. Moreover, a trivial method for generating public data by using generative adversarial networks significantly decreases the model accuracy, as reported by the authors of DMP. To overcome this problem, we propose a novel defense against MIAs that uses knowledge distillation without requiring public data. Our experiments show that the privacy protection and accuracy of our defense are comparable to those of DMP for the benchmark tabular datasets used in MIA research, Purchase100 and Texas100, and our defense has a much better privacy-utility trade-off than those of the existing defenses that also do not use public data for the image dataset CIFAR10.

An encryption scheme is Key Dependent Message (KDM) secure if it is secure even against an attacker who has access to encryptions of messages which depend on the secret key. Recent studies have revealed that this strong security notion is important both theoretically and practically. In this paper we review the defnition, and survey recent results and applications of KDM security.