Multi-modal Federated Learning (MFL) is a distributed machine learning paradigm that enables multiple participants with multi-modal data to collaboratively train a global model for multi-modal tasks without sharing their local data. MFL typically deploys the trained global model as an Embedding-as-a-Service (EaaS), allowing participants to obtain embeddings for downstream tasks. However, it increases the risk of unauthorized copying and leakage of the model. Protecting the ownership of the MFL model while maintaining model performance is challenging. In this paper, we propose the first general model ownership protection framework for MFL, named MFL-Owner. MFL-Owner decouples the watermarking process from the model training process and addresses both ownership verification and traceability, effectively safeguarding the interests of the MFL collective. MFL-Owner leverages the concept of orthogonal transformations by incorporating a linear transformation matrix with orthogonal constraints into the model, achieving high-quality ownership verification and traceability with minimal impact on model performance. To enhance the practicality of the watermark and prevent conflicts among multiple clients during tracing, we propose a trigger dataset selection method based on out-of-distribution data combined with Gaussian noise perturbation. Our experiments on multiple datasets demonstrate that MFL-Owner is effective for model ownership verification and traceability for MFL.

Multi-Party Computation (MPC), as a distributed computing paradigm, is considered to be a potential solution for providing privacy-preserving for applications following the client-server model. However, traditional MPC solutions cannot satisfy the publicly verifiable requirement of the client-server model. In this paper, we propose a blockchain-based verifiable MPC solution using Pedersen's threshold secret sharing and Lifted ElGamal encryption. We first build a data distribution method using Pedersen's threshold secret sharing and symmetric encryption to protect the privacy of inputs while ensuring robustness. Then, we propose a result processing algorithm using Lifted ElGamal encryption to safeguard the privacy of the outputs. Finally, we employ non-interactive zero-knowledge proof and Pedersen commitment to publicly verify the correctness of the encrypted outputs in the smart contract, enabling the detection of malicious parties. Theoretical analysis indicates that the proposed method can publicly verify the correctness of outputs without revealing plain-text inputs and outputs, which satisfy the privacy-preserving requirements of the client-server model. Experimental evaluations have demonstrated that our proposed approach is efficient regarding computation overhead, communication overhead, and response time in the output verification phase while achieving stronger privacy and robustness.

As a collaborative framework designed to safeguard privacy,Federated Learning(FL) seeks to protect participants’ data throughout the training process. However, the framework still faces security risks from poisoning attacks, arising from the unmonitored process of client-side model updates. Most existing solutions address scenarios where less than half of clients are malicious, i.e., which leaves a significant challenge to defend against attacks when more than half of participants are malicious. In this paper, we propose a FL scheme, named FedAMM, that resists backdoor attacks across various data distributions and malicious client ratios. We develop a novel backdoor defense mechanism to filter out malicious models, aiming to reduce the performance degradation of the model. The proposed scheme addresses the challenge of distance measurement in high-dimensional spaces by applyingPrincipal Component Analysis(PCA) to improve clustering effectiveness. We borrow the idea of critical parameter analysis to enhance discriminative ability in non-iid data scenarios, via assessing the benign or malicious nature of models by comparing the similarity of critical parameters across different models. Finally, our scheme employs a hierarchical noise perturbation to improve the backdoor mitigation rate, effectively eliminating the backdoor and reducing the adverse effects of noise on task accuracy. Through evaluations conducted on multiple datasets, we demonstrate that the proposed scheme achieves superior backdoor defense across diverse client data distributions and different ratios of malicious participants. With 80% malicious clients, FedAMM achieves low backdoor attack success rates of 1.14%, 0.28%, and 5.53% on MNIST, FMNIST, and CIFAR-10, respectively, demonstrating enhanced robustness of FL against backdoor attacks.