Crowdsourcing is a promising human-in-the-loop paradigm for processing computer hard tasks by harnessing crowd intelligence. However, canonical crowdsourcing systems mostly need to aggregate/transmit worker data and may lead to privacy-leakage. To tackle this problem, we propose a novel approach, called FedCS (Federated CrowdSourcing), to achieve privacy protection while ensuring quality. FedCS aggregates model parameters from clients to build a shared server model while keeping the training data locally on worker devices to protect data privacy. To mitigate the staleness of stragglers and boost efficiency, we introduce a semi-asynchronous federated crowdsourcing mechanism, where the parameter server performs global aggregation periodically. Moreover, due to the different frequencies of workers participating in asynchronous update, FedCS uses a staleness-aware grouping and weighted aggregation heuristic to balance the training process. To speed up the convergence rate and improve the training accuracy, FedCS deploys adaptive learning step size for worker devices by their participation frequency. We further present a task assignment algorithm to help workers choose worthy and suitable tasks for annotations and to save the budget. Extensive experiments on benchmark datasets and a real-world crowdsourcing project show that FedCS can complete secure crowdsourcing projects with high quality and low budget.

Abstract Microtask crowdsourcing is a form of crowdsourcing in which work is decomposed into a set of small, self-contained tasks, which each can typically be completed in a matter of minutes. Due to the various capabilities and knowledge background of the voluntary participants on the Internet, the answers collected from the crowd are ambiguous and the final answer aggregation is challenging. In this process, the choice of quality control strategies is important for ensuring the quality of the crowdsourcing results. Previous work on answer estimation mainly used expectation–maximization (EM) approach. Unfortunately, EM provides local optimal solutions and the estimated results will be affected by the initial value. In this paper, we extend the local optimal result of EM and propose an approximate global optimal algorithm for answer aggregation of crowdsourcing microtasks with binary answers. Our algorithm is expected to improve the accuracy of real answer estimation through further likelihood maximization. First, three worker quality evaluation models are presented based on static and dynamic methods, respectively, and the local optimal results are obtained based on the maximum likelihood estimation method. Then, a dominance ordering model (DOM) is proposed according to the known worker responses and worker categories for the specified crowdsourcing task to reduce the space of potential task-response sequence while retaining the dominant sequence. Subsequently, a Cut-point neighbor detection algorithm is designed to iteratively search for the approximate global optimal estimation in a reduced space, which works on the proposed dominance ordering model (DOM). We conduct extensive experiments on both simulated and real-world datasets, and the experimental results illustrate that the proposed approach can obtain better estimation results and has higher performance than regular EM-based algorithms.

The rapid growth of data makes it possible for us to study human behavior patterns. Knowing the patterns of human behavior is of great use to help us detect the unusual fraud human behavior. Existing fraud detection methods can be divided into two categories: pattern based and outlier detection based methods. However, because of the sparsity and complex granularity of big data, these methods have high false positive in fraud detection. In this paper, we propose an effective hybrid fraud detection method. We propose SSIsomap which improves isomap to cluster behaviors into behavior classes and propose SimLOF which improves LOF to conduct outlier detection, then we use Dempster-Shafer evidence Theory for combining behavior pattern evidence and outlier evidence, which yields a degree of belief of fraud to the new coming claim. The experiment result shows our method has significantly higher accuracy than exsiting methods in medical insurance fraud detection.