Cloud-fog Federated Learning (FL) is promising for collaborative model training in large-scale edge networks. In cloud-fog FL with constrained communication resources, selecting high-quality client models for aggregation is critical to boost the global model. However, the clients in real world hold multimodal and heterogeneous data, while existing selection strategies rarely consider the imbalance of communication cost and model convergence rates across modalities, thus seriously degrading the efficiency of model aggregation. Moreover, most previous multimodal fusion methods require aligned multimodal samples. However, the data of modality-heterogeneous clients may be isolated and unaligned in FL, so these previous methods cannot be applied to such scenarios, thereby hindering the knowledge fusion across various modalities. To address the above issues, we propose a cloud-fog FL system named COOL, which achievesunimodal aggregationat the fog layer andmultimodal fusionat the cloud layer. First, we propose a Modality-aware Online Client Selection (MOCS) strategy to assist unimodal aggregation. Unlike previous selection strategies, MOCS realizes the dynamic selection budget allocation for various modalities by monitoring the convergence gap of modalities, thus striking the performance balance among modality-heterogeneous clients. Second, to overcome the limitation of previous methods that require aligned multimodal data, we propose a Multimodal Fusion strategy with Feature Synthesis (MFFS). MFFS realizes multimodal fusion with isolated samples via adaptive feature synthesis and cross-modal attention training, thus building a more powerful multimodal predictor while preserving the client data privacy. Finally, experimental results demonstrate that COOL has superior performance compared to the existing algorithms.

As a promising paradigm, federated learning has been applied to multimodal sensing tasks due to its deployment convenience. However, the recent advances in multimodal federated learning emphasize learning a high-quality multimodal model but overlook the model usage requirements of massive unimodal clients. Moreover, the privacy risk in model sharing and client data heterogeneity impact the efficacy of federated learning. In this paper, we propose a novel multimodal federated learning system named MASA. As a departure from existing approaches, MASA simultaneously enhances the model learning efficiency of both multimodal and unimodal clients while ensuring their data privacy. First, we employ a gated cross-modal distillation scheme to achieve performance-aware knowledge transfer across modality-heterogeneous clients. To enhance the system security, MASA integrates a lightweight split-shuffle mechanism to realize the anonymization and encryption of model aggregation. Moreover, to reach personalized collaboration while protecting privacy, MASA features an attention-based spontaneous client clustering mechanism to form client cluster structures securely and distributedly. We evaluate our MASA on four public multimodal datasets for human activity recognition. The results show that our MASA outperforms leading multimodal federated learning methods on the model performance of both multimodal and unimodal clients.

Mobile Crowd Sensing is a prevalent and efficient paradigm for multi-source data collection, where Multi-source Data Fusion (MDF) plays a crucial role in constructing quality data collection services. Current MDF methods often require the majority of participating sensing sources to be credible, or assume that the workers’ credibility is either prior known or easily calculable. However, due to the presence of uncredible environments and the problem of Information Elicitation Without Verification (IEWV), these methods are impractical. It may lead to a vicious cycle where the recruitment of uncredible workers affects the quality of the estimated truth, which can further lead to misjudgments of worker credibility, thereby exacerbating the quality of subsequent recruitment. In this article, a Reliable Multi-source Data Fusion scheme with Cross Validation (RMDF-CV) is proposed to obtain reliable truth for service construction. Specifically, we first introduce the Combinatorial Multi-Armed Bandit (CMAB) model to recruit high-credibility workers by balancing exploration and exploitation. Then, we establish three-stage truth data through three different data sources: Unmanned Aerial Vehicles, credible workers, and Deep Matrix Factorization. Theoretical analyses and extensive simulations confirm the excellent performance of our RMDF-CV scheme.

In the service of Mobile Crowd Sensing (MCS), High-quality Data Collection (HDC), Bilateral Location Privacy Preservation (BLPP), and sensing cost are three pivotal issues. It is widely believed that HDC necessitates the recruitment of workers with high Quality of Service (QoS), which is related to the sensing data capabilities of the recruited workers and the worker-task distances. However, submitting high-quality data demands more resources from the workers, incurring higher costs. Meanwhile, BLPP techniques, aiming to conceal the locations of the workers and tasks, may impede the evaluation of the workers' QoS. Therefore, there is still a lack of a low-cost and BLPP high QoS data collection research. Motivated by this, we propose a Quality-Aware, Low-Cost, and Privacy-Preserving Data Collection Service (QLP-DCS) for MCS. First, we propose a matrix perturbation-based approach to achieve BLPP while preserving the partial order relationship of distances. Subsequently, we employ the Upper Confidence Bound indexes-based reverse auction recruiting workers to balance exploration and exploitation with the low sensing cost. Then, we propose a multi-level truth discovery approach and establish an effective trust verification mechanism. Theoretical analysis and extensive experiments validate the superior performance of our QLP-DCS.

As a promising distributed computing paradigm, federated learning (FL) has been widely applied to industrial manufacturing. To enhance the safety of FL system against growing privacy attacks, secret sharing techniques are introduced to realize secure model aggregation. However, the huge extra transmission in vanilla secret sharing aggravates the inherent communication bottleneck of FL system in industrial cellular networks. To reduce transmission overhead, this article proposes a lightweight FL system based on rotated partial secret sharing (FedRPS), which divides industrial terminals into several subgroups and allows them to alternately share partial local secrets. Our FedRPS ensures the global model to converge at a high rate with incomplete local parameters. Moreover, we theoretically analyze the convergence error of FedRPS, and discuss the impact of unbalanced data in subgroups on model convergence. To further improve global model performance under non-IID settings, we design a multiparty greedy grouping (MPGG) strategy to balance the training data in each subgroup. Finally, we conduct experiments on several datasets, which demonstrate our FedRPS trades for significant improvement on communication with only small loss of model accuracy. In addition, we also simulate the privacy attack scenarios to verify the better defense of FedRPS compared with previous complete secret sharing.

As a promising approach, Clustered Federated Learning (CFL) enables personalized model aggregation for heterogeneous clients. However, facing dynamic and open edge networks, previous CFL rarely considers the impact of dynamic client data on clustering validity, or sensitively identifies low-quality parameters from highly heterogeneous client models. Moreover, the device heterogeneity in each cluster leads to unbalanced model transmission delay, thus reducing the system efficiency. To tackle the above issues, this paper proposes a Robust and Efficient Clustered Federated System (REC-Fed). First, a Hierarchical Attention based Robust Aggregation (HARA) method is designed to realize layer-wise model customization for clients, meanwhile keeping the clustering validity under dynamic client data distribution. In addition, the fine-grained parameter detection in HARA provides a natural advantage to detect low-quality parameters, which improves the robustness of CFL systems. Second, to realize efficient synchronous model transmission, an Adaptive Model Transmission Optimization (AMTO) is proposed to jointly optimize the model compression and bandwidth allocation for heterogenous clients. Finally, we theoretically analyze the convergence of REC-Fed and conduct experiments on several personalization tasks, which demonstrate that our REC-Fed has significant improvement on flexibility, robustness and efficiency.

With the integration of Artificial Intelligence (AI) and Internet of Things (IoT), the Federated Edge Learning (FEL), a promising computing framework is developing. However, there are still unsolved issues on communication efficiency and data security due to the huge models and unreliable transmission links. To address these issues, this paper proposes a novel federated edge learning system, called LightFed, where the edge nodes upload only vital partial local models, and successfully achieve lightweight communication and model aggregation. First, a novel model aggregation method Model Splitting and Splicing (MSS) and a Selective Parameter Transmission (SPT) scheme are proposed. By detecting the updating gradients of local parameters and filtering significant parameters, selective rotated transmission and efficient aggregation of local models are achieved. Second, a Training Filling Model (TFM) is proposed to infer the total data distribution of edge nodes, and train a filling model to mitigate the unbalanced training data without violating the data privacy of individual users. Moreover, a blockchain-powered confusion transmission mechanism is proposed for defending the attacks from external adversaries and protecting the model information. Finally, extensive experimental results demonstrate that our LightFed significantly outperforms the existing FEL systems in terms of communication efficiency and privacy security.

The television ratings provide an effective way to analyze the popularity of TV programs and audiences’ watching habits. Most previous studies have analyzed the ratings from a single perspective. Few efforts have integrated analysis from different perspectives and explored the reasons for changes in ratings. In this paper, we design a visual analysis system called TVseer to analyze audience ratings from three perspectives: TV channels, TV programs, and audiences. The system can help users explore the factors that affect ratings, and assist them in decisions about program productions and schedules. There are six linked views in TVseer: the channel ratings view and program ratings view show ratings change information from the perspective of TV channels and programs respectively; the overlapping program competition view and the same-type program competition view indicate the competitive relationships among programs; the audience transfer view shows how audiences are moving among different channels; the audience group view displays audience groups based on their watching behavior. Besides, we also construct case studies and expert interviews to prove our system is useful and effective.