Hourglass: Enabling Efficient Split Federated Learning with Data Parallelism

In an edge-assisted federated learning (FL) system, edge servers aggregate the local models from the clients within their coverage areas to produce intermediate models for the production of the global model. This significantly reduces the communication overhead incurred during the FL process. To accelerate model convergence, FedEdge, the state-of-the-art edge-assisted FL system, trains clients' models in local federations when they wait for the global model in each training round. However, our investigation reveals that it drives the global model towards clients with excessive local training, causing model drifts that undermine model performance for other clients. To tackle this problem, this paper presents Maverick, a new edge-assisted FL system that mitigates model drifts by training personalized local models for clients through contrastive local training. It introduces a model-contrastive loss to facilitate personalized local federated training by driving clients' local models away from the global model and close to their corresponding intermediate models. In addition, Maverick includes anomalous models in contrastive local training as negative samples to accelerate the convergence of clients' local models. Extensive experiments are conducted on three widely-used models trained on three datasets to comprehensively evaluate the performance of Maverick. Compared to state-of-the-art edge-assisted FL systems, Maverick accelerates model convergence by up to 16.2x and improves model accuracy by up to 12.7%.

Training machine learning (ML) models on mobile and Web-of-Things (WoT) has been widely acknowledged and employed as a promising solution to privacy-preserving ML. However, these end-devices often suffer from constrained resources and fail to accommodate increasingly large ML models that crave great computation power. Offloading ML models partially to the cloud for training strikes a trade-off between privacy preservation and resource requirements. However, device-cloud training creates communication overheads that delay model training tremendously. This paper presents EdgeMove, the first device-edge training scheme that enables fast pipelined model training across edge devices and edge servers. It employs probing-based mechanisms to tackle the new challenges raised by device-edge training. Before training begins, it probes nearby edge servers’ training performance and bootstraps model training by constructing a training pipeline with an approximate model partitioning. During the training process, EdgeMove accommodates user mobility and system dynamics by probing nearby edge servers’ training performance adaptively and adapting the training pipeline proactively. Extensive experiments are conducted with two popular DNN models trained on four datasets for three ML tasks. The results demonstrate that EdgeMove achieves a 1.3 × -2.1 × speedup over the state-of-the-art scheme.

Machine learning (ML) is powering a rapidly-increasing number of web applications. As a crucial part of 5G, edge computing facilitates edge artificial intelligence (AI) by ML model training and inference at the network edge on edge servers. Compared with centralized cloud AI, edge AI enables low-latency ML inference which is critical to many delay-sensitive web applications, e.g., web AR/VR, web gaming and Web-of-Things applications. Existing studies of edge AI focused on resource and performance optimization in training and inference, leveraging edge computing merely as a tool to accelerate training and inference processes. However, the unique ability of edge computing to process data with context awareness, a powerful feature for building the web-of-things for smart cities, has not been properly explored. In this paper, we propose a novel framework named Pyramid that unleashes the potential of edge AI by facilitating homogeneous and heterogeneous hierarchical ML inferences. We motivate and present Pyramid with traffic prediction as an illustrative example, and evaluate it through extensive experiments conducted on two real-world datasets. The results demonstrate the superior performance of Pyramid neural networks in hierarchical traffic prediction and weather analysis.