In federated learning (FL), it is generally assumed that all data are placed at clients in the beginning of machine learning (ML) optimization (i.e., offline learning). However, in many real-world applications, ML tasks are expected to proceed in an online fashion, wherein data samples are generated as a function of time and each client has to predict a label (or make a decision) upon receiving an incoming data. To this end, online FL (OFL) has been introduced, which aims at learning a sequence of global models from distributed streaming data such that a cumulative regret is minimized. In this framework, the vanilla method (named FedOGD) by combining online gradient descent and model averaging, which is regarded as the counterpart of FedSGD in the standard FL. Despite its asymptotic optimality, FedOGD suffers from high communication costs. In this paper, we present a communication-efficient OFL method by means of intermittent transmission (enabled by client subsampling and periodic transmission) and gradient quantization. For the first time, we derive the regret bound which can reflect the impact of data-heterogeneity and communication-efficient techniques. Based on our tighter analysis, we optimize the key parameters of OFedIQ such as sampling rate, transmission period, and quantization bits. Also, we prove that the optimized OFedIQ asymptotically achieves the performance of FedOGD while reducing the communication costs by 99%. Via experiments with real datasets, we validate the effectiveness of our algorithm on various online ML tasks.

We investigate an online federated learning (in short, OFL), in which many edge nodes receive their own time-series data and train a sequence of global models under the orchestration of a central server while keeping data localized. In this framework, we propose a communication efficient federated quantized online gradient descent (FedQOGD) by means of a stochastic quantization and partial node participation. We theoretically prove that FedQOGD over T time slots can achieve an optimal sublinear regret bound ${\mathcal{O}}(\sqrt T )$ for any quantization level (e.g., 1-level quantization), even when every node can participate in a learning process sporadically. Our analysis reveals that FedQOGD yields the same asymptotic performance as the centralized counterpart (i.e., all local data are gathered at the central server) while having a low-communication overhead and preserving an edge-node privacy. Finally, we verify the effectiveness of our algorithm via experiments with a real-world MNIST dataset on online classification task.