Federated learning enables training a global model from data located at the client nodes, without data sharing and moving client data to a centralized server. Performance of federated learning in a multi-access edge computing (MEC) network suffers from slow convergence due to heterogeneity and stochastic fluctuations in compute power and communication link qualities across clients. We propose a novel coded computing framework, CodedFedL, that injects structured coding redundancy into federated learning for mitigating stragglers and speeding up the training procedure. CodedFedL enables coded computing for non-linear federated learning by efficiently exploiting distributed kernel embedding via random Fourier features that transforms the training task into computationally favourable distributed linear regression. Furthermore, clients generate local parity datasets by coding over their local datasets, while the server combines them to obtain the global parity dataset. Gradient from the global parity dataset compensates for straggling gradients during training, and thereby speeds up convergence. For minimizing the epoch deadline time at the MEC server, we provide a tractable approach for finding the amount of coding redundancy and the number of local data points that a client processes during training, by exploiting the statistical properties of compute as well as communication delays. We also characterize the leakage in data privacy when clients share their local parity datasets with the server. Additionally, we analyze the convergence rate and iteration complexity of CodedFedL under simplifying assumptions, by treating CodedFedL as a stochastic gradient descent algorithm. Finally, for demonstrating gains that CodedFedL can achieve in practice, we conduct numerical experiments using practical network parameters and benchmark datasets, in which CodedFedL speeds up the overall training time by up to 15× in comparison to the benchmark schemes.

In wireless mobile edge computing platforms, such as those supported by vehicular networks, location specific machine learning models can be trained by distributing computations using resources available at the edge. The computational and communication resources at the wireless edge are heterogeneous and unreliable, which can lead to straggler effects that significantly slow down recursive learning tasks, such as gradient descent. In this paper we propose a coded computation framework, which utilizes statistical knowledge of resource heterogeneity to determine optimal encoding and load balancing of training data using Random Linear codes, while avoiding an explicit step for decoding gradients. Results show that the proposed coding framework speeds up the training time for linear regression models, by up to ten times, when compared with repetition coding schemes.

Millimeter wave (mmWave) communication is a key enabler for realizing the data rates, capacity and latency requirements envisioned for emerging 5G systems. This paper addresses the system performance of 73 GHz mmWave access, which is representative of the systems expected to be standardized in the second phase of 3GPP specifications. In particular, we address the performance scalability of mmWave systems and characterize the tradeoffs and challenges involved in improving their coverage and capacity for different cell density regimes. Our results, based on industry standard 3GPP methodology, show that the directional communication enabled through massive MIMO capability of mmWave systems, is key towards addressing the interference limitation on performance scaling, thus illustrating the promise of mmWave communication to provide a scalable solution capable of handling the needs of future ultra-dense networks.

This paper presents a general framework for optimal task reallocation in heterogeneous distributed-computing systems and offers a rigorous analytical model for the stochastic execution time of a workload. The model takes into account the heterogeneity and stochastic nature of the tasks' service and transfer times, servers' failure times, as well as an arbitrary task-reallocation policy. The stochastic service, transfer and failure times are assumed to have general, age-dependent (non-exponential) distributions, resulting in a tandem distributed queuing system with non-Markovian dynamics. Auxiliary age variables are introduced in the analysis to capture the memory associated with the non-Markovian stochastic times, thereby enabling a regenerative age-dependent analytical characterization of the statistics of the execution time of a workload. The model is utilized to devise task reallocation policies that optimize three metrics: the average execution time of a workload, the quality-of-service in executing a workload by a prescribed deadline and the reliability in executing a workload. Implications of the non-exponential event times on these metrics are also studied. Key results are verified experimentally on a distributed-computing testbed.

In distributed computing systems (DCSs) where server nodes can fail permanently with nonzero probability, the system performance can be assessed by means of the service reliability, defined as the probability of serving all the tasks queued in the DCS before all the nodes fail. This paper presents a rigorous probabilistic framework to analytically characterize the service reliability of a DCS in the presence of communication uncertainties and stochastic topological changes due to node deletions. The framework considers a system composed of heterogeneous nodes with stochastic service and failure times and a communication network imposing random tangible delays. The framework also permits arbitrarily specified, distributed load-balancing actions to be taken by the individual nodes in order to improve the service reliability. The presented analysis is based upon a novel use of the concept of stochastic regeneration, which is exploited to derive a system of difference-differential equations characterizing the service reliability. The theory is further utilized to optimize certain load-balancing policies for maximal service reliability; the optimization is carried out by means of an algorithm that scales linearly with the number of nodes in the system. The analytical model is validated using both Monte Carlo simulations and experimental data collected from a DCS testbed.

A regeneration-based approach is undertaken to derive an expression for the expected value of total service time for an arbitrary customer distribution when scheduling is performed in a distributed queuing system. The service times and the transfer delays for customers in the shared communication medium are both considered random. The theory is applied to a distributed wireless-sensor network and the interplay between the total service time and the energy consumption of each sensor is investigated.

A regeneration-theory approach is undertaken to analytically characterize the average overall completion time in a distributed system. The approach considers the heterogeneity in the processing rates of the nodes as well as the randomness in the delays imposed by the communication medium. The optimal one-shot load balancing policy is developed and subsequently extended to develop an autonomous and distributed load-balancing policy that can dynamically reallocate incoming external loads at each node. This adaptive and dynamic load balancing policy is implemented and evaluated in a two-node distributed system. The performance of the proposed dynamic load-balancing policy is compared to that of static policies as well as existing dynamic load-balancing policies by considering the average completion time per task and the system processing rate in the presence of random arrivals of the external loads.

In many distributed computing systems that are prone to either induced or spontaneous node failures, the number of available computing resources is dynamically changing in a random fashion. A load-balancing (LB) policy for such systems should therefore be robust, in terms of workload re-allocation and effectiveness in task completion, with respect to the random absence and re-emergence of nodes as well as random delays in the transfer of workloads among nodes. In this paper two LB policies for such computing environments are presented: The first policy takes an initial LB action to preemptively counteract the consequences of random failure and recovery of nodes. The second policy compensates for the occurrence of node failure dynamically by transferring loads only at the actual failure instants. A probabilistic model, based on the concept of regenerative processes, is presented to assess the overall performance of the system under these policies. Optimal performance of both policies is evaluated using analytical, experimental and simulation-based results. The interplay between node-failure/recovery rates and the mean load-transfer delay are highlighted

A complete analytical solution to one-shot load balancing in a two-node distributed system is presented and verified experimentally. The model takes into account uncertainty in network delays and the variability in the processors speeds. The system consisting of two Transmeta processors connected through a wireless LAN, utilizing an 802.11b access point, is used as the testbed. The statistics of the network delays are estimated experimentally and used in the analytical model to show the effect of the balancing instant and gain on the overall completion times of various workloads. The analytical results are verified by comparing them to actual experiments and Monte-Carlo simulation. The effect of the communication delay on the choice of the gain and the balancing instant is clearly observed. Finally, the model is used to determine the optimal gain and balancing instant for different initial load distributions.