Federated learning (FL) with differential privacy (DP) provides a framework for collaborative machine learning, enabling clients to train a shared model while adhering to strict privacy constraints. The framework allows each client to have an individual privacy guarantee, e.g., by adding different amounts of noise to each client's model updates. One underlying assumption is that all clients spend their privacy budgets uniformly over time (learning rounds). However, it has been shown in the literature that learning in early rounds typically focuses on more coarse-grained features that can be learned at lower signal-to-noise ratios while later rounds learn fine-grained features that benefit from higher signal-to-noise ratios. Building on this intuition, we propose a time-adaptive DP-FL framework that expends the privacy budget non-uniformly across both time and clients. Our framework enables each client to save privacy budget in early rounds so as to be able to spend more in later rounds when additional accuracy is beneficial in learning more fine-grained features. We theoretically prove utility improvements in the case that clients with stricter privacy budgets spend budgets unevenly across rounds, compared to clients with more relaxed budgets, who have sufficient budgets to distribute their spend more evenly. Our practical experiments on standard benchmark datasets support our theoretical results and show that, in practice, our algorithms improve the privacy-utility trade-offs compared to baseline schemes.

Federated Learning (FL) is a privacy-centric frame-work for distributed learning where devices collaborate to develop a shared global model while keeping their raw data local. Since workers may naturally form groups based on common objectives and privacy rules, we are motivated to extend FL to such settings. As workers can contribute to multiple groups, complexities arise in understanding privacy leakage and in adhering to privacy policies. In this paper, we propose differ-ential private overlapping grouped learning (DP-OGL), which shares learning across groups through common workers. We derive formal privacy guarantees between every pair of workers under the honest-but-curious threat model with multiple group memberships. Our experiments show that DP-OGL improves privacy-utility trade-offs compared to a baseline FL system.

Coded computing was recently introduced to mitigate the effect of stragglers on distributed computing systems. This paper combines ideas of approximate and coded computing to further accelerate computation. We propose approximated coded distributed computing (ACDC) that realizes a tradeoff between accuracy and speed, allowing the distributed computing system to produce approximations that increase in accuracy over time. If a sufficient number of compute nodes finish their tasks, ACDC exactly recovers the desired computation. We theoretically provide design guidelines for ACDC, and numerically show its benefits over previous methods.

Elasticity is offered by cloud service providers to exploit under-utilized computing resources. The low-cost elastic nodes can leave and join any time during the computation cycle. The possibility of elastic events occurring together with the problem of slow nodes, referred to as stragglers, increases the uncertainty of the system, leading to computation delay. Recent results have shown that coded computing can be used to reduce the negative effect of elasticity and stragglers. In this paper, we propose two hierarchical coded elastic computing schemes that can further speed up the system by exploiting stragglers and effectively allocating tasks among available nodes. In our simulations, our scheme realizes 45% improvement in average finishing time compared to the state-of-the-art coded elastic computing scheme.

In distributed computing systems slow working nodes, known as stragglers, can greatly extend finishing times. Coded computing is a technique that enables straggler-resistant computation. Most coded computing techniques presented to date provide robustness by ensuring that the time to finish depends only on a set of the fastest nodes. However, while stragglers do compute less work than non-stragglers, in real-world commercial cloud computing systems (e.g., Amazon’s Elastic Compute Cloud (EC2)) the distinction is often a soft one. In this paper, we develophierarchicalcoded computing that exploits the work completed by all nodes, both fast and slow, automatically integrating the potential contribution of each. We first present a conceptual framework to represent the division of work amongst nodes in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. We then develop three methods of hierarchical coded computing that we termbit-interleavedcoded computation (BICC),multilevelcoded computation (MLCC), andhybridhierarchical coded computation (HHCC). In this paradigm, each worker is tasked with completing a sequence (a hierarchy) of ordered subtasks. The sequence of subtasks, and the complexity of each, is designed so that partial work completed by stragglers can be used, rather than ignored. We note that our methods can be used in conjunction with any coded computing method. We illustrate this by showing how we can use our methods to accelerate all previously developed coded computing techniques by enabling them to exploit stragglers. Under a widely studied statistical model of completion time, our approach realizes a 66% improvement in the expected finishing time. On Amazon EC2, the gain was 27% when stragglers are simulated.

In cloud computing systems slow processing nodes, often referred to as “stragglers”, can significantly extend the computation time. Recent results have shown that error correction coding can be used to reduce the effect of stragglers. In this work we introduce a scheme that, in addition to using error correction to distribute mixed jobs across nodes, is also able to exploit the work completed by all nodes, including stragglers. We first consider vector-matrix multiplication and apply maximum distance separable (MDS) codes to small blocks of sub-matrices. The worker nodes process blocks sequentially, working block-by-block, transmitting partial per-block results to the master as they are completed. Sub-blocking allows a more continuous completion process, which thereby allows us to exploit the work of a much broader spectrum of processors and reduces computation time. We then apply this technique to matrix-matrix multiplication using product code. In this case, we show that the order of computing sub-tasks is a new degree of design freedom that can be exploited to reduce computation time further. We propose a novel approach to analyze the finishing time, which is different from typical order statistics. Simulation results show that the expected computation time decreases by a factor of at least two in compared to previous methods.

This letter considers the problem of casual heart rate tracking during intensive physical exercise using simultaneous 2 channel photoplethysmographic (PPG) and 3 dimensional (3D) acceleration signals recorded from wrist. This is a challenging problem because the PPG signals recorded from wrist during exercise are contaminated by strong Motion Artifacts (MAs). In this work, a novel algorithm is proposed which consists of two main steps of MA Cancellation and Spectral Analysis. The MA cancellation step cleanses the MA-contaminated PPG signals utilizing the acceleration data and the spectral analysis step estimates a higher resolution spectrum of the signal and selects the spectral peaks corresponding to HR. Experimental results on datasets recorded from 12 subjects during fast running at the peak speed of 15 km/hour showed that the proposed algorithm achieves an average absolute error of 1.25 beat per minute (BPM). These experimental results also confirm that the proposed algorithm keeps high estimation accuracies even in strong MA conditions.