In Federated Learning (FL) with over-the-air aggregation, the quality of the signal received at the server critically depends on the receive scaling factors. While a larger scaling factor can reduce the effective noise power and improve training performance, it also compromises the privacy of devices by reducing uncertainty. In this work, we aim to adaptively design the receive scaling factors across training rounds to balance the trade-off between training convergence and privacy in an FL system under dynamic channel conditions. We formulate a stochastic optimization problem that minimizes the overall Rényi differential privacy (RDP) leakage over the entire training process, subject to a long-term constraint that ensures convergence of the global loss function. Our problem depends on unknown future information, and we observe that standard Lyapunov optimization is not applicable. Thus, we develop a new online algorithm, termed AdaScale, based on a sequence of novel per-round problems that can be solved efficiently. We further derive upper bounds on the dynamic regret and constraint violation of AdaSacle, establishing that it achieves diminishing dynamic regret in terms of time-averaged RDP leakage while ensuring convergence of FL training to a stationary point. Numerical experiments on canonical classification tasks show that our approach effectively reduces RDP and DP leakages compared with state-of-the-art benchmarks without compromising learning performance.

This paper studies power-efficient uplink transmission design for federated learning (FL) that employs over-the-air analog aggregation and multi-antenna beamforming at the server. We jointly optimize device transmit weights and receive beamforming at each FL communication round to minimize the total device transmit power while ensuring convergence in FL training. Through our convergence analysis, we establish sufficient conditions on the aggregation error to guarantee FL training convergence. Utilizing these conditions, we reformulate the power minimization problem into a unique bi-convex structure that contains a transmit beamforming optimization subproblem and a receive beamforming feasibility subproblem. Despite this unconventional structure, we propose a novel alternating optimization (AO) approach that guarantees monotonic decrease of the objective value, to allow convergence to a partial optimum. We further consider imperfect channel state information (CSI), which requires accounting for the channel estimation errors in the power minimization problem and FL convergence analysis. We propose a CSI-error-aware joint beamforming algorithm, which can substantially outperform one that does not account for channel estimation errors. Simulation with canonical classification datasets demonstrates that our proposed methods achieve significant power reduction compared to existing benchmarks across a wide range of parameter settings, while attaining the same target accuracy under the same convergence rate.

This work addresses the trade-off between convergence and the overall delay in heterogeneous distributed learning systems, where the devices encounter diverse and dynamic communication conditions. We propose to apply adaptive sparsification across the devices and over iterations, formulating an optimization problem to minimize the overall delay while ensuring a specified level of convergence. The resultant stochastic optimization problem cannot be handled by conventional Lyapunov optimization techniques due to the dependency of the per-iteration objective function on the previous iterations. To overcome this challenge, we propose AdaSparse, an online algorithm with a novel per-slot problem that can be solved optimally by searching over a finite discrete space. We further introduce a low-complexity approximation of AdaSparse, termed LC-AdaSparse, which features linear computational complexity and diminishing approximation error. We show that AdaSparse offers strong performance guarantees, simultaneously achieving sub-linear dynamic regret in terms of delay and the optimal rate in terms of convergence. Numerical experiments on classification tasks using standard datasets and various models demonstrate that our approach effectively reduces the communication delay compared with existing benchmarks, to achieve the same levels of learning accuracy.

Differential privacy (DP) limits the impact of individual training data samples by bounding their gradient norms through clipping. Conventional clipping operations assign unequal scaling factors to sample gradients with different norms, leading to a direction mismatch between the true batch gradient and the aggregation of the clipped gradients. Applying a smaller but identical scaling factor to all sample gradients alleviates this direction mismatch; however, it intensifies the magnitude mismatch by excessively reducing the aggregation norm. This work proposes a novel clipping method, termed adaptive sigmoid (AdaSig), which uses a sigmoid function with an adjustable saturation slope to clip the sample gradients. The slope is adaptively adjusted during the training process to balance the trade-off between direction mismatch and magnitude mismatch, as the statistics of sample gradients evolve over the training iterations. Despite AdaSig’s adaptive nature, our convergence analysis demonstrates that differentially private stochastic gradient descent (DP-SGD) with AdaSig clipping retains the best-known convergence rate under non-convex loss functions. Evaluating AdaSig on sentence and image classification tasks across different datasets shows that it consistently improves learning performance compared with established clipping methods.

Combining over-the-air uplink transmission and multi-antenna beamforming can improve the efficiency of federated learning (FL). However, to mitigate the significant aggregation error due to communication noise and signal distortion, pre-processing of device signals and post-processing at the server are required. In this paper, we study the optimization of receiver beamforming and device transmit weights in over-the-air FL, to minimize the total transmit power in each communication round while guaranteeing the convergence of FL. We establish sufficient convergence conditions based on the analysis of gradient descent with error and formulate a power minimization problem. An alternating optimization approach is then employed to decompose the problem into tractable subproblems, and efficient solutions are developed for these subproblems. Our proposed method is evaluated through simulation on standard image classification tasks, demonstrating its effectiveness in achieving substantial reductions in transmit power compared with existing alternatives.