This paper presents Sheaf-DMFL, a novel decentralized multimodal federated learning framework leveraging sheaf theory to enhance collaborative learning among clients with diverse modalities. By framing the multimodal federated learning problem as multitask learning, Sheaf-DMFL leverages learnable restriction maps to capture relationships between clients’ models. Specifically, each client has a set of local feature encoders for its different modalities, whose outputs are concatenated before passing through a task-specific layer. Encoders corresponding to the same modality are shared among clients, while the intrinsic correlation among their task-specific layers is captured by using the sheaf structure. Numerical experiments in a mmWave beamforming prediction scenario show that the proposed algorithm surpasses baseline methods, delivering improved convergence rates and test accuracy.

Federated averaging (FedAvg) is a well-recognized framework for distributed learning that efficiently manages communication. Several algorithms have emerged to enhance the communication efficiency of FedAvg and its variations. Some of these algorithms focus on reducing the number of communication rounds by allowing clients to skip frequent interactions with the parameter server. In this work, our primary concern is the overall energy consumption during model training in federated learning. We challenge the conventional notion that reducing the frequency of communication leads to energy savings and present evidence that for nonindependent and nonidentical (non-IID) data distribution, increasing the frequency of communication can, in fact, result in greater energy conservation. Our contribution comprises two key aspects: first, we introduce a quantized version of the recently proposed algorithm called federated primal-dual (FedPD) (Zhang et al., 2021), which we refer to as quantized FedPD (QFedPD). Importantly, we substantiate the convergence guarantees for QFedPD. Second, we explore the tradeoff between quantization and communication skipping in the proposed approach. Our analysis demonstrates that applying quantization without skipping communication, using QFedPD, yields the most significant energy-saving benefits for non-IID data distribution. Intriguingly, when dealing with non-IID data distribution, the preferred strategy is to maximize energy efficiency, allowing all clients to transmit at every iteration while quantizing their updates.

Federated learning (FL) offers a promising approach for collaborative machine learning (ML) across distributed devices. However, its adoption is hindered by the complexity of building reliable communication architectures and the need for expertise in both ML and network programming. This article presents a comprehensive solution that simplifies the orchestration of FL tasks while integrating intent-based automation. A user-friendly web application is developed supporting the federated averaging (FedAvg) algorithm, enabling users to configure parameters through an intuitive interface. The backend solution efficiently manages communication between the parameter server and edge nodes. Model compression and scheduling algorithms are implemented to optimize FL performance. Additionally, intent-based automation in FL is explored using a fine-tuned Language Model (LLM) trained on a tailored dataset, enabling users to perform FL tasks through high-level prompts. It is shown that the LLM-based automated solution achieves comparable test accuracy to the standard web-based solution while reducing transferred bytes by up to 64% and CPU time by up to 46% for FL tasks. Furthermore, neural architecture search (NAS) and hyperparameter optimization (HPO) are leveraged using the LLM to enhance performance, resulting in a 10%–20% improvement in test accuracy for the conducted FL tasks.

Federated learning is a machine learning approach where multiple devices collaboratively learn with the help of a parameter server by sharing only their local updates. While gradient-based optimization techniques are widely adopted in this domain, the curvature information that second-order methods exhibit is crucial to guide and speed up the convergence. This paper introduces a scalable second-order method, allowing the adoption of curvature information in federated large models. Our method, coined Fed-Sophia, combines a weighted moving average of the gradient with a clipping operation to find the descent direction. In addition to that, a lightweight estimation of the Hessian's diagonal is used to incorporate the curvature information. Numerical evaluation shows the superiority, robustness, and scalability of the proposed Fed-Sophia scheme compared to first and second-order baselines.

In this paper, we consider a decentralized consensus optimization problem defined over a network of inter-connected devices that collaboratively solve the problem using only local data and information exchange with their neighbours. Despite their fast convergence, Newton-type methods require sending Hessian information between devices, making them communication inefficient while violating the devices' privacy. By formulating the Newton direction learning problem as a sum of separable functions subjected to a consensus constraint, our proposed approach learns an inexact Newton direction alongside the global model using the proximal primal-dual (Prox-PDA) algorithm. Our algorithm, coined DIN, avoids sharing Hessian information between devices since each device shares a model-sized vector, concealing the first- and second-order information, reducing the network's burden and improving communication and energy efficiencies. Numerical simulations corroborate that DIN exhibits higher communication efficiency in terms of communication rounds while consuming less communication and computation energy compared to existing second-order decentralized baselines.

In this paper, we propose a decentralized Newton-type approach to solve the problem of decentralized federated learning (FL). Notably, our proposed algorithm leverages the fast convergence of the second-order methods while avoid sending the hessian matrix at each iteration. Therefore, the proposed approach significantly reduces the communication cost and preserves the privacy. Specifically, we alternate between two problems. The inner problem approximates the inverse Hessian-gradient product which is formulated as a quadratic optimization problem and approximately solved in a decentralized manner using one step of the group alternating direction method of multipliers (GADMM) method. The outer problem learns the model, which is solved by performing one decentralized Newton step at every iteration. Moreover, to reduce the communication-overhead per iteration, a quantized version (leveraging stochastic quantization) is also proposed. Simulation results illustrate that our algorithm outperforms the baselines of GADMM, Q-GADMM, Newton tracking, and Decentralized SGD, and provides energy and communication-efficient solutions for bandwidth-limited systems under different SNR regimes.

Newton-type methods are popular in federated learning due to their fast convergence. Still, they suffer from two main issues, namely: low communication efficiency and low privacy due to the requirement of sending Hessian information from clients to parameter server (PS). In this work, we introduced a novel framework called FedNew in which there is no need to transmit Hessian information from clients to PS, hence resolving the bottleneck to improve communication efficiency. In addition, FedNew hides the gradient information and results in a privacy-preserving approach compared to the existing state-of-the-art. The core novel idea in FedNew is to introduce a two level framework, and alternate between updating the inverse Hessian-gradient product using only one alternating direction method of multipliers (ADMM) step and then performing the global model update using Newton’s method. Though only one ADMM pass is used to approximate the inverse Hessian-gradient product at each iteration, we develop a novel theoretical approach to show the converging behavior of FedNew for convex problems. Additionally, a significant reduction in communication overhead is achieved by utilizing stochastic quantization. Numerical results using real datasets show the superiority of FedNew compared to existing methods in terms of communication costs.

In this paper, we propose a communication-efficient alternating direction method of multipliers (ADMM)-based algorithm for solving a distributed learning problem in the stochastic non-convex setting. Our approach runs a few stochastic gradient descent (SGD) steps to solve the local problem at each worker instead of finding the exact/approximate solution as proposed by existing ADMM-based works. By doing so, the proposed framework strikes a good balance between the computation and communication costs. Extensive simulation results show that our algorithm significantly outperforms existing stochastic ADMM in terms of communication-efficiency, notably in the presence of non-independent and identically distributed (non-IID) data.

In this paper, we propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM (CQ-GGADMM), leverages the worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM), and pushes the frontier in communication efficiency by extending its applicability to generalized network topologies, while incorporating link censoring for negligible updates after quantization. We theoretically prove that CQ-GGADMM achieves the linear convergence rate when the local objective functions are strongly convex under some mild assumptions. Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed, compared to the censored decentralized ADMM, and the worker grouping method of GADMM.

In this paper, we propose an energy-efficient federated meta-learning framework. The objective is to enable learning a meta-model that can be fine-tuned to a new task with a few number of samples in a distributed setting and at low computation and communication energy consumption. We assume that each task is owned by a separate agent, so a limited number of tasks is used to train a meta-model. Assuming each task was trained offline on the agent's local data, we propose a lightweight algorithm that starts from the local models of all agents, and in a backward manner using projected stochastic gradient ascent (P-SGA) finds a meta-model. The proposed method avoids complex computations such as computing hessian, double looping, and matrix inversion, while achieving high performance at significantly less energy consumption compared to the state-of-the-art methods such as MAML and iMAML on conducted experiments for sinusoid regression and image classification tasks.

In this article, we study the problem of robust reconfigurable intelligent surface (RIS)-aided downlink communication over heterogeneous RIS types in the supervised learning setting. By modeling downlink communication over heterogeneous RIS designs as different workers that learn how to optimize phase configurations in a distributed manner, we solve this distributed learning problem using a distributionally robust formulation in a communication-efficient manner, while establishing its rate of convergence. By doing so, we ensure that the global model performance of the worst-case worker is close to the performance of other workers. Simulation results show that our proposed algorithm requires fewer communication rounds (about 50% lesser) to achieve the same worst-case distribution test accuracy compared to competitive baselines.

Split-learning (SL) has recently gained popularity due to its inherent privacy-preserving capabilities and ability to enable collaborative inference for devices with limited computational power. Standard SL algorithms assume an ideal underlying digital communication system and ignore the problem of scarce communication bandwidth. However, for a large number of agents, limited bandwidth resources, and time-varying commu-nication channels, the communication bandwidth can become the bottleneck. To address this challenge, in this work, we propose a novel SL framework to solve the remote inference problem that introduces an additional layer at the agent side and constrains the choices of the weights and the biases to ensure over the air aggregation. Hence, the proposed approach maintains constant communication cost with respect to the number of agents enabling remote inference under limited bandwidth. Numerical results show that our proposed algorithm significantly outper-forms the digital implementation in terms of communication-efficiency” especially as the number of agents grows large.

Today’s intelligent applications can achieve high performance accuracy using machine learning (ML) techniques, such as deep neural networks (DNNs). Traditionally, in a remote DNN inference problem, an edge device transmits raw data to a remote node that performs the inference task. However, this may incur high transmission energy costs and puts data privacy at risk. In this paper, we propose a technique to reduce the total energy bill at the edge device by utilizing model compression and time-varying model split between the edge and remote nodes. The time-varying representation accounts for time-varying channels and can significantly reduce the total energy at the edge device while maintaining high accuracy (low loss). We implement our approach in an image classification task using the MNIST dataset, and the system environment is simulated as a trajectory navigation scenario to emulate different channel conditions. Numerical simulations show that our proposed solution results in minimal energy consumption and CO2emission compared to the considered baselines while exhibiting robust performance across different channel conditions and bandwidth regime choices.

In this article, we propose a communication-efficient decentralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). To reduce the number of communication links, every worker in Q-GADMM communicates only with two neighbors, while updating its model via the group alternating direction method of multipliers (GADMM). Moreover, each worker transmits the quantized difference between its current model and its previously quantized model, thereby decreasing the communication payload size. However, due to the lack of centralized entity in decentralized ML, the spatial sparsity and payload compression may incur error propagation, hindering model training convergence. To overcome this, we develop a novel stochastic quantization method to adaptively adjust model quantization levels and their probabilities, while proving the convergence of Q-GADMM for convex objective functions. Furthermore, to demonstrate the feasibility of Q-GADMM for non-convex and stochastic problems, we propose quantized stochastic GADMM (Q-SGADMM) that incorporates deep neural network architectures and stochastic sampling. Simulation results corroborate that Q-GADMM significantly outperforms GADMM in terms of communication efficiency while achieving the same accuracy and convergence speed for a linear regression task. Similarly, for an image classification task using DNN, Q-SGADMM achieves significantly less total communication cost with identical accuracy and convergence speed compared to its counterpart without quantization, i.e., stochastic GADMM (SGADMM).

Wireless connectivity is instrumental in enabling scalable federated learning (FL), yet wireless channels bring challenges for model training, in which channel randomness perturbs each worker’s model update while multiple workers’ updates incur significant interference under limited bandwidth. To address these challenges, in this work we formulate a novel constrained optimization problem, and propose an FL framework harnessing wireless channel perturbations and interference for improving privacy, bandwidth-efficiency, and scalability. The resultant algorithm is coinedanalog federated ADMM (A-FADMM)based on analog transmissions and the alternating direction method of multipliers (ADMM). In A-FADMM, all workers upload their model updates to the parameter server (PS) using a single channel via analog transmissions, during which all models are perturbed and aggregated over-the-air. This not only saves communication bandwidth, but also hides each worker’s exact model update trajectory from any eavesdropper including the honest-but-curious PS, thereby preserving data privacy against model inversion attacks. We formally prove the convergence and privacy guarantees of A-FADMM for convex functions under time-varying channels, and numerically show the effectiveness of A-FADMM under noisy channels and stochastic non-convex functions, in terms of convergence speed and scalability, as well as communication bandwidth and energy efficiency.

Underwater optical wireless communication is an emerging field that can provide reliable connectivity for future generation Internet of Underwater Things devices. In this article, we propose a communication system based on single and superposition of Laguerre-Gaussian modes to transfer information and rely on a convolutional neural network for the mode identification in an underwater environment. A 100% recovery fidelity is reported at clear and turbid water. Beyond 90% of identification, accuracy is achieved under different laboratory-emulated underwater turbulence conditions. The practical implementation of the proposed spatial-mode-based communication scheme is further discussed.

The level crossing rate (LCR) and the average outage duration (AOD) are two important second order statistics that allow a deeper understanding of the behavior of the channel. In this paper, we study these metrics in order to assess the performance of free space optical (FSO) communication links in the presence of weak atmospheric turbulence and rice-induced pointing errors. More specifically, we derive an integral and a Gauss-Laguerre quadrature representation for both the LCR and the AOD in the single hop case and for their respective bounds in the multihop case. Selected numerical simulations are presented to show the accuracy of the derived results and to study the effect of certain system parameters on these two performance metrics.

In this paper, we are interested in determining the cumulative distribution function of the sum of α - μ random variables in the setting of rare event simulations. To this end, we present an efficient importance sampling estimator. The main result of this work is the bounded relative error property of the proposed estimator. This result is used to accurately estimate the outage probability of multibranch maximum ratio combining and equal gain diversity receivers over α-μ fading channels. Selected numerical simulations are discussed to show the robustness of our estimator compared to naive Monte Carlo.

In this paper, we are interested in determining the cumulative distribution function of the sum of α - μ, κ - μ, and η - μ random variables in the setting of rare event simulations. To this end, we present a simple and efficient importance sampling approach. The main result of this work is the bounded relative error property of the proposed estimators. Capitalizing on this result, we accurately estimate the outage probability of multibranch maximum ratio combining and equal gain diversity receivers over α - μ, κ - μ, and η - μ fading channels. Selected numerical simulations are discussed to show the robustness of our estimators compared with naive Monte Carlo estimators.

The Gamma-Gamma distribution has recently emerged in a number of applications ranging from modeling scattering and reverberation in sonar and radar systems to modeling atmospheric turbulence in wireless optical channels. In this respect, assessing the outage probability achieved by some diversity techniques over this kind of channels is of major practical importance. In many circumstances, this is related to the difficult question of analyzing the statistics of a sum of Gamma-Gamma random variables. Answering this question is not a simple matter. This is essentially because outage probabilities encountered in practice are often very small, and hence, the use of classical Monte Carlo methods is not a reasonable choice. This lies behind the main motivation of this paper. In particular, this paper proposes a new approach to estimate the left tail of the sum of Gamma-Gamma variates. More specifically, we propose robust importance sampling schemes that efficiently evaluates the outage probability of diversity receivers over Gamma-Gamma fading channels. The proposed estimators satisfy the well-known bounded relative error criterion for both maximum ratio combining and equal gain combining cases. We show the accuracy and the efficiency of our approach compared with naive Monte Carlo via some selected numerical simulations.

When assessing the performance of the free space optical (FSO) communication systems, the outage probability encountered is generally very small, and thereby the use of nave Monte Carlo simulations becomes prohibitively expensive. To estimate these rare event probabilities, we propose in this paper an importance sampling approach which is based on the exponential twisting technique to offer fast and accurate results. In fact, we consider a variety of turbulence regimes, and we investigate the outage probability of FSO communication systems, under a generalized pointing error model based on the Beckmann distribution, for both single and multihop scenarios. Selected numerical simulations are presented to show the accuracy and the efficiency of our approach compared with naive Monte Carlo.

Over the past few years, free-space optical (FSO) communication has gained significant attention. In fact, FSO can provide cost-effective and unlicensed links, with high-bandwidth capacity and low error rate, making it an exciting alternative to traditional wireless radio-frequency communication systems. However, the system performance is affected not only by the presence of atmospheric turbulences, which occur due to random fluctuations in the air refractive index but also by the existence of pointing errors. Metrics, such as the outage probability which quantifies the probability that the instantaneous signal-to-noise ratio is smaller than a given threshold, can be used to analyze the performance of this system. In this work, we consider weak and strong turbulence regimes, and we study the outage probability of an FSO communication system under a generalized pointing error model with both a nonzero boresight component and different horizontal and vertical jitter effects. More specifically, we use an importance sampling approach which is based on the exponential twisting technique to offer fast and accurate results.