We propose a Reinforcement Learning (RL) framework for hybrid truck-drone Last-Mile Delivery (LMD), where drones launch from a truck and either rendezvous with it or return to the depot after serving customers, subject to vehicle range limits and customer-defined deadlines. Deliveries completed after the latest allowable service time are treated as deadline violations (lateness) and are penalized in the objective. To provide an exact reference, a Mixed Integer Linear Programming (MILP) model is formulated. It serves both as a benchmark for evaluating the RL approach and as a practical method when the delivery set is small or when minimizing lateness is the primary objective, since the exact MILP solution is guaranteed to be optimal but become computationally expensive to solve as problem size grows due to the combinatorial increase in routing and coordination possibilities. The RL method is introduced as a scalable alternative for larger hybrid truck-drone deployments because it maintains practical computation times through iterative learning and avoids the exponential growth in solving effort associated with exact optimization. The RL framework employs an$\epsilon $-greedy policy with decay and a reward structure that mirrors the MILP objective. Traffic congestion is incorporated through a scaling matrix that adjusts truck travel times on road segments, while drones remain unaffected due to aerial operation. RL is further benchmarked against Genetic Algorithm (GA) and Particle Swarm Optimiztion (PSO), which are also formulated to align with the same MILP objective to ensure a consistent basis for comparison. Results show that the RL component scales more efficiently with increasing delivery size while maintaining competitive solution quality.

Federated averaging (FedAvg) is a well-recognized framework for distributed learning that efficiently manages communication. Several algorithms have emerged to enhance the communication efficiency of FedAvg and its variations. Some of these algorithms focus on reducing the number of communication rounds by allowing clients to skip frequent interactions with the parameter server. In this work, our primary concern is the overall energy consumption during model training in federated learning. We challenge the conventional notion that reducing the frequency of communication leads to energy savings and present evidence that for nonindependent and nonidentical (non-IID) data distribution, increasing the frequency of communication can, in fact, result in greater energy conservation. Our contribution comprises two key aspects: first, we introduce a quantized version of the recently proposed algorithm called federated primal-dual (FedPD) (Zhang et al., 2021), which we refer to as quantized FedPD (QFedPD). Importantly, we substantiate the convergence guarantees for QFedPD. Second, we explore the tradeoff between quantization and communication skipping in the proposed approach. Our analysis demonstrates that applying quantization without skipping communication, using QFedPD, yields the most significant energy-saving benefits for non-IID data distribution. Intriguingly, when dealing with non-IID data distribution, the preferred strategy is to maximize energy efficiency, allowing all clients to transmit at every iteration while quantizing their updates.

In this paper, we consider a decentralized consensus optimization problem defined over a network of inter-connected devices that collaboratively solve the problem using only local data and information exchange with their neighbours. Despite their fast convergence, Newton-type methods require sending Hessian information between devices, making them communication inefficient while violating the devices' privacy. By formulating the Newton direction learning problem as a sum of separable functions subjected to a consensus constraint, our proposed approach learns an inexact Newton direction alongside the global model using the proximal primal-dual (Prox-PDA) algorithm. Our algorithm, coined DIN, avoids sharing Hessian information between devices since each device shares a model-sized vector, concealing the first- and second-order information, reducing the network's burden and improving communication and energy efficiencies. Numerical simulations corroborate that DIN exhibits higher communication efficiency in terms of communication rounds while consuming less communication and computation energy compared to existing second-order decentralized baselines.

In this paper, we propose a decentralized Newton-type approach to solve the problem of decentralized federated learning (FL). Notably, our proposed algorithm leverages the fast convergence of the second-order methods while avoid sending the hessian matrix at each iteration. Therefore, the proposed approach significantly reduces the communication cost and preserves the privacy. Specifically, we alternate between two problems. The inner problem approximates the inverse Hessian-gradient product which is formulated as a quadratic optimization problem and approximately solved in a decentralized manner using one step of the group alternating direction method of multipliers (GADMM) method. The outer problem learns the model, which is solved by performing one decentralized Newton step at every iteration. Moreover, to reduce the communication-overhead per iteration, a quantized version (leveraging stochastic quantization) is also proposed. Simulation results illustrate that our algorithm outperforms the baselines of GADMM, Q-GADMM, Newton tracking, and Decentralized SGD, and provides energy and communication-efficient solutions for bandwidth-limited systems under different SNR regimes.

Newton-type methods are popular in federated learning due to their fast convergence. Still, they suffer from two main issues, namely: low communication efficiency and low privacy due to the requirement of sending Hessian information from clients to parameter server (PS). In this work, we introduced a novel framework called FedNew in which there is no need to transmit Hessian information from clients to PS, hence resolving the bottleneck to improve communication efficiency. In addition, FedNew hides the gradient information and results in a privacy-preserving approach compared to the existing state-of-the-art. The core novel idea in FedNew is to introduce a two level framework, and alternate between updating the inverse Hessian-gradient product using only one alternating direction method of multipliers (ADMM) step and then performing the global model update using Newton’s method. Though only one ADMM pass is used to approximate the inverse Hessian-gradient product at each iteration, we develop a novel theoretical approach to show the converging behavior of FedNew for convex problems. Additionally, a significant reduction in communication overhead is achieved by utilizing stochastic quantization. Numerical results using real datasets show the superiority of FedNew compared to existing methods in terms of communication costs.

In this paper, we propose a communication-efficient alternating direction method of multipliers (ADMM)-based algorithm for solving a distributed learning problem in the stochastic non-convex setting. Our approach runs a few stochastic gradient descent (SGD) steps to solve the local problem at each worker instead of finding the exact/approximate solution as proposed by existing ADMM-based works. By doing so, the proposed framework strikes a good balance between the computation and communication costs. Extensive simulation results show that our algorithm significantly outperforms existing stochastic ADMM in terms of communication-efficiency, notably in the presence of non-independent and identically distributed (non-IID) data.

In this paper, we propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM (CQ-GGADMM), leverages the worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM), and pushes the frontier in communication efficiency by extending its applicability to generalized network topologies, while incorporating link censoring for negligible updates after quantization. We theoretically prove that CQ-GGADMM achieves the linear convergence rate when the local objective functions are strongly convex under some mild assumptions. Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed, compared to the censored decentralized ADMM, and the worker grouping method of GADMM.

In this paper, we propose an energy-efficient federated meta-learning framework. The objective is to enable learning a meta-model that can be fine-tuned to a new task with a few number of samples in a distributed setting and at low computation and communication energy consumption. We assume that each task is owned by a separate agent, so a limited number of tasks is used to train a meta-model. Assuming each task was trained offline on the agent's local data, we propose a lightweight algorithm that starts from the local models of all agents, and in a backward manner using projected stochastic gradient ascent (P-SGA) finds a meta-model. The proposed method avoids complex computations such as computing hessian, double looping, and matrix inversion, while achieving high performance at significantly less energy consumption compared to the state-of-the-art methods such as MAML and iMAML on conducted experiments for sinusoid regression and image classification tasks.

Split-learning (SL) has recently gained popularity due to its inherent privacy-preserving capabilities and ability to enable collaborative inference for devices with limited computational power. Standard SL algorithms assume an ideal underlying digital communication system and ignore the problem of scarce communication bandwidth. However, for a large number of agents, limited bandwidth resources, and time-varying commu-nication channels, the communication bandwidth can become the bottleneck. To address this challenge, in this work, we propose a novel SL framework to solve the remote inference problem that introduces an additional layer at the agent side and constrains the choices of the weights and the biases to ensure over the air aggregation. Hence, the proposed approach maintains constant communication cost with respect to the number of agents enabling remote inference under limited bandwidth. Numerical results show that our proposed algorithm significantly outper-forms the digital implementation in terms of communication-efficiency” especially as the number of agents grows large.

This article deals with the problem of distributed machine learning, in which agents update their models based on their local datasets, and aggregate the updated models collaboratively and in a fully decentralized manner. In this paper, we tackle the problem of information heterogeneity arising in multi-agent networks where the placement of informative agents plays a crucial role in the learning dynamics. Specifically, we propose BayGo, a novel fully decentralized joint Bayesian learning and graph optimization framework with proven fast convergence over a sparse graph. Under our framework, agents are able to learn and communicate with the most informative agent to their own learning. Unlike prior works, our framework assumes no prior knowledge of the data distribution across agents nor does it assume any knowledge of the true parameter of the system. The proposed alternating minimization based framework ensures global connectivity in a fully decentralized way while minimizing the number of communication links. We theoretically show that by optimizing the proposed objective function, the estimation error of the posterior probability distribution decreases exponentially at each iteration. Via extensive simulations, we show that our framework achieves faster convergence and higher accuracy compared to fully-connected and star topology graphs.

Today’s intelligent applications can achieve high performance accuracy using machine learning (ML) techniques, such as deep neural networks (DNNs). Traditionally, in a remote DNN inference problem, an edge device transmits raw data to a remote node that performs the inference task. However, this may incur high transmission energy costs and puts data privacy at risk. In this paper, we propose a technique to reduce the total energy bill at the edge device by utilizing model compression and time-varying model split between the edge and remote nodes. The time-varying representation accounts for time-varying channels and can significantly reduce the total energy at the edge device while maintaining high accuracy (low loss). We implement our approach in an image classification task using the MNIST dataset, and the system environment is simulated as a trajectory navigation scenario to emulate different channel conditions. Numerical simulations show that our proposed solution results in minimal energy consumption and CO2emission compared to the considered baselines while exhibiting robust performance across different channel conditions and bandwidth regime choices.

Machine learning (ML) is a promising enabler for the fifth-generation (5G) communication systems and beyond. By imbuing intelligence into the network edge, edge nodes can proactively carry out decision-making and, thereby, react to local environmental changes and disturbances while experiencing zero communication latency. To achieve this goal, it is essential to cater for high ML inference accuracy at scale under the time-varying channel and network dynamics, by continuously exchanging fresh data and ML model updates in a distributed way. Taming this new kind of data traffic boils down to improving the communication efficiency of distributed learning by optimizing communication payload types, transmission techniques, and scheduling, as well as ML architectures, algorithms, and data processing methods. To this end, this article aims to provide a holistic overview of relevant communication and ML principles and, thereby, present communication-efficient and distributed learning frameworks with selected use cases.

In this article, we propose a communication-efficient decentralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). To reduce the number of communication links, every worker in Q-GADMM communicates only with two neighbors, while updating its model via the group alternating direction method of multipliers (GADMM). Moreover, each worker transmits the quantized difference between its current model and its previously quantized model, thereby decreasing the communication payload size. However, due to the lack of centralized entity in decentralized ML, the spatial sparsity and payload compression may incur error propagation, hindering model training convergence. To overcome this, we develop a novel stochastic quantization method to adaptively adjust model quantization levels and their probabilities, while proving the convergence of Q-GADMM for convex objective functions. Furthermore, to demonstrate the feasibility of Q-GADMM for non-convex and stochastic problems, we propose quantized stochastic GADMM (Q-SGADMM) that incorporates deep neural network architectures and stochastic sampling. Simulation results corroborate that Q-GADMM significantly outperforms GADMM in terms of communication efficiency while achieving the same accuracy and convergence speed for a linear regression task. Similarly, for an image classification task using DNN, Q-SGADMM achieves significantly less total communication cost with identical accuracy and convergence speed compared to its counterpart without quantization, i.e., stochastic GADMM (SGADMM).

Wireless connectivity is instrumental in enabling scalable federated learning (FL), yet wireless channels bring challenges for model training, in which channel randomness perturbs each worker’s model update while multiple workers’ updates incur significant interference under limited bandwidth. To address these challenges, in this work we formulate a novel constrained optimization problem, and propose an FL framework harnessing wireless channel perturbations and interference for improving privacy, bandwidth-efficiency, and scalability. The resultant algorithm is coinedanalog federated ADMM (A-FADMM)based on analog transmissions and the alternating direction method of multipliers (ADMM). In A-FADMM, all workers upload their model updates to the parameter server (PS) using a single channel via analog transmissions, during which all models are perturbed and aggregated over-the-air. This not only saves communication bandwidth, but also hides each worker’s exact model update trajectory from any eavesdropper including the honest-but-curious PS, thereby preserving data privacy against model inversion attacks. We formally prove the convergence and privacy guarantees of A-FADMM for convex functions under time-varying channels, and numerically show the effectiveness of A-FADMM under noisy channels and stochastic non-convex functions, in terms of convergence speed and scalability, as well as communication bandwidth and energy efficiency.

In this paper, we propose a communication-efficient decen-tralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). Every worker in Q-GADMM communicates only with two neighbors, and updates its model via the group alternating direct method of multiplier (GADMM), thereby ensuring fast convergence while reducing the number of communication rounds. Furthermore, each worker quantizes its model updates before transmissions, thereby decreasing the communication payload sizes. We prove that Q-GADMM converges to the optimal solution for convex loss functions, and numerically show that Q-GADMM yields 7x less communication cost while achieving almost the same accuracy and convergence speed compared to GADMM without quantization.

This article proposes a communication-efficient decentralized deep learning algorithm, coined layer-wise federated group ADMM (L-FGADMM). To minimize an empirical risk, every worker in L-FGADMM periodically communicates with two neighbors, in which the periods are separately adjusted for different layers of its deep neural network. A constrained optimization problem for this setting is formulated and solved using the stochastic version of GADMM proposed in our prior work. Numerical evaluations show that by less frequently exchanging the largest layer, L-FGADMM can significantly reduce the communication cost, without compromising the convergence speed. Surprisingly, despite less exchanged information and decentralized operations, intermittently skipping the largest layer consensus in L-FGADMM creates a regularizing effect, thereby achieving the test accuracy as high as federated learning (FL), a baseline method with the entire layer consensus by the aid of a central entity.

In this paper, we tackle the problem of remote monitoring (e.g., remote factory) in which a number of sensor nodes are transmitting time sensitive measurements to a remote monitoring site. We assume that packets generated by different sensors have different sizes. Moreover, different sensors have different Maximum Allowable Transfer Intervals (MATIs). We consider minimizing a metric that maintains a trade-off between minimizing the average MATI violation of all sensors, and minimizing the probability that the MATI violation of each sensor exceeds a predefined threshold. We formulate the problem as a stochastic optimization problem with integer constraints. In order to solve this problem, we first relax the original intractable formulation to a tractable problem. Then, we use the Lyapunov stochastic optimization framework to solve the relaxed problem. Simulation results show that the proposed algorithm outperforms the considered baselines in terms of minimizing the probability of the MATI violation for all sensors.

When the data is distributed across multiple servers, lowering the communication cost between the servers (or workers) while solving the distributed learning problem is an important problem and is the focus of this paper. In particular, we propose a fast, and communication-efficient decentralized framework to solve the distributed machine learning (DML) problem. The proposed algorithm, Group Alternating Direction Method of Multipliers (GADMM) is based on the Alternating Direction Method of Multipliers (ADMM) framework. The key novelty in GADMM is that it solves the problem in a decentralized topology where at most half of the workers are competing for the limited communication resources at any given time. Moreover, each worker exchanges the locally trained model only with two neighboring workers, thereby training a global model with a lower amount of communication overhead in each exchange. We prove that GADMM converges to the optimal solution for convex loss functions, and numerically show that it converges faster and more communication-efficient than the state-of-the-art communication-efficient algorithms such as the Lazily Aggregated Gradient (LAG) and dual averaging, in linear and logistic regression tasks on synthetic and real datasets. Furthermore, we propose Dynamic GADMM (D-GADMM), a variant of GADMM, and prove its convergence under the time-varying network topology of the workers.

This paper proposes and evaluates a novel algorithm for streaming video over HTTP. The problem is formulated as a non-convex optimization problem which is constrained by the predicted available bandwidth, chunk deadlines, available video rates, and buffer occupancy. The objective is to optimize a QoE metric that maintains a tradeoff between maximizing the playback rate of every chunk and ensuring fairness among different chunks for the minimum re-buffering time. We propose FastScan, a low complexity algorithm that solves the problem. The online adaptations for dynamic bandwidth environments are proposed with imperfect available bandwidth prediction. The results of experiments driven by variable bit rate (VBR) encoded video, video platform system (dash.js), and cellular bandwidth traces of a public dataset reveal the robustness of the online version of FastScan algorithm and demonstrate its significant performance improvement as compared to the considered state-of-the-art video streaming algorithms. For example, on an experiment conducted over 100 real cellular available bandwidth traces of a public dataset that spans different available bandwidth regimes, our proposed algorithm (FastScan) achieves the minimum re-buffering (stall) time and the maximum average playback rate in every single trace as compared to Bola, Festive, BBA, RB, FastMPC, and Pensieve algorithms.

Most client hosts are equipped with multiple network interfaces (e.g., WiFi and cellular networks). Simultaneous access of multiple interfaces can significantly improve the users' quality of experience (QoE) in video streaming. An intuitive approach to achieve it is to use Multi-path TCP (MPTCP). However, the deployment of MPTCP, especially with link preference, requires OS kernel update at both the client and server side, and a vast amount of commercial content providers do not support MPTCP. Thus, in this paper, we realize a multi-path video streaming algorithm in the application layer instead, by considering Scalable Video Coding (SVC), where each layer of every chunk can be fetched from only one of the orthogonal paths. We formulate the quality decisions of video chunks subject to the available bandwidth of the different paths, chunk deadlines, and link preferences as an optimization problem. The objective is to to optimize a QoE metric that maintains a tradeoff between maximizing the playback rate of every chunk and ensuring fairness among chunks. The proposed metric prefers to use bandwidth of the links to optimize a concave utility function of the chunk quality. Even though the formulation is a non-convex discrete optimization, we provide a quadratic complexity algorithm which is shown to be optimal in some special cases. We further propose an online algorithm where several challenges including bandwidth prediction errors, are addressed. Extensive emulated experiments in a real testbed with real traces of public dataset reveal the robustness of our scheme and demonstrate its significant performance improvement compared to other multi-path algorithms.

With the advent of emerging mission-critical applications, sampling information updates and scheduling mobile traffic in a timely manner are very challenging. In addition, maintaining fresh information and low latency communication is important to these applications. To that end, in this paper, we first derive closed form expressions for an upper bound on the latency tail probability (LTP) and the average age of information (AoI) in M/G/1 systems, where shifted exponential service time is considered. Different from the majority of existing work in this domain, our analysis is derived under different update size assumption with different priority levels. Next, we have developed novel policies for sampling and scheduling the information updates over the choice of one of the parallel links, e.g., WiFi and LTE links. Then, a joint minimization of AoI and LTP is formulated and efficient algorithms are provided. Our evaluation results show that our proposed approaches outperform the state-of-the-art algorithms and some competitive baselines.

One of the 5G promises is to provide Ultra Reliable Low Latency Communications (URLLC) which targets an end to end communication latency that is <; 1ms. The very low latency requirement of URLLC entails a lot of work in all networking layers. In this paper, we focus on the physical layer, and in particular, we propose a novel formulation of the massive MIMO uplink detection problem. We introduce an objective function that is a sum of strictly convex and separable functions based on decomposing the received vector into multiple vectors. Each vector represents the contribution of one of the transmitted symbols in the received vector. Proximal Jacobian Alternating Direction Method of Multipliers (PJADMM) is used to solve the new formulated problem in an iterative manner where at every iteration all variables are updated in parallel and in a closed form expression. The proposed algorithm provides a lower complexity and much faster processing time compared to the conventional MMSE detection technique and other iterative-based techniques, especially when the number of single antenna users is close to the number of base station (BS) antennas. This improvement is obtained without any matrix inversion. Simulation results demonstrate the efficacy of the proposed algorithm in reducing detection processing time in the multi-user uplink massive MIMO setting.

Age of Information (AoI) measures the freshness of the information at a remote location. AoI reflects the time that is elapsed since the generation of the packet by a transmitter. In this paper, we consider a remote monitoring problem (e.g., remote factory) in which a number of sensor nodes are transmitting time sensitive measurements to a remote monitoring site. We consider minimizing a metric that maintains a trade-off between minimizing the sum of the expected AoI of all sensors and minimizing an Ultra Reliable Low Latency Communication (URLLC) term. The URLLC term is considered to ensure that the probability the AoI of each sensor exceeds a predefined threshold is minimized. Moreover, we assume that sensors tolerate different threshold values and generate packets at different sizes. Motivated by the success of machine learning in solving large networking problems at low complexity, we develop a low complexity reinforcement learning based algorithm to solve the proposed formulation. We trained our algorithm using the state-of-the-art actor-critic algorithm over a set of public bandwidth traces. Simulation results show that the proposed algorithm outperforms the considered baselines in terms of minimizing the expected AoI and the threshold violation of each sensor.

In this paper, we formulate a problem that is a variant of the knapsack problem. Even though the problem is NP-hard in general, we consider a special case of the problem where the problem is in P. For this special case, the proposed algorithm is linear time complexity in the number of bins. The proposed framework is a generalization of the framework that has been used recently in the context of finding rate adaptation algorithms for video streaming.

In this paper, we propose a cooperative multi-user video streaming system, termed GiantClient, for videos encoded using scalable video coding (SVC). The proposed system allows a group of users to watch a video on a single screen. The users, who may have different data plans from different carriers or different levels of energy, can collaborate to fetch the SVC-encoded video at high quality and avoid running into re-buffering. Using SVC, each layer of every chunk of the video can be fetched by only one of the cooperating users. Therefore, we formulate the streaming problem that obtains the quality and the fetching policy decisions as an optimization problem. The objective is to optimize a novel quality-of-experience metric that maintains a tradeoff between maximizing the quality of every chunk and ensuring fairness among all video chunks for the minimum re-buffering time. The problem is constrained with the available bandwidth, the chunk deadlines, and the imposed maximum contribution constraints by users. Moreover, we propose a low-complexity algorithm to solve the proposed optimization problem. A real implementation of the system with real SVC-encoded videos and real bandwidth traces reveal the robustness and performance of the proposed algorithm.

In this paper, we study the problem of Quality of Experience (QoE) aware resource allocation in wireless systems. In particular, we consider application-aware joint Bandwidth-Power allocation for a small cell. We optimize a QoE metric for multi-user video streaming in a small cell that maintains a trade-off between maximizing the playback rate of each user and ensuring proportional fairness (PF) among users. We formulate the application-driven joint bandwidth-power allocation as a non-convex optimization problem. However, we develop a polynomial complexity algorithm, and we show that the proposed algorithm achieves the optimal solution of the proposed optimization problem. Simulation results show that the proposed QoE-aware algorithm significantly improves the average QoE. Moreover, it outperforms the weighted sum rate allocation which is the state-of-the-art physical resource allocation scheme.

In this paper, we consider the energy-bandwidth allocation for a network of multiple users, where the transmitters each powered by both an energy harvester and conventional grid, access the network orthogonally on the assigned frequency band. The tradeoff among the weighted sum throughput, the use of grid energy, and the amount of energy cooperation is studied through an optimization objective which is a linear combination of these quantities. To solve the problem efficiently, an iterative algorithm is proposed using the Proximal Jacobian ADMM. We show that this algorithm converges to the optimal solution with an overall complexity of O(N2K2). Numerical results show that the proposed algorithms can make efficient use of the harvested energy, grid energy, energy cooperation, and the available bandwidth.