In this paper, we propose a hybrid learning framework that combines federated and split learning, termed semi-federated learning (SemiFL), in which over-the-air computation is utilized for gradient aggregation. A key idea is to strategically adjust the learning rate by manipulating over-the-air distortion for improving SemiFL’s convergence. Specifically, we intentionally amplify amplitude distortion to increase the learning rate in the non-stable region, thereby accelerating convergence and reducing communication energy consumption. In the stable region, we suppress noise perturbation to maintain a small learning rate for improving SemiFL’s final convergence. Theoretical results demonstrate the antagonistic effects of over-the-air distortion in different regions, under both independent and identically distributed (IID) and non-IID data settings. Then, we formulate two energy consumption minimization problems, one for each region, which implements a two-region mean square error threshold configuration scheme. Accordingly, we propose two resource allocation algorithms with closed-form solutions. Simulation results show that under different network and data distribution conditions, strategically manipulating over-the-air distortion can efficiently adjust the learning rate to improve SemiFL’s convergence. Moreover, energy consumption can be reduced by using the proposed algorithms.

In existing deep learning-based semantic communication systems, centralized training of semantic models brings a risk of privacy leakage, whereas distributed training imposes a huge computational burden on user equipments (UEs). To address these challenges, we propose a hybrid federated learning (Hybrid-FL) framework to alleviate the computational burden on UEs while protecting the user privacy. Specifically, each UE uploads local gradients and semantic symbols to the base station for the collaborative training of global and local semantic models. Furthermore, we propose a joint communication and computation scheme for supporting the model aggregation and semantics transmission. To gain deep insights, we expose the joint impact of communication and computation on the convergence behavior of Hybrid-FL by deriving an upper bound. Then, we formulate a mixed-integer nonlinear programming problem to improve the convergence performance of Hybrid-FL, which is then effectively solved by using our proposed algorithm that developed based on alternating and matching theory. Experimental results demonstrate that Hybrid-FL outperforms conventional FL by achieving a 20% accuracy gain and a 80% latency reduction.

The emergence of large language models (LLMs) with multi-task generalization capabilities is expected to improve the performance of artificial intelligence (AI)-as-a-service provision in 6G networks. By fine-tuning LLMs, AI services can become more precise and tailored to the demands of different downstream tasks. However, centralized fine-tuning paradigms pose a potential risk to user privacy, and existing distributed fine-tuning methods incur significant wireless transmission burdens due to the large-scale parameter transmission of LLMs. To tackle these challenges, by leveraging the low rank feature in LLM fine-tuning, we propose a wireless over-the-air federated learning (AirFL) based low-rank adaptation (LoRA) framework that integrates LoRA and over-the-air computation (AirComp) to achieve efficient fine-tuning and aggregation. Based on multiple-input multiple-output (MIMO) and orthogonal frequency division multiplexing (OFDM), we design a multi-stream AirComp scheme to fulfill the aggregation requirement of AirFL-LoRA. Furthermore, by deriving an optimality gap, we gain theoretical insights into the joint impact of rank selection and gradient aggregation distortion on the fine-tuning performance of AirFL-LoRA. Next, we formulate a non-convex problem to minimize the optimality gap, which is solved by the proposed backtracking-based alternating algorithm and the manifold optimization algorithm iteratively. Through fine-tuning LLMs for different downstream tasks, experimental results reveal that the AirFL-LoRA framework outperforms the state-of-the-art baselines on both training loss and perplexity, closely approximating the performance of FL with ideal aggregation.

To enhance the robustness and convergence of hierarchical federated learning (HFL) in wireless networks with imperfect channel state information (CSI), a quantized HFL (QHFL) framework is proposed. Considering the local training and communication latency, the outage probability of quantized gradient transmission is modeled under imperfect CSI. Then, the convergence of the proposed QHFL with transmission outage and gradient quantization is analyzed. Simulation results demonstrate the correlation between quantization accuracy and transmission outage, along with their joint impact on the HFL convergence, which align with the insight behind the convergence analysis.

In existing federated learning (FL), the base station (BS) coordinates devices to collaboratively train a shared model by avoiding the transmission of raw data. To achieve communication-efficient model uploading, over-the-air computation (AirComp) is often employed to aggregate model parameters. However, in conventional AirComp assisted FL, the BS’s abundant computation resources are underutilized due to its non-involvement in model training. Meanwhile, transmission failures resulting from fluctuating wireless channels impair the quality of model aggregation. In this paper, we propose a retransmission-based semi-federated learning (SemiFL) framework, wherein devices upload model parameters and public privacy-free data for enabling a hybrid implementation of FL and centralized learning (CL). In our new framework, the BS leverages its abundant computation resources to aid CL model training, which mitigates the resource wastage while alleviating local computational burden of devices. Moreover, the proposed new retransmission mechanism effectively overcomes detrimental transmission failures resulting from the fluctuating quasi-static channel, aiming to guarantee improved learning performance of SemiFL. Successful transmission probabilities of both retransmission-based AirComp and retransmission-based digital communication are provided in closed forms. To attain deep insights, we derive an optimality gap to capture the convergence behavior of retransmission-based SemiFL. Then, we formulate a non-convex long-term problem to minimize a weighted sum of overall latency and energy consumption by jointly optimizing communication, computation, and learning parameters. Extensive experimental results show that our retransmission-based SemiFL obtains 21.9%, 30.5%, and 44.1% accuracy gains on three datasets, while efficaciously reducing latency and energy consumption compared to benchmarks. Meanwhile, our scheme enhances learning performance on the fluctuating quasi-static channel compared to state-of-the-art schemes.

In this paper, we propose a semi-federated learning (SemiFL) framework to ameliorate the performance of conventional federated learning. The base station and devices are coordinated to collaboratively train a shared model. However, due to the rapidly fluctuating channels and irrationally assigned local learning workloads, SemiFL encounters excessive latency. To overcome the challenges, we propose a retransmission-based over-the-air computation mechanism to facilitate model aggregation and data mixing over quasi-static channels. The closed-form probability of successful aggregation is derived, while the communication latency is modeled based on the Pascal distribution. Further, we establish an optimality gap to characterize the convergence performance of SemiFL, wherein the minimum number of iterations for attaining a specific local target accuracy is identified. Next, a joint resource allocation and local target accuracy assignment problem is formulated to minimize the latency of each round, subject to the decay rate, central processing unit (CPU) frequency, and transmit power. To address this non-convex problem, we develop an algorithm using the closed-form solutions for the normalizing factors and CPU frequencies. Simulation results on two real-world datasets confirm the superiority of SemiFL over benchmarks in terms of latency and learning performance.

In this paper, we propose a semi-federated learning (SemiFL) framework that allows computing-limited clients to collaboratively train a shared model with resource-abundant clients. Specifically, by supporting the coexistence of model-updating and data-offloading, the SemiFL framework enables both centralized and federated learning in a hybrid fashion. Due to the decoding error, we consider the practical case with residual interference. To improve uplink throughput for centralized learning while reducing aggregation distortion for federated learning, we formulate a non-convex optimization problem to jointly optimize the transmit power and receive strategy. Then, we propose an efficient algorithm to solve the challenging problem by using successive convex approximation. Simulation results demonstrate the effectiveness of our SemiFL framework for heterogeneous networks, and reveal the impact of imperfect signal decoding on communication rates.

Implementing existing federated learning in massive Internet of Things (IoT) networks faces critical challenges, such as imbalanced and statistically heterogeneous data and device diversity. To this end, we propose a semi-federated learning (SemiFL) framework to provide a potential solution for the realization of intelligent IoT. By seamlessly integrating the centralized and federated paradigms, our SemiFL framework shows high scalability in terms of the number of IoT devices even in the presence of computing-limited sensors. Furthermore, compared to traditional learning approaches, the proposed SemiFL can make better use of distributed data and computing resources, due to the collaborative model training between the edge server and local devices. Simulation results show the effectiveness of our SemiFL framework for massive IoT networks. The code can be found athttps://github.com/niwanli/SemiFL_IoT.

Under the organization of the base station (BS), wireless federated learning (FL) enables collaborative model training among multiple devices. However, the BS is merely responsible for aggregating local updates during the training process, which incurs a waste of the computational resources at the BS. To tackle this issue, we propose a semi-federated learning (SemiFL) paradigm to leverage the computing capabilities of both the BS and devices for a hybrid implementation of centralized learning (CL) and FL. Specifically, each device sends both local gradients and data samples to the BS for training a shared global model. To improve communication efficiency over the same time-frequency resources, we integrate over-the-air computation for aggregation and non-orthogonal multiple access for transmission by designing a novel transceiver structure. To gain deep insights, we conduct convergence analysis by deriving a closed-form optimality gap for SemiFL and extend the result to two extra cases. In the first case, the BS uses all accumulated data samples to calculate the CL gradient, while a decreasing learning rate is adopted in the second case. Our analytical results capture the destructive effect of wireless communication and show that both FL and CL are special cases of SemiFL. Then, we formulate a non-convex problem to reduce the optimality gap by jointly optimizing the transmit power and receive beamformers. Accordingly, we propose a two-stage algorithm to solve this intractable problem, in which we provide closed-form solutions to the beamformers. Extensive simulation results on two real-world datasets corroborate our theoretical analysis, and show that the proposed SemiFL outperforms conventional FL and achieves 3.2% accuracy gain on the MNIST dataset compared to state-of-the-art benchmarks.

Over-the-air federated learning (AirFL) allows devices to train a learning model in parallel and synchronize their local models using over-the-air computation. The integrity of AirFL is vulnerable due to the obscurity of the local models aggregated over the air. This paper presents a novel framework to balance the accuracy and integrity of AirFL, where multi-antenna devices and base station (BS) are jointly optimized with a reconfigurable intelligent surface (RIS). The key contributions include a new and non-trivial problem jointly considering the model accuracy and integrity of AirFL, and a new framework that transforms the problem into tractable subproblems. Under perfect channel state information (CSI), the new framework minimizes the aggregated model’s distortion and retains the local models’ recoverability by optimizing the transmit beamformers of the devices, the receive beamformers of the BS, and the RIS configuration in an alternating manner. Under imperfect CSI, the new framework delivers a robust design of the beamformers and RIS configuration to combat non-negligible channel estimation errors. As corroborated experimentally, the novel framework can achieve comparable accuracy to the ideal FL while preserving local model recoverability under perfect CSI, and improve the accuracy when the number of receive antennas is small or moderate under imperfect CSI.

This paper investigates the model aggregation process in an over-the-air federated learning (AirFL) system, where an intelligent reflecting surface (IRS) is deployed to assist the transmission from users to the base station (BS). Since the successive interference cancellation (SIC) is adopted as a basis to decode local model parameters and analyze their statistic characteristics for detecting malicious devices, the quality-of-service (QoS) requirement is ensured. The objective of this paper is to minimize the mean-square-error by jointly optimizing the receive beamforming vector at the BS, transmit power allocation at users, and phase shift matrix of the IRS, subject to the transmit power constraint for devices, unit-modulus constraint for reflecting elements, SIC decoding order constraint and QoS constraint. To address this complicated problem, alternating optimization is employed to decompose it into three subproblems, where the optimal receive beamforming vector is obtained by solving the first subproblem with the Lagrange dual method. Then, the convex relaxation method is applied to the transmit power allocation subproblem to find a suboptimal solution. Eventually, the phase shift matrix subproblem is addressed by invoking the semidefinite relaxation. Simulation results validate the availability of IRS and the effectiveness of the proposed scheme in improving federated learning performance.