Over-the-air federated learning (OTA FL) provides a joint computation and communication approach to design FL systems with improved efficiency. By leveraging the superposition property of wireless channels, OTA FL enables the automatic aggregation of intermediate parameters-such as gradients-across a large number of clients, significantly reducing communication overhead while concurrently enhancing transmission privacy. However, gradient aggregation requires all clients to use identical model architectures, a condition often impractical in real-world scenarios due to variations in client hardware and computational capabilities. This mismatch can lead to scalability issues and system incompatibilities. To address this challenge, we propose a model-agnostic method based on model prototypes that enables collaborative training across clients with heterogeneous models. The proposed method bypasses the requirements of the conventional model weight/gradient updates with prototype vector aggregation, without requiring the model structures of all clients to be identical. To the best of our knowledge, the proposed method is the first to explore the prototypical model-heterogeneous OTA FL with desirable training performance and extremely low communication cost. We conducted extensive experiments to verify the efficacy of the proposed method. The results show that our approach not only significantly reduces communication overhead but also exploits the superior capabilities of large models to enhance the performance of smaller models.

Federated Long-Tailed Learning (Fed-LT), a paradigm wherein data collected from decentralized local clients manifests a globally prevalent long-tailed distribution, has garnered considerable attention in recent times. In the context of Fed-LT, existing works have predominantly centered on addressing the data imbalance issue to enhance the efficacy of the generic global model while neglecting the performance at the local level. In contrast, conventional Personalized Federated Learning (pFL) techniques are primarily devised to optimize personalized local models under the presumption of a balanced global data distribution. This paper introduces an approach termed Federated Local and Generic Model Training in Fed-LT (FedLoGe), which enhances both local and generic model performance through the integration of representation learning and classifier alignment within a neural collapse framework. Our investigation reveals the feasibility of employing a shared backbone as a foundational framework for capturing overarching global trends, while concurrently employing individualized classifiers to encapsulate distinct refinements stemming from each client’s local features. Building upon this discovery, we establish the Static Sparse Equiangular Tight Frame Classifier (SSE-C), inspired by neural collapse principles that naturally prune extraneous noisy features and foster the acquisition of potent data representations. Furthermore, leveraging insights from imbalance neural collapse's classifier norm patterns, we develop Global and Local Adaptive Feature Realignment (GLA-FR) via an auxiliary global classifier and personalized Euclidean norm transfer to align global features with client preferences. Extensive experimental results on CIFAR-10/100-LT, ImageNet, and iNaturalist demonstrate the advantage of our method over state-of-the-art pFL and Fed-LT approaches.