The extensive local training in Hierarchical Federated Learning (HFL) imposes a substantial computational energy burden on end devices, a problem intensified by inherent system and data heterogeneity. While prior works attempt to mitigate this by using heterogeneous models or adjusting local training, they often suffer from critical drawbacks such as accuracy degradation from update biases and an inability to adapt to the dynamic nature of device resources. This paper introduces FedE2AD (Federated Energy-Efficient Adaptive Distillation), a novel cloud-network collaboration framework that leverages dynamic distillation to holistically optimize the energy-accuracy balance. At its core, FedE2AD implements this collaboration through a multi-level optimization approach. At the cloud layer, a Dynamic Model Allocation strategy intelligently assigns model architectures by assessing device status from static, dynamic, and data-centric perspectives. At the device layer, Variable Local Iterations enable real-time adaptation to fluctuating computational power. Crucially, to counteract model divergence, FedE2AD employs a Dual Knowledge Sharing mechanism at the edge layer, which uniquely combines direct aggregation of shared structures with data-free model distillation to ensure robust knowledge transfer. Experiments conducted on simulation platforms show that FedE2AD markedly outperforms existing methods. For instance, on the CIFAR-10 dataset under strong heterogeneity, it reduces single-round computation energy by 21.1% and increases final model accuracy by 1.42% compared to HDHRFL.

Extensive local training in Hierarchical Federated Learning (HFL) imposes high energy demands on resourceconstrained devices, a problem exacerbated by system heterogeneity which also causes performance variability. To address this, we propose HFHEMS, a novel method utilizing heterogeneous models and distillation to improve energy efficiency in HFL. HFHEMS employs dynamic model allocation to tailor computational loads to device capabilities and uses variable local iterations for real-time training adjustment. To preserve model accuracy, it integrates a dual knowledge sharing strategy with data-free distillation, enhancing knowledge transfer. Experiments confirm HFHEMS significantly reduces computation energy while maintaining robust accuracy, thus achieving a superior energy-accuracy trade-off.

Neural Architecture Search (NAS) can discover the optimal neural network architecture within a given SuperNet through automated search, which can improve model performance and reduce computational overhead on resource-constrained devices. Due to the vast SuperNet requiring substantial computational resources for training and evaluation, the search process is costly and difficult to apply directly on End Devices (EDs) with limited computational resources. Moreover, existing methods utilize zero-cost proxies to reduce computational costs in NAS, but overlook limited computational resources on EDs and waste a large amount of computational resources on the cloud server. Deploying NAS on the cloud server can effectively address this issue. The cloud server is used to search for the optimal Subnet, and EDs only need to train the Subnet based on local data. To this end, we propose a nonlinear aggregation-based Neural Architecture Search method based on Feature and Gradient zero-cost proxies (FG-NAS). Specifically, EDs upload local data characteristics to the cloud server, and then the cloud server uses FG-NAS to obtain an optimal SubNet model from the SuperNet based on the uploaded data characteristics. Finally, the cloud server sends the optimal SubNet to EDs, which can reduce the computational burden on EDs. Furthermore, FG-NAS evaluates the accuracy of neural architectures by considering both feature proxies during forward propagation and gradient proxies during backward propagation. Experiments on three datasets demonstrate that compared to current mainstream zero-cost proxy methods, FG-NAS can improve evaluation accuracy by an average of 1.04% and reduce single-network evaluation time by up to 2.45%.

In real-world Industrial Internet of Things (IIoT) scenarios, due to the limited storage capacity of IIoT devices, fresh data continuously received by diverse devices will overwrite the outdated data and change the local data distribution. However, state-of-the-art studies have demonstrated that federated learning tends to focus on training with fresh data, and the latest global model may forget the historical update directions (i.e., catastrophic forgetting). This issue can significantly degrade the global model accuracy. Existing methods primarily focus on integrating outdated data characteristics into fresh data but overlook the large parameter update gap between global and local models during global aggregation. This gap can cause the global model updates to deviate from the optimal direction. To this end, we propose a federated adaptive weighted aggregation method based on model consistency (FedAWAC). Specifically, FedAWAC measures the model consistency on devices and dynamically adjusts the aggregation weights of each local model, thereby guiding the global model toward optimal updates. Furthermore, FedAWAC integrates$\mathcal {M}$historical global models most correlated to the latest global model on the cloud server to overcome catastrophic forgetting. Experiments on four different datasets (nonidentically and independently distributed settings) indicate that compared to five baselines, FedAWAC can improve global model accuracy by an average of 1.86%, reduce the forgetting rate by an average of 3.91%, and save average memory usage by up to 2.57 GB.