Pareto Front Learning (PFL) has been one of the effective means to resolve multi-objective optimization problems through exploring all optimal solutions to learn the entire Pareto front. Pareto Hypernetwork (PHN) is a new promising way to generate the sequence of Pareto-optimal solutions that can be further used as potential solutions to constitute the Pareto front. However, the existing PHN-based approaches suffer from two performance issues: They take as inputs human-crafted preference vector or chunk embedding, rather than the input data samples, and thus vulnerable to data distribution shifts. Such approaches cannot optimize all potential solutions when forming the Pareto front, as they merely optimize the loss pertaining to one single input at a time of optimization round. To improve the quality of the Pareto front, we propose IOP, a novel Idempotent-like Optimization method to learn the entire Pareto front accurately and enhance Hypernetwork's adaptability to distribution shifts. In particular, IOP performs idempotent-like optimization by exploiting manifold space mapping, so that the target networks generated by the optimized Hypernetwork can effectively handle samples with similar distributions of the input samples, without the pre-defined human-crafted inputs. IOP maximizes the Hypervolume indicator that is composed of all potential solutions at a higher level. Experimental results demonstrate that IOP outperforms the state-of-the-art methods by 4.7% on average in producing the Pareto front and has a 10.5% improvement in adaptability.

Continuous learning faces the challenge of catastrophic forgetting. Our research findings indicate that in unsupervised federated continual learning (UFCL), the limited model capacity and interference among participants are the key factors contributing to this problem. Specifically, the fixed capacity of the model restricts its ability to retain historical knowledge. Besides, the indiscriminate aggregation of weights from multiple participants can cause interference, damaging the model memory. To address these challenges, we propose FedFRR, a federated anti-forgetting representation learning approach. FedFRR fits the participants’ data distribution through a weighted combination of primary network units (PNU) in the model and optimizes model memory by adjusting the structure of PNUs. Additionally, FedFRR addresses interference by truncating the PNU with less weight change, thus reducing the scope of weight aggregation. The experimental results demonstrate that FedFRR achieves state-of-the-art performance, significantly enhancing the model’s anti-forgetting ability.

In federated learning, the non-IID data generated from heterogeneous clients may reduce the global model efficiency. Previous studies use personalization as a common approach to adapt the global model to these clients (called the local model). However, client’s data distribution may change dynamically with its location or environment, which can degrade the performance of the local model, leading to a new Dynamic Personalized Federated Learning (DPFL) problem. This paper proposes a novel approach to reduce the impact of the dynamic distribution on the local model based on metalearning and distribution distance measurement named Fed-3DA. It calculates the distribution distance periodically to perceive the distribution change on the client and adjust the local model preferences from a global meta-model through the distribution representation. Our experiments on public datasets show that Fed-3DA can effectively reduce the performance fluctuation of the local model in DPFL scenarios.