Model editing plays a crucial role in the cost-effective development of large language models, and the challenge of evolving knowledge facilitates its sequential extension, namely lifelong model editing. However, progress on standard and lifelong editing has historically followed separate tracks, overlooking the potential of generalizing standard methods to lifelong scenarios. By establishing this bridge, we can provide robust baselines in lifelong scenarios and ensure that lifelong editing benefits from the ongoing advancements in standard editing technologies. In response, this paper proposes a general framework, Sim*ulating Ideal E*ditor (SimIE), which restores the strong performance of parameter-modifying methods from standard model editing in a lifelong context. SimIE formulates the ideal parameter shift as the minimum-norm solution to a linear system, constructed using the Moore-Penrose inverse, and subsequently enables recursive updates by truncating the limiting expression of the Moore-Penrose inverse under two mild assumptions. Theoretically, we demonstrate that if either assumption is not met, the solution provided by SimIE remains near-optimal in a statistical sense or stable against perturbations introduced by the sequential editing, but a trade-off between optimality and stability arises when both assumptions fail. Extensive experiments validate the effectiveness of SimIE, which allows standard algorithms to achieve performance comparable to specialized lifelong model editing methods. Our code is available at https://github.com/YamingGuo98/SimIE.

The increasingly popular federated learning still faces the practical challenge of non-independent and identically distributed data. Most efforts to address this issue focus on limiting local updates or enhancing model aggregation. However, these methods either restrict the learning capacity of local models or overlook the negative knowledge transfer caused by local objective divergences. In contrast, we observe that the global update can be re-expressed as a weighted sum of the gradients of samples from different classes. Therefore, we hypothesize that the competition among local updates may arise from the conflict between the gradients of samples belonging to different classes. Inspired by this insight, we introduce the novel perspective of orthogonal class gradients, aimed at eliminating interference between updates from different classes without the aforementioned drawbacks. To this end, this paper presentsFedOCF, which implements orthogonal class gradient constraints by encouraging orthogonality among features of different classes. Specifically,FedOCFmaintains a generator to learn features that are orthogonal for different classes and utilizes it to regularize features learned in local learning. Theoretically, we also demonstrate thatFedOCFcan improve generalization performance through feature conditional distribution alignment during local learning. Extensive experiments validate the excellent performance ofFedOCFin various heterogeneous scenarios.