Sequential knowledge editing techniques aim to continuously update knowledge in large language models at low cost, preventing models from generating outdated or incorrect information. However, existing sequential editing methods suffer from a significant decline in editing success rates after long-term editing. Through theoretical analysis and experiments, our findings reveal that as the number of edits increases, the model's output increasingly deviates from the desired target, leading to a drop in editing success rates. We refer to this issue as the superimposed noise accumulation problem. Our further analysis demonstrates that the problem is related to the erroneous activation of irrelevant knowledge and conflicts between activated knowledge. Based on this analysis, a method named DeltaEdit is proposed that reduces conflicts between knowledge through dynamic orthogonal constraint strategies. Experiments show that DeltaEdit significantly reduces superimposed noise, achieving a 16.8% improvement in editing performance over the strongest baseline.

Data augmentation through linearly interpolating inputs and modeling targets of random samples has significantly improved predictive performance.However, data augmentation based on linear interpolation generates semantically cluttered and ambiguous text, resulting in ineffective augmentation.To address these issues, in this paper, we propose a novel data augmentation approach called Attention-based Smooth Data Augmentation (ASMix).ASMix accepts the smoothed embeddings of pairwise data predicted by a masked language model (MLM) instead of one-hot embeddings, which makes the inputs more informative and context-rich.We employ the attention mechanism to select discriminative and more-attentioned parts of text hidden representations and mix up the parts containing key semantics in the hidden representations of pairwise data through a multi-token replacement strategy to augment the data of the minority class, which greatly reduces redundant information in the representations that hurts the performance of the model.On several public imbalanced text classification benchmarks, ASMix outperforms state-of-the-art data augmentation methods.In minority classes, the performance improvement of ASMix is particularly prominent.