Adversarial training has been proven to be a powerful regularization technique to improve language models. In this work, we propose a novel randomdropped weightattackadversarial training method (DropAttack) for natural language understanding. Our DropAttack improves the generalization of models by minimizing the internal adversarial risk caused by a multitude of attack combinations. Specifically, DropAttack enhances the adversarial attack space by intentionally adding worst-case adversarial perturbations to the weight parameters and randomly dropping the specific proportion of attack perturbations. To extensively validate the effectiveness of DropAttack,12public English natural language understanding datasets were used. Experiments on the GLUE benchmark show that when DropAttack is applied only to the finetuning stage, it is able to improve the overall test scores of the BERT-base pre-trained model from 78.3 to 79.7 and RoBERTa-large pre-trained model from 88.1 to 88.8. Further, DropAttack also significantly improves models trained from scratch. Theoretical analysis reveals that DropAttack performs potential gradient regularization on the input and weight parameters of the model. Moreover, visualization experiments show that DropAttack can push the minimum risk of the neural network to a lower and flatter loss landscape.

As one of the most popular social media platforms, Twitter has become a primary source of information for many people.Unfortunately, both valid information and rumors are propagated on Twitter due to the lack of an automatic information verification system.Twitter users communicate by replying to other users' messages, forming a conversation structure.Using this structure, users can decide whether the information in the source tweet is a rumor by reading the tweet's replies, which voice other users' stances on the tweet.The majority of rumor detection researchers process such tweets based on time, ignoring the conversation structure.To reap the benefits of the Twitter conversation structure, we developed a model to detect rumors by modeling conversation structure as a graph.Thus, our model's improved representation of the conversation structure enhances its rumor detection accuracy.The experimental results on two rumor datasets show that our model outperforms several baseline models, including a state-of-theart model.