Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Zhuoer Wang, Marcus Collins, Nikhita Vedula, Simone Filice, Shervin Malmasi, Oleg Rokhlenko. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

We present a new benchmark dataset called PARADE for paraphrase identification that requires specialized domain knowledge.PA-RADE contains paraphrases that overlap very little at the lexical and syntactic level but are semantically equivalent based on computer science domain knowledge, as well as nonparaphrases that overlap greatly at the lexical and syntactic level but are not semantically equivalent based on this domain knowledge.Experiments show that both state-of-the-art neural models and non-expert human annotators have poor performance on PARADE.For example, BERT after fine-tuning achieves an F1 score of 0.709, which is much lower than its performance on other paraphrase identification datasets.PARADE can serve as a resource for researchers interested in testing models that incorporate domain knowledge.We make our data and code freely available.1