Jie Huang, Xin Liao, Junjie Wang, Mingyang Li, Wenshuo Wang, Ziyou Jiang, Shoubin Li, Qing Wang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Deep reinforcement learning has achieved remarkable success in competitive games, surpassing human performance in applications ranging from business competitions to video games. In competitive environments, agents face the challenge of adapting to continuously shifting adversary strategies, necessitating the ability to handle diverse scenarios. Existing studies primarily focus on evaluating agent robustness either through perturbing observations, which has practical limitations, or through training adversarial agents to expose weaknesses, which lacks strategy diversity exploration. There are also studies which rely on curiosity-based mechanism to explore the diversity, yet they may lack direct guidance to enhance identified decision-making flaws. In this paper, we propose a novel diversity-oriented testing framework (called AdvTest) to test the competitive game agent via constraint-guided adversarial agent training. Specifically, AdvTest adds constraints as the explicit guidance during adversarial agent training to make it capable of defeating the target agent using diverse strategies. To realize the method, three challenges need to be addressed, i.e., what are the suitable constraints, when to introduce constraints, and which constraint should be added. We experimentally evaluate AdvTest on the commonly-used competitive game environment, StarCraft II. The results on four maps show that AdvTest exposes more diverse failure scenarios compared with the commonly-used and state-of-the-art baselines.

Multi-agent systems (MASs) have achieved remarkable success in multi-robot control, intelligent transportation, and multiplayer games, etc. Thorough testing for MAS is urgently needed to ensure its robustness in the face of constantly changing and unexpected scenarios. Existing methods mainly focus on single-agent system testing and cannot be directly applied to MAS testing due to the complexity of MAS. To our best knowledge, there are fewer studies on MAS testing. While several studies have focused on adversarial attacks on MASs, they primarily target failure detection from an attack perspective, i.e., discovering failure scenarios, while ignoring the diversity of scenarios. In this paper, to highlight a typical balance between exploration (diversifying behaviors) and exploitation (detecting failures), we propose an advanced testing framework for MAS called with diversity-guided exploration and adaptive critical state exploitation. It incorporates both individual diversity and team diversity, and designs an adaptive perturbation mechanism to perturb the action at the critical states, so as to trigger more and more diverse failure scenarios of the system. We evaluate MASTest on two popular MAS simulation environments: Coop Navi and StarCraft II. Results show that the average distance of the resulting failure scenarios is increased by 29.55%-103.57% and 74.07%-370.00% on two environments compared to the baselines. Also, the failure patterns found by MASTest are improved by 71.44%-300.00% and 50%-500.00% on two experimental environments compared to the baselines.

Requirements linking is the core of software system maintenance and evolution, and it is critical to assuring software quality. In practice, however, the requirements links are frequently absent or incorrectly labeled, and reconstructing such ties is time-consuming and error-prone. Numerous learning-based approaches have been put forth to address the problem. However, these approaches will lose effectiveness for the Cold-Start projects with few labeled samples. To this end, we propose RADIATION, an adversarial-based domain adaptation approach for cross-domain requirements linking. Generally, RADIATION firstly adopts an IDF-based Masking strategy to filter the domain-specific features. Then it pre-trains a linking model in the source domain with sufficient labeled samples and adapts the model to target domains using a distance-enhanced adversarial technique without using any labeled target samples. Evaluation on five public datasets shows that RADIATION could achieve 66.4% precision, 89.2% recall, and significantly outperform state-of-the-art baselines by 13.4% -42.9% F1. In addition, the designed components, i.e., IDF-based Masking and Distance-enhanced Loss, could significantly improve performance.

As the bridge between users and software, Graphical User Interface (GUI) is critical to the app accessibility. Scaling up the font or display size of GUI can help improve the visual impact, readability, and usability of an app, and is frequently used by the elderly and people with vision impairment. Yet this can easily lead to scaling issues such as text truncation, component overlap, which negatively influence the acquirement of the right information and the fluent usage of the app. Previous techniques for UI display issue detection and cross-platform inconsistency detection cannot work well for these scaling issues. In this paper, we propose an automated method, dVermin, for scaling issue detection, through detecting the inconsistency of a view under the default and a larger display scale. The evaluation result shows that dVermin achieves 97% precision and 97% recall in issue page detection, and 84% precision and 91% recall for issue view detection, outperforming two state-of-the-art baselines by a large margin. We also evaluate dVermin with popular Android apps on F-droid, and successfully uncover 21 previously-undetected scaling issues with 20 of them being confirmed/fixed.

Natural language (NL) documentation is the bridge between software managers and testers, and NL test cases are prevalent in system-level testing and other quality assurance activities. Due to reasons such as requirements redundancy, parallel testing, tester turn-over within long evolving history, there are inevitably lots of redundant test cases, which significantly increase the cost. Previous redundancy detection approaches typically treat the textual descriptions as a whole to compare their similarity and suffer from low precision. Our observation reveals that a test case can have explicit test-oriented entities, such as tested function Components, Constraints, etc; and there are also specific relations between these entities. This inspires us with a potential opportunity for accurate redundancy detection. In this paper, we first define five test-oriented entity categories and four associated relation categories, and re-formulate the NL test case redundancy detection problem as the comparison of detailed testing content guided by the test-oriented entities and relations. Following that, we propose Tscope, a fine-grained approach for redundant NL test case detection by dissecting test cases into atomic test tuple(s) with the entities restricted by associated relations. To serve as the test case dissection, Tscope designs a context-aware model for the automatic entity and relation extraction. Evaluation on 3,467 test cases from ten projects shows Tscope could achieve 91.8% precision, 74.8% recall and 82.4% F1, significantly outperforming state-of-the-art approaches and commonly-used classifiers. This new formulation of the NL test case redundant detection problem can motivate the follow-up studies in further improving this task and other related tasks involving NL descriptions.

Online chatting is gaining popularity and plays an increasingly significant role in software development. When discussing functionalities, developers might reveal their desired features to other developers. Automated mining techniques towards retrieving feature requests from massive chat messages can benefit the requirements gathering process. But it is quite challenging to perform such techniques because detecting feature requests from dialogues requires a thorough understanding of the contextual information, and it is also extremely expensive on annotating feature-request dialogues for learning. To bridge that gap, we recast the traditional text classification task of mapping single dialog to its class into the task of determining whether two dialogues are similar or not by incorporating few-shot learning. We propose a novel approach, named FRMiner, which can detect feature-request dialogues from chat messages via deep Siamese network. We design a BiLSTM-based dialog model that can learn the contextual information of a dialog in both forward and reverse directions. Evaluation on the real-world projects shows that our approach achieves average precision, recall and F1-score of 88.52%, 88.50% and 88.51%, which confirms that our approach could effectively detect hidden feature requests from chat messages, thus can facilitate gathering comprehensive requirements from the crowd in an automated way.

Peer code review has been proven to be an effective practice for quality assurance, and widely adopted by commercial companies and open source communities as GitHub. However, identifying an appropriate code reviewer for a pull request is a non-trivial task considering the large number of candidate reviewers. Several approaches have been proposed for reviewer recommendation, yet none of them has conducted a complete comparison to explore which one is more effective. This paper aims at conducting an experimental evaluation of the commonly-used and state-of-the-art approaches for code reviewer recommendation. We begin with a systematic review of approaches for code reviewer recommendation, and choose six approaches for experimental evaluation. We then implement these approaches and conduct reviewer recommendation on 12 large-scale open source projects with 53,005 pull requests spanning two years. Results show that there is no golden rule when selecting code reviewer recommendation approaches, and the best approach varies in terms of different evaluation metrics (e.g., Top-5 Accuracy, MRR) and experimental projects. Nevertheless, TIE, which utilizes the textual similarity and file path similarity, is the most promising one. We also explore the sensitivity of these approaches to training data, and compare their time cost. This approach provides new insights and practical guidelines for choosing approaches for reviewer recommendation.

In open software development environment, a large number of feature requests with mixed quality are often posted by stakeholders and usually managed in issue tracking systems. Thoroughly understanding and analyzing the real intents that feature requests imply is a labor-intensive and challenging task. In this paper, we introduce an approach to understand feature requests automatically. We generate a set of fuzzy rules based on natural language processing techniques that classify each sentence in feature requests into a set of categories: Intent, Explanation, Benefit, Drawback, Example and Trivia. Consequently, the feature requests can be automatically structured based on the classification results. We conduct experiments on 2,112 sentences taken from 602 feature requests of nine popular open source projects. The results show that our method can reach a high performance on classifying sentences from feature requests. Moreover, when applying fuzzy rules on machine learning methods, the performance can be improved significantly.