Graphical User Interface (GUI) agents have gained substantial attention due to their impressive capabilities to complete tasks through multiple interactions within GUI environments.However, existing agents primarily focus on enhancing the accuracy of individual actions and often lack effective mechanisms for detecting and recovering from errors.To address these shortcomings, we propose the BacktrackAgent, a robust framework that incorporates a backtracking mechanism to improve task completion efficiency.BacktrackAgent includes verifier, judger, and reflector components as modules for error detection and recovery, while also applying judgment rewards to further enhance the agent's performance.Additionally, we develop a training dataset specifically designed for the backtracking mechanism, which considers the outcome pages after action executions.Experimental results show that BacktrackAgent has achieved performance improvements in both task success rate and step accuracy on Mobile3M and Auto-UI benchmarks.Our data and code will be released upon acceptance.

Qinzhuo Wu, Wei Liu, Jian Luan, Bin Wang. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Recently, tool-augmented LLMs have gained increasing attention.Given an instruction, toolaugmented LLMs can interact with various external tools in multiple rounds and provide a final answer.However, previous LLMs were trained on overly detailed instructions, which included API names or parameters, while real users would not explicitly mention these API details.This leads to a gap between trained LLMs and real-world scenarios.In addition, most works ignore whether the interaction process follows the instruction.To address these issues, we constructed a training dataset called MGToolBench, which contains statement and category-level instructions to better reflect realworld scenarios.In addition, we propose Tool-Planner, a two-stage reinforcement learning framework that utilizes path planning and two feedback mechanisms to enhance the LLM's task completion and instruction-following capabilities.Experimental results show that Tool-Planner significantly improves the Match Rate, Pass Rate and Win Rate by 26.8%, 20.2%, and 5.6% compared to the SOTA model.Human evaluation verifies that the multi-granularity instructions can better align with users' usage habits.Our data and code are available at https://github.com/XiaoMi/toolplanner.

Taking several topic words and a math expression as input, the aim of math word problem generation is to generate a problem that can be answered by the given expression and related to these topic words. Considerable progress has been achieved by sequence-to-sequence neural network models in many natural language generation tasks, but these models do not effectively consider the characteristics of the math word problem generation task. They may generate problems that are unrelated to the topic words and expressions, and problems that cannot be solved. In this paper, we propose a new model, MWPGen, for automatically generating math word problems. MWPGen has a topic-expression co-attention mechanism to extract relevant information between topic words and expressions. Further, we fine-tune MWPGen with the solving result of the generated problem as the reward for reinforcement learning. MWPGen shows improved performance in popular automatic evaluation metrics and improves the solvability of generated problems.

Qinzhuo Wu, Qi Zhang, Zhongyu Wei, Xuanjing Huang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

With the advancements in natural language processing tasks, math word problem solving has received increasing attention.Previous methods have achieved promising results but ignore background common-sense knowledge not directly provided by the problem.In addition, during generation, they focus on local features while neglecting global information.To incorporate external knowledge and global expression information, we propose a novel knowledge-aware sequence-to-tree (KA-S2T) network in which the entities in the problem sequences and their categories are modeled as an entity graph.Based on this entity graph, a graph attention network is used to capture knowledge-aware problem representations.Further, we use a tree-structured decoder with a state aggregation mechanism to capture the long-distance dependency and global expression information.Experimental results on the Math23K dataset revealed that the KA-S2T model can achieve better performance than previously reported best results.

Expert finding is a task designed to enable recommendation of the right person who can provide high-quality answers to a requester's question. Most previous works try to involve a content-based recommendation, which only superficially comprehends the relevance between a requester's question and the expertise of candidate experts by exploring the content or topic similarity between the requester's question and the candidate experts' historical answers. However, if a candidate expert has never answered a question similar to the requester's question, then existing methods have difficulty making a correct recommendation. Therefore, exploring the implicit relevance between a requester's question and a candidate expert's historical records by perception and reasoning should be taken into consideration. In this study, we propose a novel \textslrecurrent memory reasoning network (RMRN) to perform this task. This method focuses on different parts of a question, and accordingly retrieves information from the histories of the candidate expert.Since only a small percentage of historical records are relevant to any requester's question, we introduce a Gumbel-Softmax-based mechanism to select relevant historical records from candidate experts' answering histories. To evaluate the proposed method, we constructed two large-scale datasets drawn from Stack Overflow and Yahoo! Answer. Experimental results on the constructed datasets demonstrate that the proposed method could achieve better performance than existing state-of-the-art methods.