Recent years have witnessed a rapid development of mobile GUI agents powered by large language models (LLMs), which can autonomously execute diverse device-control tasks based on natural language instructions. The increasing accuracy of these agents on standard benchmarks has raised expectations for large-scale real-world deployment, and there are already several commercial agents released and used by early adopters. However, are we really ready for GUI agents integrated into our daily devices as system building blocks? We argue that an important pre-deployment validation is missing to examine whether the agents can maintain their performance under real-world threats. Specifically, unlike existing common benchmarks that are based on simple static app contents (they have to do so to ensure environment consistency between different tests), real-world apps are filled with contents from untrustworthy third parties, such as advertisement emails, user-generated posts and medias, etc. These contents may inevitably appear in the agents' observation space and influence the task execution process. Systematic investigation of this problem is challenging since the real-world app contents are significantly skewed—testing on normal real-world apps usually cannot uncover any potential risk since most app contents are benign. To this end, we introduce a scalable app content instrumentation framework to enable flexible and targeted content modifications within existing applications. Leveraging this framework, we create a test suite comprising both a dynamic task execution environment and a static dataset of challenging GUI states. The dynamic environment encompasses 122 reproducible tasks, and the static dataset consists of over 3,000 scenarios constructed from commercial apps. We perform experiments on both open-source and commercial GUI agents. Our findings reveal that all examined agents can be significantly degraded due to third-party contents, with an average misleading rate of 42.0% and 36.1% in dynamic and static environments respectively. The framework and benchmark has been released at https://agenthazard.github.io.

Fairness in clustering has received significant attention. Dickerson et al. in 2023 first proposed the doubly constrained fair clustering problem that aims two fairness constraints, namely, (1) the Group Fairness (GF), which requires that different groups within each cluster have a certain degree of representation, and (2) the Diversity in Center Selection fairness (DS), which requires that the selected centers represent a diverse range of different groups. However, their algorithm only focuses on the k-center objective. In this paper, we generalize the doubly constrained fair clustering to $$\ell _p$$ norm objectives with general p, thus including k-Center, k-Median, and k-Means as special cases. We propose the first approximation algorithm for the doubly constrained fair clustering problem with general p-norms. In polynomial time, our algorithm finds an $$O(\Delta ^{\frac{1}{p}})$$ -approximate clustering that violates the GF constraint by an additive factor of 5 and satisfies the DS constraint, where $$\Delta $$ is the largest size of clusters in the solution. Our main contribution is a novel method to select centers using the min cost network flow approach. Finally, we conduct experiments to validate our algorithm. The experimental results show that the clustering cost of our algorithm, while simultaneously considering both of the GF and DS constraints, is nearly identical to that of the clustering algorithm which only considers the GF constraint.

Graphical User Interface (GUI) agents have gained substantial attention due to their impressive capabilities to complete tasks through multiple interactions within GUI environments.However, existing agents primarily focus on enhancing the accuracy of individual actions and often lack effective mechanisms for detecting and recovering from errors.To address these shortcomings, we propose the BacktrackAgent, a robust framework that incorporates a backtracking mechanism to improve task completion efficiency.BacktrackAgent includes verifier, judger, and reflector components as modules for error detection and recovery, while also applying judgment rewards to further enhance the agent's performance.Additionally, we develop a training dataset specifically designed for the backtracking mechanism, which considers the outcome pages after action executions.Experimental results show that BacktrackAgent has achieved performance improvements in both task success rate and step accuracy on Mobile3M and Auto-UI benchmarks.Our data and code will be released upon acceptance.

Menglong Cui, Pengzhi Gao, Wei Liu, Jian Luan, Bin Wang. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Peng Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Ranran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Sanjay Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip S. Yu, Wenpeng Yin. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Pengzhi Gao, Ruiqing Zhang, Zhongjun He, Hua Wu, Haifeng Wang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

With the development of human-computer interaction and mobile sensors, emotion recognition based on physiological signals has aroused a lively discussion among scholars. The main difficulty faced is the small amount of data which leads to poor training results. In this paper, we proposed a multimodal emotion recognition using CNN-SVM and data augmentation (CSDAMER). Electrocardiography (ECG), galvanic skin response (GSR) and respiration (RSP) are utilized as input data, which are less requiring on the collection environment and can be collected by mobile sensors. To improve the training effect of model, data augmentation is performed by transformations, such as inversion, recombination and noise injection. Moreover, the convolutional layer of the convolutional neural network (CNN) is leveraged to extract the high-level features of the physiological signals, and then the features are input into the support vector machine (SVM) classifier to obtain the recognition results. The experimental results show that CSDAMER achieves 80.7% and 79.92% accuracy in arousal and valance, respectively. Compared with CNN alone, the accuracy of arousal and valance is increased by 12.87% and 9.95%. Meanwhile, the addition of the data augmentation improves the accuracy in arousal and valance by 21.94% and 25.73%.

We introduce Bi-SimCut: a simple but effective training strategy to boost neural machine translation (NMT) performance.It consists of two procedures: bidirectional pretraining and unidirectional finetuning.Both procedures utilize SimCut, a simple regularization method that forces the consistency between the output distributions of the original and the cutoff sentence pairs.Without leveraging extra dataset via back-translation or integrating large-scale pretrained model, Bi-SimCut achieves strong translation performance across five translation benchmarks (data sizes range from 160K to 20.2M): BLEU scores of 31.16 for en → de and 38.37 for de → en on the IWSLT14 dataset, 30.78 for en → de and 35.15 for de → en on the WMT14 dataset, and 27.17 for zh → en on the WMT17 dataset.Sim-Cut is not a new method, but a version of Cutoff (Shen et al., 2020) simplified and adapted for NMT, and it could be considered as a perturbation-based method.Given the universality and simplicity of SimCut and Bi-SimCut, we believe they can serve as strong baselines for future NMT research.

This paper studies the low-rank matrix recovery problem from partially lost and partially corrupted measurements. It shows both analytically and numerically that the recovery performance can be greatly enhanced if one further exploits the temporal correlations among a sequence of low-rank matrices. The matrix recovery problem is formulated as a non-convex optimization problem, and the recovery error is quantified analytically. A fast iterative algorithm is proposed to solve the non-convex problem, and every sequence generated by the algorithm converges to a critical point of the optimization problem. The method is numerically evaluated on the synthetic datasets.