Multi-modal large language model (MLLM)-based web agents interact with webpage environments by generating actions based on screenshots of the webpages. In this work, we propose WebInject, a prompt injection attack that manipulates the webpage environment to induce a web agent to perform an attacker-specified action. Our attack adds a perturbation to the raw pixel values of the rendered webpage. After these perturbed pixels are mapped into a screenshot, the perturbation induces the web agent to perform the attacker-specified action. We formulate the task of finding the perturbation as an optimization problem. A key challenge in solving this problem is that the mapping between raw pixel values and screenshot is non-differentiable, making it difficult to backpropagate gradients to the perturbation. To overcome this, we train a neural network to approximate the mapping and apply projected gradient descent to solve the reformulated optimization problem. Extensive evaluation on multiple datasets shows that WebInject is highly effective and significantly outperforms baselines.

For safety reasons, large language models (LLMs) are trained to refuse harmful user instructions, such as assisting dangerous activities. We study an open question in this work: does the desired safety refusal, typically enforced in chat contexts, generalize to non-chat and agentic use cases? Unlike chatbots, LLM agents equipped with general-purpose tools, such as web browsers and mobile devices, can directly influence the real world, making it even more crucial to refuse harmful instructions. In this work, we primarily focus on red-teaming browser agents – LLMs that leverage information via web browsers. To this end, we introduce Browser Agent Red teaming Toolkit (BrowserART), a comprehensive test suite designed specifically for red-teaming browser agents. BrowserART consists of 100 diverse browser-related harmful behaviors (including original behaviors and ones sourced from HarmBench (Mazeika et al., 2024) and AirBench 2024 (Zeng et al., 2024b)) across both synthetic and real websites. Our empirical study on state-of-the-art browser agents reveals that while the backbone LLM refuses harmful instructions as a chatbot, the corresponding agent does not. Moreover, attack methods designed to jailbreak refusal-trained LLMs in the chat settings transfer effectively to browser agents. With human rewrites, GPT-4o and o1-preview -based browser agents pursued 98 and 63 harmful behaviors (out of 100), respectively. Therefore, simply ensuring LLM’s refusal to harmful instruc- tions in chats is not sufficient to ensure that the downstream agents are safe. We publicly release BrowserART and call on LLM developers, policymakers, and agent developers to collaborate on improving agent safety.

We interact with computers on an everyday basis, be it in everyday life or work, and many aspects of work can be done entirely with access to a computer and the Internet. At the same time, thanks to improvements in large language models (LLMs), there has also been a rapid development in AI agents that interact with and affect change in their surrounding environments. But how performant are AI agents at helping to accelerate or even autonomously perform work-related tasks? The answer to this question has important implications for both industry looking to adopt AI into their workflows, and for economic policy to understand the effects that adoption of AI may have on the labor market. To measure the progress of these LLM agents' performance on performing real-world professional tasks, in this paper, we introduce TheAgentCompany, an extensible benchmark for evaluating AI agents that interact with the world in similar ways to those of a digital worker: by browsing the Web, writing code, running programs, and communicating with other coworkers. We build a self-contained environment with internal web sites and data that mimics a small software company environment, and create a variety of tasks that may be performed by workers in such a company. We test baseline agents powered by both closed API-based and open-weights language models (LMs), and find that with the most competitive agent, 30% of the tasks can be completed autonomously. This paints a nuanced picture on task automation with LM agents -- in a setting simulating a real workplace, a good portion of simpler tasks could be solved autonomously, but more difficult long-horizon tasks are still beyond the reach of current systems. For more information and demos, refer to https://the-agent-company.com.

The piecewise and discontinuous output constraints are prevalent in practical engineering systems but remain insufficiently explored in the literature. Unlike existing results, the discontinuous constraints considered here exhibit two salient characteristics: 1) the bounded constraint boundary function undergoes a jump discontinuity at a specific instant during the initial operational period and 2) the system output becomes unconstrained thereafter. These features render traditional methods inadequate because the derivatives of the constraint boundary function do not exist. To address this problem, we propose a systematic neuroadaptive control framework for nonlinear systems that integrates two novel shift functions with a new barrier Lyapunov function (BLF). The presented control scheme not only ensures good tracking performance under such piecewise and discontinuous output constraints but also can be applied to several other common yet critical constraint scenarios without involving any adjustment to the controller structure. Theoretical analysis and simulation results verify the effectiveness and practicality of the proposed approach.

Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, Daniel Fried. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

With advances in generative AI, there is now potential for autonomous agents to manage daily tasks via natural language commands. However, current agents are primarily created and tested in simplified synthetic environments, leading to a disconnect with real-world scenarios. In this paper, we build an environment for language-guided agents that is highly realistic and reproducible. Specifically, we focus on agents that perform tasks on the web, and create an environment with fully functional websites from four common domains: e-commerce, social forum discussions, collaborative software development, and content management. Our environment is enriched with tools (e.g., a map) and external knowledge bases (e.g., user manuals) to encourage human-like task-solving. Building upon our environment, we release a set of benchmark tasks focusing on evaluating the functional correctness of task completions. The tasks in our benchmark are diverse, long-horizon, and designed to emulate tasks that humans routinely perform on the internet. We experiment with several baseline agents, integrating recent techniques such as reasoning before acting. The results demonstrate that solving complex tasks is challenging: our best GPT-4-based agent only achieves an end-to-end task success rate of 14.41%, significantly lower than the human performance of 78.24%. These results highlight the need for further development of robust agents, that current state-of-the-art large language models are far from perfect performance in these real-life tasks, and that \ours can be used to measure such progress.\footnote{Code, data, environment reproduction instructions, video demonstrations are available in the supplementary.}

LLMs can now act as autonomous agents that interact with digital environments and complete specific objectives (e.g., arranging an online meeting). However, accuracy is still far from satisfactory, partly due to a lack of large-scale, direct demonstrations for digital tasks. Obtaining supervised data from humans is costly, and automatic data collection through exploration or reinforcement learning relies on complex environmental and content setup, resulting in datasets that lack comprehensive coverage of various scenarios. On the other hand, there is abundant knowledge that may indirectly assist task completion, such as online tutorials that were created for human consumption. In this work, we present Synatra, an approach that effectively transforms this indirect knowledge into direct supervision at scale. We define different types of indirect knowledge, and carefully study the available sources to obtain it, methods to encode the structure of direct demonstrations, and finally methods to transform indirect knowledge into direct demonstrations. We use 100k such synthetically-created demonstrations to finetune a 7B CodeLlama, and demonstrate that the resulting agent surpasses all comparably sized models on three web-based task benchmarks Mind2Web, MiniWoB++ and WebArena, as well as surpassing GPT-3.5 on WebArena and Mind2Web. In addition, while synthetic demonstrations prove to be only 3% the cost of human demonstrations (at $0.031 each), we show that the synthetic demonstrations can be more effective than an identical number of human demonstrations collected from limited domains.

Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks either lack an interactive environment or are limited to environments specific to certain applications or domains, failing to reflect the diverse and complex nature of real-world computer use, thereby limiting the scope of tasks and agent scalability. To address this issue, we introduce OSWorld, the first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across various operating systems such as Ubuntu, Windows, and macOS. OSWorld can serve as a unified, integrated computer environment for assessing open-ended computer tasks that involve arbitrary applications. Building upon OSWorld, we create a benchmark of 369 computer tasks involving real web and desktop apps in open domains, OS file I/O, and workflows spanning multiple applications. Each task example is derived from real-world computer use cases and includes a detailed initial state setup configuration and a custom execution-based evaluation script for reliable, reproducible evaluation. Extensive evaluation of state-of-the-art LLM/VLM-based agents on OSWorld reveals significant deficiencies in their ability to serve as computer assistants. While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge. Comprehensive analysis using OSWorld provides valuable insights for developing multimodal generalist agents that were not possible with previous benchmarks. Our code, environment, baseline models, and data are publicly available at this https URL.

Since the rise of neural natural-language-tocode models (NL→Code) that can generate long expressions and statements rather than a single next-token, one of the major problems has been reliably evaluating their generated output.In this paper, we propose CodeBERTScore: an evaluation metric for code generation, which builds on BERTScore (Zhang et al., 2020).Instead of encoding only the generated tokens as in BERTScore, CodeBERTScore also encodes the natural language input preceding the generated code, thus modeling the consistency between the generated code and its given natural language context as well.We perform an extensive evaluation of CodeBERTScore across four programming languages.We find that Code-BERTScore achieves a higher correlation with human preference and with functional correctness than all existing metrics.That is, generated code that receives a higher score by Code-BERTScore is more likely to be preferred by humans, as well as to function correctly when executed.We release five language-specific pretrained models to use with our publicly available code.Our language-specific models have been downloaded more than 1,000,000 times from the Huggingface Hub. 1

Large language models (LLMs) have demonstrated an impressive ability to perform arithmetic and symbolic reasoning tasks, when provided with a few examples at test time ("few-shot prompting"). Much of this success can be attributed to prompting methods such as "chain-of-thought", which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem. While LLMs seem to be adept at this sort of step-by-step decomposition, LLMs often make logical and arithmetic mistakes in the solution part, even when the problem is decomposed correctly. In this paper, we present Program-Aided Language models (PAL): a novel approach that uses the LLM to read natural language problems and generate programs as the intermediate reasoning steps, but offloads the solution step to a runtime such as a Python interpreter. With PAL, decomposing the natural language problem into runnable steps remains the only learning task for the LLM, while solving is delegated to the interpreter. We demonstrate this synergy between a neural LLM and a symbolic interpreter across 13 mathematical, symbolic, and algorithmic reasoning tasks from BIG-Bench Hard and others. In all these natural language reasoning tasks, generating code using an LLM and reasoning using a Python interpreter leads to more accurate results than much larger models. For example, PAL using Codex achieves state-of-the-art few-shot accuracy on GSM8K, surpassing PaLM which uses chain-of-thought by absolute 15% top-1.

Abstract Natural language generation has witnessed significant advancements due to the training of large language models on vast internet-scale datasets. Despite these advancements, there exists a critical challenge: These models can inadvertently generate content that is toxic, inaccurate, and unhelpful, and existing automatic evaluation metrics often fall short of identifying these shortcomings. As models become more capable, human feedback is an invaluable signal for evaluating and improving models. This survey aims to provide an overview of recent research that has leveraged human feedback to improve natural language generation. First, we introduce a taxonomy distilled from existing research to categorize and organize the varied forms of feedback. Next, we discuss how feedback can be described by its format and objective, and cover the two approaches proposed to use feedback (either for training or decoding): directly using feedback or training feedback models. We also discuss existing datasets for human-feedback data collection, and concerns surrounding feedback collection. Finally, we provide an overview of the nascent field of AI feedback, which uses large language models to make judgments based on a set of principles and minimize the need for human intervention. We also release a website of this survey at feedback-gap-survey.info.

This work is concerned with the prescribed performance tracking control for a family of nonlinear nontriangular structure systems under uncertain initial conditions and partial measurable states. By combining neural network and variable separation technique, a state observer with a simple structure is constructed for output-based finite-time tracking control, wherein the issue of algebraic loop arising from a nontriangular structure is circumvented. Meanwhile, by using an error transformation, the developed control scheme is able to ensure tracking with a prescribed accuracy within a pregiven time at a preassigned convergence rate under any bounded initial condition, eliminating the long-standing initial condition dependence issue inherited with conventional prescribed performance control methods, and guaranteeing the predeterminability of convergence time simultaneously. Two simulation examples also demonstrate the effectiveness of the presented control strategy.

This work focuses on the issue of event-triggered practical prescribed time tracking control for a type of uncertain nonlinear systems subject to actuator saturation and unmeasurable states as well as time-varying unknown control coefficients. First, a state observer with simple structure is constructed by means of neural network technology to estimate the unmeasurable system states under time-varying control coefficients. Then, with the help of one-to-one nonlinear mapping of the tracking error, an event-triggered output feedback control scheme is developed to steer the tracking error into a residual set of predefined accuracy within a preassigned settling time. Unlike existing related control methods, there is no need to involve finite-time state observer or fractional power feedback of system states, and thus, the control solution presented here is less complex and more acceptable. The key technique in control design lies in the establishment of an alternative first-order auxiliary system for dealing with the impact arisen from the input saturation. In our proposed approach, a new bounded function related to auxiliary variable and new dynamics of the auxiliary system are skillfully utilized such that the upper bound of the difference between actual input and designed input signal is not involved in implementation of the controller.

Shuyan Zhou, Li Zhang, Yue Yang, Qing Lyu, Pengcheng Yin, Chris Callison-Burch, Graham Neubig. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

We address the general task of structured commonsense reasoning: given a natural language input, the goal is to generate a graph such as an event or a reasoning-graph.To employ large language models (LMs) for this task, existing approaches "serialize" the output graph as a flat list of nodes and edges.Although feasible, these serialized graphs strongly deviate from the natural language corpora that LMs were pre-trained on, hindering LMs from generating them correctly.In this paper, we show that when we instead frame structured commonsense reasoning tasks as code generation tasks, pre-trained LMs of code are better structured commonsense reasoners than LMs of natural language, even when the downstream task does not involve source code at all.We demonstrate our approach across three diverse structured commonsense reasoning tasks.In all these natural language tasks, we show that using our approach, a code generation LM (CODEX) outperforms natural-LMs that are fine-tuned on the target task (e.g., T5) and other strong LMs such as GPT-3 in the few-shot setting.Our code and data are available at https: //github.com/madaan/CoCoGen .

This article investigates the issue of neuroadaptive tracking control for a family of unknown multi-input multi-output (MIMO) nonlinear uncertain systems subject to extreme actuation failures. Different from most existing methods that are built upon partial loss of actuation effectiveness, here, in this article, we explicitly consider the situation that some actuators at some particular channel completely fail to work, an issue that has not been well addressed. By integrating the neural network approximation technique with two error transformations, a neuroadaptive fault-tolerant control strategy is developed with two attractive features: 1) it is capable of coping with the scenario that some of the actuators suffer from extreme actuation faults without the need for fault detection and diagnosis (FDD)/fault detection and isolation (FDI) or actuator switching and 2) the tracking error is forced to converge to a prescribed residual (symmetric or asymmetric) boundary at a preassignable decay mode within a prechosen finite settling time despite actuator failures and external disturbances. Numerical simulation studies confirm the effectiveness and benefits of the presented control approach.

Traditional named entity recognition models use gazetteers (lists of entities) as features to improve performance.Although modern neural network models do not require such handcrafted features for strong performance, recent work (Wu et al., 2018) has demonstrated their utility for named entity recognition on English data.However, designing such features for low-resource languages is challenging, because exhaustive entity gazetteers do not exist in these languages.To address this problem, we propose a method of "soft gazetteers" that incorporates ubiquitously available information from English knowledge bases, such as Wikipedia, into neural named entity recognition models through cross-lingual entity linking.Our experiments on four low-resource languages show an average improvement of 4 points in F1 score. 1

Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts. The first step of (X)EL is candidate generation, which retrieves a list of plausible candidate entities from the target-language KB for each mention. Approaches based on resources from Wikipedia have proven successful in the realm of relatively high-resource languages, but these do not extend well to low-resource languages with few, if any, Wikipedia pages. Recently, transfer learning methods have been shown to reduce the demand for resources in the low-resource languages by utilizing resources in closely related languages, but the performance still lags far behind their high-resource counterparts. In this paper, we first assess the problems faced by current entity candidate generation methods for low-resource XEL, then propose three improvements that (1) reduce the disconnect between entity mentions and KB entries, and (2) improve the robustness of the model to low-resource scenarios. The methods are simple, but effective: We experiment with our approach on seven XEL datasets and find that they yield an average gain of 16.9% in Top-30 gold candidate recall, compared with state-of-the-art baselines. Our improved model also yields an average gain of 7.9% in in-KB accuracy of end-to-end XEL. 1

In this article, we propose a one-step control design approach for pure-feedback nonlinear systems in the presence of unmatched and nonvanishing external disturbances. Different from the commonly utilized backstepping design, the proposed method, integrated with the dynamic surface control (DSC) technique, only involves one-step design with one single Lyapunov function in the whole control synthesis, which derives the actual control and the intermediate controls simultaneously in a collective way, avoiding the repetitive design procedures and multiple Lyapunov functions, yet circumventing the issue of "explosion of complexity." Furthermore, with this method, the increase in system order does not increase the design and analysis complexity. Numerical simulation examples confirm and validate the effectiveness of the proposed method.

The task of entity linking aims to identify concepts mentioned in a text fragments and link them to a reference knowledge base.Entity linking in long text has been well studied in previous work.However, short text entity linking is more challenging since the texts are noisy and less coherent.To better utilize the local information provided in short texts, we propose a novel neural network framework, Aggregated Semantic Matching (ASM), in which two different aspects of semantic information between the local context and the candidate entity are captured via representationbased and interaction-based neural semantic matching models, and then two matching signals work jointly for disambiguation with a rank aggregation mechanism.Our evaluation shows that the proposed model outperforms the state-of-the-arts on public tweet datasets.

This paper studies the prescribed performance tracking control problem for a class of multi-input multi-output strict-feedback systems with asymmetric nonsmooth actuator characteristics and output constraints as well as unexpected external disturbances. By combining a novel speed transformation with barrier Lyapunov function, a neural adaptive control scheme is developed that is able to achieve given tracking precision within preassigned finite time at prespecified converging mode. At each of the first $n-1$ steps of backstepping design, we make use of the radial basis function neural networks to cope with the uncertainties arising from unknown and time-varying virtual control gains, and in the last step, we introduce a matrix factorization technique to remove the restrictive requirement on the unknown control gain matrix and its NN-approximation, simplifying control design. Furthermore, to reduce the number of parameters to be online updated, we introduce a virtual parameter to handle the lumped uncertainties, resulting in a control scheme with low complexity and inexpensive computations. The effectiveness of the proposed control strategy is validated by systematic stability analysis and numerical simulation.