Red teaming aims to assess how large language models (LLMs) can produce content that violates norms, policies, and rules set forth during their safety training. However, most existing automated methods in literature are not representative of the way common users exploit the multi-turn conversational nature of AI models. While manual testing addresses this gap, it is an inefficient and often expensive process. To address these limitations, we introduce the Generative Offensive Agent Tester (GOAT), an automated agentic red teaming system that simulates plain language adversarial conversations while leveraging multiple adversarial prompting techniques to identify vuLnerabilities in LLMs. We instantiate GOAT with 7 red teaming attacks by prompting a general purpose model in a way that encourages reasoning through the choices of methods available, the current target model’s response, and the next steps. Our approach is designed to be extensible and efficient, allowing human testers to focus on exploring new areas of risk while automation covers the scaled adversarial stress-testing of known risk territory. We present the design and evaluation of GOAT, demonstrating its effectiveness in identifying vulnerabilities in state-of-the-art LLMs, with an ASR@10 of 96% against smaller models such as Llama 3.1 8B, and 91% against Llama 3.1 70B and 94% for GPT-4o when evaluated against larger models on the JailbreakBench dataset.

Several recent authors have advocated for financial markets to move from continuous clearing to discrete or batched clearing, as a way to defeat the latency arms race: the never-ending quest for small advantages in time to access markets. How frequently should such a modern batch auction clear? We conduct a systematic simulation-based investigation on the relationship between clearing frequency and metrics of market quality, such as allocative efficiency, comparing the performance of discrete and continuous auction mechanisms under empirical equilibrium behavior of all participating traders. In effect we perform empirical mechanism design on frequent batch auctions. We find that in a wide array of environments, equilibrium efficiency is improved for small positive intervals but falls off dramatically when there are too few opportunities to trade. The result is a large range of batch frequencies that are near optimally efficient; this range is wider in thick markets.