The safety and robustness of Large Language Models (LLMs) are major challenges in developing generative AI applications. One key issue is the vulnerability to prompt jailbreak attacks, which pose a significant threat to building secure and resilient LLM-based applications. In this work, we present a framework for understanding and evaluating the behaviors of popular LLMs by categorizing their responses into five distinct exposure levels. Additionally, we introduce a novel language attack that circumvents LLMs’ defenses by translating jailbreak prompts into languages such as Arabic, Chinese, and Greek. Despite ongoing efforts to enhance LLMs’ safety, we find that nearly all popular LLMs can be jailbroken. Our findings offer detailed insights into LLMs’ behavior, improve diagnostic capabilities, and support targeted safety improvements.

The safety and robustness of large language models (LLMs) based applications remain critical challenges in artificial intelligence. Among the key threats to these applications are prompt hacking attacks, which can significantly undermine the security and reliability of LLM-based systems. In this work, we offer a comprehensive and systematic overview of three distinct types of prompt hacking: jailbreaking, leaking, and injection, addressing the nuances that differentiate them despite their overlapping characteristics. To enhance the evaluation of LLM-based applications, we propose a novel framework that categorizes LLM responses into five distinct classes, moving beyond the traditional binary classification. This approach provides more granular insights into the AI’s behavior, improving diagnostic precision and enabling more targeted enhancements to the system’s safety and robustness.