Task-oriented LLM-based agents are increasingly used in domains with strict policies, such as refund eligibility or cancellation rules.The challenge lies in ensuring that the agent consistently adheres to these rules and policies, appropriately refusing any request that would violate them, while still maintaining a helpful and natural interaction.This calls for the development of tailored design and evaluation methodologies to ensure agent resilience against malicious user behavior.We propose a novel threat model that focuses on adversarial users aiming to exploit policy-adherent agents for personal benefit.To address this, we present CRAFT, a multi-agent red-teaming system that leverages policy-aware persuasive strategies to undermine a policy-adherent agent in a customerservice scenario, outperforming conventional jailbreak methods such as DAN prompts, emotional manipulation, and coercive.Building upon the existing τ -bench benchmark, we introduce τ -break, a complementary benchmark designed to rigorously assess the agent's robustness against manipulative user behavior.Finally, we evaluate several straightforward yet effective defense strategies.While these measures provide some protection, they fall short, highlighting the need for stronger, researchdriven safeguards to protect policy-adherent agents from adversarial attacks. 1 .

Dialog is a core building block of human natural language interactions.It contains multiparty utterances used to convey information from one party to another in a dynamic and evolving manner.The ability to compare dialogs is beneficial in many real world use cases, such as conversation analytics for contact center calls and virtual agent design.We propose a novel adaptation of the edit distance metric to the scenario of dialog similarity.Our approach takes into account various conversation aspects such as utterance semantics, conversation flow, and the participants.We evaluate this new approach and compare it to existing document similarity measures on two publicly available datasets.The results demonstrate that our method outperforms the other approaches in capturing dialog flow, and is better aligned with the human perception of conversation similarity.

Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-tuning a state-of-the-art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers' performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-the-art techniques for data augmentation, specifically those applicable to text classification tasks with little data.

Decision makers often need to take into account multiple conflicting objectives when selecting a solution for their problem. This can result in a potentially large number of candidate solutions to be considered. Visualizing a Pareto Frontier, the optimal set of solutions to a multi-objective problem, is considered a difficult task when the problem at hand spans more than three objective functions. We introduce a novel visual-interactive approach to facilitate coping with multi-objective problems. We propose a characterization of the Pareto Frontier data and the tasks decision makers face as they reach their decisions. Following a comprehensive analysis of the design alternatives, we show how a semantically-enhanced Self-Organizing Map, can be utilized to meet the identified tasks. We argue that our newly proposed design provides both consistent orientation of the 2D mapping as well as an appropriate visual representation of individual solutions. We then demonstrate its applicability with two real-world multi-objective case studies. We conclude with a preliminary empirical evaluation and a qualitative usefulness assessment.

A serious tool gap exists at the start of the software lifecy-cle, before requirements formulation. Pre-requirements analysts gather information, organize it to gain insight, en-vision possible futures, and present insights and recom-mendations to stakeholders. They typically use office tools, which give great freedom, but no help with consistency management, change propagation, or information migration to downstream tools. Despite these downsides, office tools are still favored over modeling tools, which are constrain-ing and difficult to use. We introduce the notion of flexible modeling tools, which blend the advantages of office and modeling tools. We propose a conceptual architecture for such tools, and outline research challenges to be met in realizing them. We briefly describe the Business Insight Toolkit, a prototype tool embodying this architecture.

Diagrams play a key role in the information systems domain. However to be meaningful, the diagrams are understood by interpreting visual cues in specific, conventionalized ways, termed conceptual models. One of the major pain points of conceptual models, specified as visual languages, is the inability to capture these visual languages effectively in conventional modeling tools. Instead, conceptual models are drawn using drawing tools and sometimes even by hand. We propose an automatic procedure to derive the syntactic building blocks of graph-based conceptual models. This high-level specification of the visual language can then serve as input for the automatic construction of syntax-aware diagram editors. Our aim is to achieve minimum effort on the part of the users when they eventually work with the graphical editor to produce a new diagram using the proposed syntax.

Schema integration is the process by which schemata from heterogeneous databases are conceptually integrated into a single cohesive schema. In this work we propose a modeling framework for schema integration, capturing the inherent uncertainty accompanying the integration process. The model utilizes a fuzzy framework to express a confidence measure, associated with the outcome of a schema integration process. In this paper we provide a systematic analysis of the process properties and establish a criterion for evaluating the quality of matching algorithms, which map attributes among heterogeneous schemata.