The performance of large language model (LLM) inference engines heavily relies on deployment-specific parameters such as batch size, parallelism, CUDA graphs, and GPU and network topology, however, with every new LLM released, the optimal setting of these parameters becomes a problem of interest. Additionally, as developers find new applications for LLMs, the parameter space only increases, with large scale inference deployments that span multiple nodes (i.e. llm-d [5]). In this work, we demonstrate ''llm-tuna'', an open-source framework that automates parameter search for vLLM [4] based inference workloads across a diverse set of hardware setups. Our framework utilizes Bayesian optimization via Optuna to select the best configuration while parallelizing trials across multiple nodes to enable large scale studies.

We propose to demonstrate the design, implementation, and the capabilities of an anomaly detection (AD) system integrated with a relational database management system (DBMS). Our AD system is trained by extracting relevant features from the parse-tree representation of the SQL commands, and then uses the DBMS roles as the classes for the bayesian classifier. In the detection phase, the maximum apriori probability role is chosen by the classifier which, if not matching the role associated with the SQL command, raises an alarm. We have implemented such system in the PostgreSQL DBMS, integrated with the statistics collection and the query processing mechanism of the DBMS. During the demonstration, our audience will be given the choice of training our system using either synthetic role-based SQL query traces based on probability sampling, or by entering their own set of training queries. In the subsequent detection mode, the audience can test the detection capabilities of the system by submitting arbitrary SQL commands. We will also allow the audience to generate arbitrary work loads to measure the overhead of the training phase and the detection phase of our AD mechanism on the performance of the DBMS.

The intrusion response component of an overall intrusion detection system is responsible for issuing a suitable response to an anomalous request. We propose the notion of database response policies to support our intrusion response system tailored for a DBMS. Our interactive response policy language makes it very easy for the database administrators to specify appropriate response actions for different circumstances depending upon the nature of the anomalous request. The two main issues that we address in context of such response policies are that of policy matching, and policy administration. For the policy matching problem, we propose two algorithms that efficiently search the policy database for policies that match an anomalous request. We also extend the PostgreSQL DBMS with our policy matching mechanism, and report experimental results. The experimental evaluation shows that our techniques are very efficient. The other issue that we address is that of administration of response policies to prevent malicious modifications to policy objects from legitimate users. We propose a novel Joint Threshold Administration Model (JTAM) that is based on the principle of separation of duty. The key idea in JTAM is that a policy object is jointly administered by at least k database administrator (DBAs), that is, any modification made to a policy object will be invalid unless it has been authorized by at least k DBAs. We present design details of JTAM which is based on a cryptographic threshold signature scheme, and show how JTAM prevents malicious modifications to policy objects from authorized users. We also implement JTAM in the PostgreSQL DBMS, and report experimental results on the efficiency of our techniques.

Countering threats to an organization's internal databases from database applications is an important area of research. In this paper, we propose a novel framework based on anomaly detection techniques, to detect malicious behaviour of database application programs. Specifically, we create a fingerprint of an application program based on SQL queries submitted by it to a database. We then use association rule mining techniques on this fingerprint to extract useful rules. These rules succinctly represent the normal behaviour of the database application. We then apply an anomaly detection algorithm to detect queries that do not conform to these rules. We further demonstrate how this model can be used to detect SQL Injection attacks on databases. We show the validity and usefulness of our approach on synthetically generated datasets and SQL Injected queries. Experimental results show that our techniques are effective in addressing various types of SQL Injection threat scenarios.

A considerable effort has been recently devoted to the development of database management systems (DBMS) which guarantee high assurance security and privacy. An important component of any strong security solution is represented by intrusion detection (ID) systems, able to detect anomalous behavior by applications and users. To date, however, there have been very few ID mechanisms specifically tailored to database systems. In this paper, we propose such a mechanism. The approach we propose to ID is based on mining database traces stored in log files. The result of the mining process is used to form user profiles that can model normal behavior and identify intruders. An additional feature of our approach is that we couple our mechanism with role based access control (RBAC). Under a RBAC system permissions are associated with roles, usually grouping several users, rather than with single users. Our ID system is able to determine role intruders, that is, individuals that while holding a specific role, have a behavior different from the normal behavior of the role. An important advantage of providing an ID mechanism specifically tailored to databases is that it can also be used to protect against insider threats. Furthermore, the use of roles makes our approach usable even for databases with large user population. Our preliminary experimental evaluation on both real and synthetic database traces show that our methods work well in practical situations