Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods. To address this issue, we introduce HarmBench, a standardized evaluation framework for automated red teaming. We identify several desirable properties previously unaccounted for in red teaming evaluations and systematically design HarmBench to meet these criteria. Using HarmBench, we conduct a large-scale comparison of 18 red teaming methods and 33 target LLMs and defenses, yielding novel insights. We also introduce a highly efficient adversarial training method that greatly enhances LLM robustness across a wide range of attacks, demonstrating how HarmBench enables codevelopment of attacks and defenses. We open source HarmBench at https://github.com/centerforaisafety/HarmBench.

We present a nonparametric statistical framework for the quantification, analysis, and propagation of data uncertainty in direct volume rendering (DVR). The state-of-the-art statistical DVR framework allows for preserving the transfer function (TF) of the ground truth function when visualizing uncertain data; however, the existing framework is restricted to parametric models of uncertainty. In this paper, we address the limitations of the existing DVR framework by extending the DVR framework for nonparametric distributions. We exploit the quantile interpolation technique to derive probability distributions representing uncertainty in viewing-ray sample intensities in closed form, which allows for accurate and efficient computation. We evaluate our proposed nonparametric statistical models through qualitative and quantitative comparisons with the mean-field and parametric statistical models, such as uniform and Gaussian, as well as Gaussian mixtures. In addition, we present an extension of the state-of-the-art rendering parametric framework to 2D TFs for improved DVR classifications. We show the applicability of our uncertainty quantification framework to ensemble, downsampled, and bivariate versions of scalar field datasets.

Sparse signal recovery from limited and/or degraded samples is fundamental to many applications, such as medical imaging, remote sensing, astronomical and seismic imaging. Discrete wavelet transform (DWT) has been commonly used for sparse representation of signals; nevertheless, due to its shift-variant nature, pseudo-Gibbs artifacts are present in the recovered signals. Using the redundant shift-invariant wavelet transform (SWT) is the ideal solution to obtain shift invariance; however, high redundancy factor of SWT limits its application in practical settings. We propose a dictionary splitting approach for sparse recovery from incomplete data, which leverages the ideas of cycle spinning in combination with Bregman splitting. The proposed method significantly improves the conventional signal reconstruction with DWT, offers the advantages of SWT, and overcomes high redundancy factor of SWT. We solve parallel sparse recovery problems with orthogonal dictionaries (DWT and its permuted versions), while we impose consistency between the results by updating the recovered image at each iteration. Our experiments demonstrate that few shifts are sufficient to achieve reconstruction accuracy as high as recovery with SWT, and significantly reduces its computational cost and redundancy factor.

With uncertainty present in almost all modalities of data acquisition, reduction, transformation, and representation, there is a growing demand for mathematical analysis of uncertainty propagation in data processing pipelines. In this paper, we present a statistical framework for quantification of uncertainty and its propagation in the main stages of the visualization pipeline. We propose a novel generalization of Irwin-Hall distributions from the statistical viewpoint of splines and box-splines, that enables interpolation of random variables. Moreover, we introduce a probabilistic transfer function classification model that allows for incorporating probability density functions into the volume rendering integral. Our statistical framework allows for incorporating distributions from various sources of uncertainty which makes it suitable in a wide range of visualization applications. We demonstrate effectiveness of our approach in visualization of ensemble data, visualizing large datasets at reduced scale, iso-surface extraction, and visualization of noisy data.

The problem of isosurface extraction in uncertain data is an important research problem and may be approached in two ways. One can extract statistics (e.g., mean) from uncertain data points and visualize the extracted field. Alternatively, data uncertainty, characterized by probability distributions, can be propagated through the isosurface extraction process. We analyze the impact of data uncertainty on topology and geometry extraction algorithms. A novel, edge-crossing probability based approach is proposed to predict underlying isosurface topology for uncertain data. We derive a probabilistic version of the midpoint decider that resolves ambiguities that arise in identifying topological configurations. Moreover, the probability density function characterizing positional uncertainty in isosurfaces is derived analytically for a broad class of nonparametric distributions. This analytic characterization can be used for efficient closed-form computation of the expected value and variation in geometry. Our experiments show the computational advantages of our analytic approach over Monte-Carlo sampling for characterizing positional uncertainty. We also show the advantage of modeling underlying error densities in a nonparametric statistical framework as opposed to a parametric statistical framework through our experiments on ensemble datasets and uncertain scalar fields.

Signal reconstruction from the smallest possible Fourier measurements has been a key motivation in the compressed sensing research. We present an approach that exploits the interdependency and structural sparsity of partial derivatives for lowering the sampling rates necessary for accurate reconstruction. Our experiments show that for signals that are sparse in the gradient domain our proposed method significantly outperforms the existing approaches including the total variation (TV) based CS reconstruction.

Abstract In this paper, we investigate compressed sensing principles to devise an in‐situ data reduction framework for visualization of volumetric datasets. We exploit the universality of the compressed sensing framework and show that the proposed method offers a refinable data reduction approach for volumetric datasets. The accurate reconstruction is obtained from partial Fourier measurements of the original data that are sensed without any prior knowledge of specific feature domains for the data. Our experiments demonstrate the superiority of surfacelets for efficient representation of volumetric data. Moreover, we establish that the accuracy of reconstruction can further improve once a more effective basis for a sparser representation of the data becomes available.