Benchmarks play a significant role in how technology companies communicate about model capabilities and how researchers and the public understand generative AI systems. However, existing benchmarks have been criticized for their failure to adequately capture real-world usages (i.e. ecological validity) or to measure underlying concepts (i.e. construct validity). Building on approaches in HCI, we adopt a human-centered design process to address such critiques. Working within the journalism domain we engaged 23 professionals in a workshop which informed the design of a domain-oriented evaluation “cookbook”. Our workshop findings surface domain-specific challenges and tensions faced by designers in translating specific tasks to evaluation constructs, aligning metrics with domain-specific values, and balancing needs among different stakeholders when constructing evaluations. Through an instantiation of design-based approaches for benchmark creation in the journalism domain, this work not only produces an evaluation structure for journalism practitioners to experiment with, but also lays out design requirements for AI evaluations that are contextualized, value-aligned, and cultivate evaluative literacy for domain end-users.

Numerous social movements (SMs) around the world help support the UN's Sustainable Development Goals (SDGs). Understanding how key events shape SMs is key to the achievement of the SDGs. We have developed SMART (Social Media Analysis & Reasoning Tool) to track social movements related to the SDGs. SMART was designed by a multidisciplinary team of AI researchers, journalists, communications scholars and legal experts. This paper describes SMART's transformer-based multivariate time series Discourse Evolution Engine for Predictions about Social Movements (DEEP) to predict the volume of future articles/posts and the emotions expressed. DEEP outputs probabilistic forecasts with uncertainty estimates, providing critical support for editorial planning and strategic decision-making. We evaluate DEEP with a case study of the #MeToo movement by creating a novel longitudinal dataset (433K Reddit posts and 121K news articles) from September 2024 to June 2025, which is publicly available for research purposes.

Point packings in the unit square are placements of n points in the unit square that maximize the minimum distance between any two of the points. Such packings are surrogates for the 2D-stock cutting problem. In this study we examine a greedy generative representation for the point packing problem and extend the problem to higher dimensions. This representation uses a greedy algorithm to select points generated as whole-number linear combinations of vectors. This means that sets of vectors are evolved. The lattice generated by the vectors, taken modulo one, yields a set of points that can be greedily filtered to a dense point packing. The focus of evolution is the choice of vector generators for the lattice. A parameter study is performed comparing two mutation operators, different rates of application for mutation, and different population sizes. The generative representation is found to efficiently locate large point packings, while using relatively few real-valued parameters. The number of real parameters used to specify a point packing may be chosen. This novel control value is shown to have a substantial impact on results. A preliminary application of point packings, as population initializers, is demonstrated. Using a point packing as an initial population for an evolutionary optimizer can yield improved performance by providing more even sampling of the optimization domain and, in this study, it is shown that use of a point packing improves performance in a higher dimensional test problem, but not in a lower dimensional one.

The traditional method of fitting an approximate cumulative probability distribution to a data set is to bin the data in narrow bins and obtain a step function approximation. This technique suffices for many applications, but the resulting object is not a differentiable function making recovery of the underlying probability distribution function impossible. In this study, a unique group theoretic representation is used to define evolvable data warps that can be used to recover continuous, infinitely differentiable versions of the inverse cumulative distribution function. The use of a group theoretic representation permits a simple calculation to transform the evolved object into a cumulative distribution function and, via differentiation, into a probability distribution function. The group used to define the evolvable data warps is the group of bijections of the unit interval. The generators used by evolution are chosen to be differentiable in order to enable the computation of probability distribution functions. Experiments are run using a simple type of evolutionary algorithm to evolve approximate CDFs on seven data sets. The first data set is used to perform a parameter study on the representation length used to evolve the approximate CDFs and comparing two variations of the representation - one of which uses a representational control called gene expression and one of which does not.