Multimodal vision-language models (VLMs) continue to achieve ever-improving scores on chart understanding benchmarks. Yet, we find that this progress does not fully capture the breadth of visual reasoning capabilities essential for interpreting charts. We introduce EncQA, a novel benchmark informed by the visualization literature, designed to provide systematic coverage of visual encodings and analytic tasks that are crucial for chart understanding. EncQA provides 2,076 synthetic question-answer pairs, enabling balanced coverage of six visual encoding channels (position, length, area, color quantitative, color nominal, and shape) and eight tasks (find extrema, retrieve value, find anomaly, filter values, compute derived value exact, compute derived value relative, correlate values, and correlate values relative). Our evaluation of 9 state-of-the-art VLMs reveals that performance varies significantly across encodings within the same task, as well as across tasks. Contrary to expectations, we observe that performance does not improve with model size for many task-encoding pairs. Our results suggest that advancing chart understanding requires targeted strategies addressing specific visual reasoning gaps, rather than solely scaling up model or dataset size.

To deploy machine learning models on-device, practitioners use compression algorithms to shrink and speed up models while maintaining their high-quality output. A critical aspect of compression in practice is model comparison, including tracking many compression experiments, identifying subtle changes in model behavior, and negotiating complex accuracy-efficiency trade-offs. However, existing compression tools poorly support comparison, leading to tedious and, sometimes, incomplete analyses spread across disjoint tools. To support real-world comparative workflows, we develop an interactive visual system called COMPRESS AND COMPARE. Within a single interface, COMPRESS AND COMPARE surfaces promising compression strategies by visualizing provenance relationships between compressed models and reveals compression-induced behavior changes by comparing models' predictions, weights, and activations. We demonstrate how COMPRESS AND COMPARE supports common compression analysis tasks through two case studies, debugging failed compression on generative language models and identifying compression artifacts in image classification models. We further evaluate COMPRESS AND COMPARE in a user study with eight compression experts, illustrating its potential to provide structure to compression workflows, help practitioners build intuition about compression, and encourage thorough analysis of compression's effect on model behavior. Through these evaluations, we identify compression-specific challenges that future visual analytics tools should consider and COMPRESS AND COMPARE visualizations that may generalize to broader model comparison tasks.

Each year, expert-level performance is attained in increasingly-complex multiagent domains, where notable examples include Go, Poker, and StarCraft II. This rapid progression is accompanied by a commensurate need to better understand how such agents attain this performance, to enable their safe deployment, identify limitations, and reveal potential means of improving them. In this paper we take a step back from performance-focused multiagent learning, and instead turn our attention towards agent behavior analysis. We introduce a model-agnostic method for discovery of behavior clusters in multiagent domains, using variational inference to learn a hierarchy of behaviors at the joint and local agent levels. Our framework makes no assumption about agents' underlying learning algorithms, does not require access to their latent states or policies, and is trained using only offline observational data. We illustrate the effectiveness of our method for enabling the coupled understanding of behaviors at the joint and local agent level, detection of behavior changepoints throughout training, discovery of core behavioral concepts, demonstrate the approach's scalability to a high-dimensional multiagent MuJoCo control domain, and also illustrate that the approach can disentangle previously-trained policies in OpenAI's hide-and-seek domain.

The rise of social media in the enterprise has enabled new ways for employees to speak up and communicate openly with colleagues. This rich textual data can potentially be mined to better understand the opinions and sentiment of employees for the benefit of the organization. In this paper, we introduce Enterprise Social Pulse (ESP) -- a tool designed to support analysts whose job involves understanding employee chatter. ESP aggregates and analyzes data from internal and external social media sources while respecting employee privacy. It surfaces the data through a user interface that supports organic results and keyword search, data segmentation and filtering, and several analytics and visualization features. An evaluation of ESP was conducted with 19 Human Resources professionals. Results from a survey and interviews with participants revealed the value and willingness to use ESP, but also surfaced challenges around deploying an employee social media listening solution in an organization.

US federal legislation is a common subject of discussion and advocacy on the web, inspired by the open government movement. While the contents of these bills are freely available for download, understanding them is a significant challenge to experts and average citizens alike due to their length, complex language, and obscure topics. To make these important documents more accessible to the general public, we present Many Bills (http://manybills.us): a web-based set of visualization tools that reveals the underlying semantics of a bill. Using machine learning techniques, we classify each bill's sections based on existing document-level categories. We then visualize the resulting topic substructure of these bills. These visualizations provide an overview-and-detail view of bills, enabling users to read individual sections of a bill and compare topic patterns across multiple bills. Through an overview of the site's user activity and interviews with active users, this paper highlights how Many Bills makes the tasks of reading bills, identifying outlier sections in bills, and understanding congressperson's legislative activity more manageable.

Reading congressional legislation, also known as bills, is often tedious because bills tend to be long and written in complex language. In IBM Many Bills, an interactive web-based visualization of legislation, users of different backgrounds can browse bills and quickly explore parts that are of interest to them. One task users have is to be able to locate sections that don't seem to fit with the overall topic of the bill. In this paper, we present novel techniques to determine which sections within a bill are likely to be outliers by employing approaches from information retrieval. The most promising techniques first detect the most topically relevant parts of a bill by ranking its sections, followed by a comparison between these topically relevant parts and the remaining sections in the bill. To compare sections we use various dissimilarity metrics based on Kullback-Leibler Divergence. The results indicate that these techniques are more successful than a classification based approach. Finally, we analyze how the dissimilarity metrics succeed in discriminating between sections that are strong outliers versus those that are 'milder' outliers.

We introduce a programming environment entitled Share that is designed to encourage loosely bound cooperation between individuals within communities of practice through the sharing of code. Loosely bound cooperation refers to the opportunity community members have to assist and share resources with one another while maintaining their autonomy and independent practice. We contrast this model with forms of collaboration that enable large numbers of distributed individuals to collaborate on large scale works where they are guided by a shared vision of what they are collectively trying to achieve. We hypothesize that providing fine-grained, publicly visible attribution of code sharing activity within a community can provide socially motivated encouragement for code sharing. We present an overview of the design of our tool and the objectives that guided its design and a discussion of a small-scale deployment of our prototype among members of a particular community of practice.

In this paper we describe Taking Sides, a performance using a real-time speech visualization software system called TextEngine. Taking Sides is a collaboration between our research studio and Montreal hip-hop artist Dwayne Hanley. Our primary goal was to create a strong conceptual link between the text visualization, the content of the artist's lyrics, and his performance style. Additionally we wanted to test the flexibility of TextEngine in developing customized performance applications. Pursuing these goals led us through a three month development effort that cycled tightly between design, performance and programmatic iterations.