Visualization design is often a demanding process that involves trying different encodings, exploring different data transformations, and refining details. While there have been important studies of these workflows, the lower-level, code-intensive practices remain underexplored. Exploratory notebook tools have allowed designers to rapidly iterate on visualizations. We use publicly-available version histories of notebooks to study how users work in these environments, observing both how they build new visualizations from existing templates or previous work and how they refine visualizations over time. We examine the interplay between data manipulation and visualization, and classify the types of changes made when updating visual encodings. We also analyze the impact of different frameworks by comparing two code-oriented libraries and a chart wizard. Finally, we examine how interactions with notebooks have changed over the years, including after the widespread availability of AI. These analyses help us understand how users iterate to produce visualizations over time using different frameworks.

The Future Work section of a scientific article outlines potential research directions by identifying gaps and limitations of a current study. This section serves as a valuable resource for early-career researchers seeking unexplored areas and experienced researchers looking for new projects or collaborations. In this study, we generate future work suggestions from a scientific article. To enrich the generation process with broader insights and reduce the chance of missing important research directions, we use context from related papers using RAG. We experimented with various Large Language Models (LLMs) integrated into Retrieval-Augmented Generation (RAG). We incorporate an LLM feedback mechanism to enhance the quality of the generated content and introduce an LLM-as-a-judge framework for robust evaluation, assessing key aspects such as novelty, hallucination, and feasibility. Our results demonstrate that the RAG-based approach using GPT-4o mini, combined with an LLM feedback mechanism, outperforms other methods based on both qualitative and quantitative evaluations. Moreover, we conduct a human evaluation to assess the LLM as an extractor, generator, and feedback provider.

Exploring data relations across multiple views has been a common task in many domains such as bioinformatics, cybersecurity, and healthcare. To support this, various techniques (e.g., visual links and brushing & linking) are used to show related visual elements across views via lines and highlights. However, understanding the relations using these techniques, when many related elements are scattered, can be difficult due to spatial distance and complexity. To address this, we present iTrace, an interactive visualization technique to effectively trace cross-view data relationships. iTrace leverages the concept of interactive focus transitions, which allows users to see and directly manipulate their focus as they navigate between views. By directing the user’s attention through smooth transitions between related elements, iTrace makes it easier to follow data relationships. We demonstrate the effectiveness of iTrace with a user study, and we conclude with a discussion of how iTrace can be broadly used to enhance data exploration in various types of visualizations.

Limitations in a scientific article refer to the inherent shortcomings, constraints, or weaknesses of a study that can affect its results or limit the generalizability of its findings. One type of limitation is visual-related limitations that focus on issues such as unclear charts, diagrams, captions, or descriptions in scientific papers. In this work, we focus on generating image descriptions based on some questions from charts and graphs using multi-modal Large Language Models (LLMs) such as QWen, Llama, Llava, Pali-GEMMA, and GPT-4o. Using an LLM-as-a-judge evaluation approach, where two LLMs acted as evaluators, we found that GPT-4o outperformed the other models in generating accurate and coherent chart descriptions.

In recent years, there has been a growing concern and emphasis on conducting research beyond academic or scientific research communities, benefiting society at large. A well-known approach to measuring the impact of research on society is enumerating its policy citation(s). Despite the importance of research in informing policy, there is no concrete evidence to suggest the research’s relevance in cited policy documents. This is concerning because it may increase the possibility of evidence used in policy being manipulated by individual, social, or political biases that may lead to inappropriate, fragmented, or archaic research evidence in policy. Therefore, it is crucial to identify the degree of relevance between research articles and citing policy documents. In this paper, we examined the scale of contextual relevance of youth-focused research in the referenced US policy documents using natural language processing techniques, state-of-the-art pre-trained Large Language Models (LLMs), and statistical analysis. Our experiments and analysis concluded that youth-related research articles that get US policy citations are mostly relevant to the citing policy documents.

This study explores the representation of youth in US policy documents by analyzing how research on youth topics is cited within these policies. The research focuses on three key questions: identifying the frequently discussed topics in youth research that receive citations in policy documents, discerning patterns in youth research that contribute to higher citation rates in policy, and comparing the alignment between topics in youth research and those in citing policy documents. Through this analysis, the study aims to shed light on the relationship between academic research and policy formulation, highlighting areas where youth issues are effectively integrated into policy and contributing to the broader goal of enhancing youth engagement in societal decision-making processes.

The reproducibility of scientific articles is central to the advancement of science. Despite this importance, evaluating reproducibility remains challenging due to the scarcity of ground truth data. Predictive models can address this limitation by streamlining the tedious evaluation process. Typically, a paper's reproducibility is inferred based on the availability of artifacts such as code, data, or supplemental information, often without extensive empirical investigation. To address these issues, we utilized artifacts of papers as fundamental units to develop a novel, dual-spectrum framework that focuses on author-centric and external-agent perspectives. We used the author-centric spectrum, followed by the external-agent spectrum, to guide a structured, model-based approach to quantify and assess reproducibility. We explored the interdependencies between different factors influencing reproducibility and found that linguistic features such as readability and lexical diversity are strongly correlated with papers achieving the highest statuses on both spectrums. Our work provides a model-driven pathway for evaluating the reproducibility of scientific research.

Composite materials are widely used in aircraft structures because of their superior mechanical properties. How-ever, their complex failure modes require sophisticated inspection methods to ensure structural integrity. Ultrasonic testing (UT) is a common non-destructive inspection (NDI) technique for aircraft composites that can detect internal and external defects with high resolution and accuracy. Despite their effectiveness, traditional UT methods rely on the manual interpretation of ultrasonic signals, which is time-consuming, labor-intensive, and subjec-tive. Furthermore, processing such large-scale data, particularly across materials of varying thicknesses, significantly increases the computational demands of deep learning model optimization. To overcome these challenges, we propose an efficient sparse mixture-of-experts (MoE) model with a multi-level loss function and introduce four novel training objectives to improve compu-tational efficiency and accuracy in identifying surface defects in composite aircraft materials. We evaluated our approach on material with multiple thicknesses or domains comprising various defects. Our experimental results demonstrate higher accuracy and F1-Score, with only 10% training epochs compared to baseline MoE.

While the conventional software components implement pre-defined specifications, Machine Learning (ML)-enabled Software Components (MLSC) learn the domain specifications from the training samples. Thus, the MLSC's data-driven and inductive reasoning becomes highly reliant on the quality of the training dataset, which are often arbitrarily collected in ad hoc manners. The random collection of samples leads to a significant gap between the actual specifications of a real-world concept, and the picture that a dataset represents of the concept, reducing MLSC generalizability, particularly in perceptual tasks where understanding the environment is an important factor of accurate prediction.

Multiple-view visualization (MV) has been heavily used in visual analysis tools for sensemaking of data in various domains (e.g., bioinformatics, cybersecurity and text analytics). One common task of visual analysis with multiple views is to relate data across different views. For example, to identify threats, an intelligence analyst needs to link people from a social network graph with locations on a crime-map, and then search for and read relevant documents. Currently, exploring cross-view data relationships heavily relies on view-coordination techniques (e.g., brushing and linking), which may require significant user effort on many trial-and-error attempts, such as repetitiously selecting elements in one view, and then observing and following elements highlighted in other views. To address this, we present SightBi, a visual analytics approach for supporting cross-view data relationship explorations. We discuss the design rationale of SightBi in detail, with identified user tasks regarding the use of cross-view data relationships. SightBi formalizes cross-view data relationships as biclusters, computes them from a dataset, and uses a bi-context design that highlights creating stand-alone relationship-views. This helps preserve existing views and offers an overview of cross-view data relationships to guide user exploration. Moreover, SightBi allows users to interactively manage the layout of multiple views by using newly created relationship-views. With a usage scenario, we demonstrate the usefulness of SightBi for sensemaking of cross-view data relationships.

Online and in the real world, communities are bonded together by emotional consensus around core issues. Emotional responses to scientific findings often play a pivotal role in these core issues. When there is too much diversity of opinion on topics of science, emotions flare up and give rise to conflict. This conflict threatens positive outcomes for research. Emotions have the power to shape how people process new information. They can color the public's understanding of science, motivate policy positions, even change lives. And yet little work has been done to evaluate the public's emotional response to science using quantitative methods. In this paper, we use a dataset of responses to scholarly articles on Facebook to analyze the dynamics of emotional valence, intensity, and diversity. We present a novel way of weighting click-based reactions that increases their comprehensibility, and use these weighted reactions to develop new metrics of aggregate emotional responses. We use our metrics along with LDA topic models and statistical testing to investigate how users' emotional responses differ from one scientific topic to another. We find that research articles related to gender, genetics, or agricultural/environmental sciences elicit significantly different emotional responses from users than other research topics. We also find that there is generally a positive response to scientific research on Facebook, and that articles generating a positive emotional response are more likely to be widely shared---a conclusion that contradicts previous studies of other social media platforms.