Limitations in a scientific article refer to the inherent shortcomings, constraints, or weaknesses of a study that can affect its results or limit the generalizability of its findings. One type of limitation is visual-related limitations that focus on issues such as unclear charts, diagrams, captions, or descriptions in scientific papers. In this work, we focus on generating image descriptions based on some questions from charts and graphs using multi-modal Large Language Models (LLMs) such as QWen, Llama, Llava, Pali-GEMMA, and GPT-4o. Using an LLM-as-a-judge evaluation approach, where two LLMs acted as evaluators, we found that GPT-4o outperformed the other models in generating accurate and coherent chart descriptions.

In recent years, there has been a growing concern and emphasis on conducting research beyond academic or scientific research communities, benefiting society at large. A well-known approach to measuring the impact of research on society is enumerating its policy citation(s). Despite the importance of research in informing policy, there is no concrete evidence to suggest the research’s relevance in cited policy documents. This is concerning because it may increase the possibility of evidence used in policy being manipulated by individual, social, or political biases that may lead to inappropriate, fragmented, or archaic research evidence in policy. Therefore, it is crucial to identify the degree of relevance between research articles and citing policy documents. In this paper, we examined the scale of contextual relevance of youth-focused research in the referenced US policy documents using natural language processing techniques, state-of-the-art pre-trained Large Language Models (LLMs), and statistical analysis. Our experiments and analysis concluded that youth-related research articles that get US policy citations are mostly relevant to the citing policy documents.

This study explores the representation of youth in US policy documents by analyzing how research on youth topics is cited within these policies. The research focuses on three key questions: identifying the frequently discussed topics in youth research that receive citations in policy documents, discerning patterns in youth research that contribute to higher citation rates in policy, and comparing the alignment between topics in youth research and those in citing policy documents. Through this analysis, the study aims to shed light on the relationship between academic research and policy formulation, highlighting areas where youth issues are effectively integrated into policy and contributing to the broader goal of enhancing youth engagement in societal decision-making processes.

The reproducibility of scientific articles is central to the advancement of science. Despite this importance, evaluating reproducibility remains challenging due to the scarcity of ground truth data. Predictive models can address this limitation by streamlining the tedious evaluation process. Typically, a paper's reproducibility is inferred based on the availability of artifacts such as code, data, or supplemental information, often without extensive empirical investigation. To address these issues, we utilized artifacts of papers as fundamental units to develop a novel, dual-spectrum framework that focuses on author-centric and external-agent perspectives. We used the author-centric spectrum, followed by the external-agent spectrum, to guide a structured, model-based approach to quantify and assess reproducibility. We explored the interdependencies between different factors influencing reproducibility and found that linguistic features such as readability and lexical diversity are strongly correlated with papers achieving the highest statuses on both spectrums. Our work provides a model-driven pathway for evaluating the reproducibility of scientific research.