Large language models (LLMs) are powerful tools capable of handling diverse tasks. Comparing and selecting appropriate LLMs for specific tasks requires systematic evaluation methods, as models exhibit varying capabilities across different domains. However, finding suitable benchmarks is difficult given the many available options. This complexity not only increases the risk of benchmark misuse and misinterpretation but also demands substantial effort from LLM users, seeking the most suitable benchmarks for their specific needs. To address these issues, we introduce BenchmarkCards, an intuitive and validated documentation framework that standardizes critical benchmark attributes such as objectives, methodologies, data sources, and limitations. Through user studies involving benchmark creators and users, we show that BenchmarkCards can simplify benchmark selection and enhance transparency, facilitating informed decision-making in evaluating LLMs.Data & Code:github.com/SokolAnn/BenchmarkCards huggingface.co/datasets/ASokol/BenchmarkCards

We present Ventana a la Verdad, a chatbot designed to make the Clar- ification Archive and the reports of the Colombian Truth Commis- sion [6] more accessible to a wider audience. These archives contain a wealth of documents, interviews, and testimonies from Colom- bia's armed conflict, but navigating them can be challenging due to their volume and complexity. Using existing large language models (LLMs) and natural language processing techniques, our chatbot allows users to interact with the archives through natural language queries, receiving relevant and contextually appropriate responses. In the sensitive context of peace and reconciliation, where misin- formation or hallucinations can have significant adverse effects, ensuring the accuracy and reliability of information is paramount. This tool aims to facilitate better understanding and engagement with historical content, supporting educational and research efforts. We discuss the development of the chatbot, the challenges encoun- tered, and its potential impact on making the Colombian Truth Commission's archives more accessible. The chatbot is available by link here: http://ventanaverdad.lucyapps.net:1337/