In recent years, the research community, but also the general public, has raised serious questions about the reproducibility and replicability of scientific work. Since many studies include some kind of computational work, these issues are also a technological challenge, not only in computer science, but also in most research domains. Computational replicability and reproducibility are not easy to achieve due to the variety of computational environments that can be used. Indeed, it is challenging to recreate the same environment via the same frameworks, code, programming languages, dependencies, and so on. We propose a framework, known as SciRep, that supports the configuration, execution, and packaging of computational experiments by defining their code, data, programming languages, dependencies, databases, and commands to be executed. After the initial configuration , the experiments can be executed any number of times, always producing exactly the same results. Our approach allows the creation of a reproducibility package for experiments from multiple scientific fields, from medicine to computer science, which can be re-executed on any computer. The produced package acts as a capsule, holding absolutely everything necessary to re-execute the experiment. To evaluate our framework, we compare it with three state-of-the-art tools and use it to reproduce 18 experiments extracted from published scientific articles. With our approach, we were able to execute 16 (89%) of those experiments, while the others reached only 61%, thus showing that our approach is effective. Moreover, all the experiments that were executed produced the results presented in the original publication. Thus, SciRep was able to reproduce 100% of the experiments it could run.

Computational reproducibility-the ability to reexecute a scientific experiment using the same code, data, and configuration-should be straightforward. However, researchers often struggle with inconsistencies in documentation, missing dependencies, and environment setup, which undermines the credibility of scientific results. To address this, we propose a conversational, text-based tool that aids researchers in reproducing and packaging computational experiments into a single file. This file can be re-executed with a double-click on any machine, requiring only a single tool. SciConv is designed to support two key scenarios: (i) enabling researchers to prepare their own experiments in a reproducible, shareable format, and (ii) helping other researchers reproduce existing experiments from shared code repositories. In both cases, the tool reduces technical overhead and simplifies environment configuration through conversational interaction. We evaluated the tool through two studies. In the first, we reproduced 15 of 18 published experiments, with most requiring little or no user interaction. In the second, we conducted a user study comparing our tool with a professional platform, using the System Usability Scale (SUS) and NASA Task Load Index (TLX). The results show a statistically significant advantage for our tool in both usability and workload, demonstrating its effectiveness in supporting reproducibility.

Computational reproducibility remains a critical yet unresolved issue across scientific disciplines, often hindered by complex configuration requirements and technical barriers. We present SciConv, a novel conversational tool designed to assist researchers in creating and executing reproducible computational experiments using natural language. By leveraging large language models (llMs), SciConv automates the detection of dependencies and programming languages, and packages experiments into portable artifacts with minimal manual input. Unlike traditional platforms based on graphical user interfaces (e.g., web-based), SciConv features a chat-based interface that guides researchers interactively through the reproducibility workflow. This paper introduces the architecture, design principles, and interaction model of SciConv, and discusses its potential to lower the technical barriers to reproducibility.

In science, it is very important to be able to recreate the same computing environment to reproduce the same results achieved by previous scientific experiments. However, it is challenging to create a shareable package with the same environment using the same programming languages, frameworks, or data sources due to the diversity of researchers’ knowledge and the variety of computational environments used. In this work, I propose designing and constructing a conversational user interface that allows researchers to upload experiment files and clarify the necessary information via text communication to create a reproducible experiment package. My approach uses an integrated Large Language Model (LLM) that allows the platform to infer, whenever possible, some information (e.g., the programming language used, the main file to be executed, the parameters needed to be inserted when executing) to reduce the amount of information asked to the user. With this work, I intend to use an LLM to guide the researcher through this procedure, thereby reducing the time spent and the number of interactions required to create a reproducible package.

User studies are paramount for advancing science. In particular, the empirical evaluation of programmer-oriented tools is important to validate research ideas and prototypes, as well as production-ready tools. Previous research has collected several tools used by the software engineering and behavioral science communities to design and run studies. In this work, we study tools used in software engineering studies and identify their features. Furthermore, we analyze three behavioral science experiment tools to identify design ideas that might be adapted to programmer user studies. With this work, we present the set of features currently offered by software engineering tools to support researchers in the design and execution of programmer user studies. We also present the characteristics of some tools used in behavioral science experiments to identify design ideas that can be adapted to programmer user studies.

In recent years, the research community has raised serious questions about the replicability and reproducibility of scientific work. In particular, since many studies include some kind of computing work, these are also technological challenges, not only in computer science but in most research domains. Replicability and reproducibility are not easy to achieve, not only because researchers have diverse proficiency in computing technologies, but also because of the variety of computational environments that can be used. Indeed, it is challenging to recreate the same environment using the same frameworks, code, programming languages, dependencies, and so on. In this work, we propose a vision for an Integrated Development Environment allowing the creation, configuration, execution, packaging, and sharing of scientific computational experiments. Such a framework should allow researchers to easily set the code and data used and define the programming languages, code, dependencies, databases, or commands to execute to achieve consistent results for each experiment. With this work, we intend to aid researchers by integrating into the same platform all the stages of the design, execution, and analysis of a computational experiment.

In the research community, it is very important to be able to recreate the same computing environment to reproduce the same results achieved by previous experiments. However, it is a challenge to recreate the same environment using the same programming languages, frameworks or data sources due to the diversity of researchers’ knowledge and the variety of computational environments used.In this work, we propose to design and build a platform that allows researchers to create, (re-)execute and process computational experiments in a systematic and user-friendly manner. Our platform also includes the cataloging and a searching mechanism to improve data reusability as well as metadata classification to satisfy the needs of different research communities. With this work, we intend to improve the reusability of data and experiments and the reproducibility in the research domain.