Recent advancements in AI and data science have led to the increased use of Jupyter notebooks. As such, various AI-Based automated tools have been also developed to automatically document notebooks. However, a key challenge is the absence of suitable datasets for training AI models. In this paper, we outline a valuable roadmap for developing a dataset of (markdown, code) pairs centered on functions in Jupyter notebooks. The roadmap encompasses four high-level steps: structural filtering, structural processing, conceptual filtering, and conceptual processing. Our proposed roadmap leads to providing a quality dataset for training AI models on Jupyter notebooks.

Computational notebooks, like Kaggle notebooks, offer an integrated platform for coding and documentation, yet the latter's quality often falls short as scientists may neglect this crucial aspect. This paper addresses the need for improved and efficient code documentation generation in computational notebooks. As recent literature emphasizes integrating code's inherent structure into documentation generation models, our research explores unutilized structural characteristics, incorporating metrics from code sequences to enable better code documentation suggestions. Evidenced by the improved BLEU scores, our proposed method significantly outperforms the conventional model in a preliminary 10-fold cross-validation experiment and further provides a flexible foundation for integrating source code metrics into diverse code documentation generation models.

In software development, code documentation is crucial for collaboration and maintenance, especially as projects become more complex. However, it is often neglected due to the tedious effort it requires. This paper explores automating documentation generation for computational notebooks, focusing on the impact of code metrics such as lines of code, API popularity, and complexity on this task. Using a dataset of 22K code-documentation pairs, we compare deep learning models with and without code metric augmentation. The results show that incorporating these metrics significantly improves the accuracy of documentation generation, underscoring the connection between code metrics and quality documentation.