Data-driven models can predict filesystem I/O time proportion, a key metric for guiding I/O tuning in HPC applications, from performance data. This work evaluates whether the XGBoost-based model we proposed in our prior work [13], remains effective when the underlying profiling tool changes. Using I/O metrics collected with Score-P and DFTracer, we analyze two video processing workloads and train predictive models. Results show that predictive accuracy remains stable across profiling tools while revealing workload-specific differences in I/O behavior, suggesting that the relationship between configuration parameters and the proportion of filesystem I/O time is preserved despite variations in measurement methodology. Furthermore, the findings indicate that configuration-driven models can approximate I/O time proportion without requiring runtime tracing for every execution, reducing dependence on instrumentation in future machine learning-driven performance analysis workflows.

The current state of practice in HPC is that performance metrics reported by storage benchmarks are disconnected from those obtained through application-level performance analysis tools, making it difficult for users to determine whether performance tuning efforts are effective or whether observed performance indicates underutilization of the system. In this work, we propose an approach to align IO500 storage benchmark results with application-level performance by deconstructing benchmark components and recalculating their metrics. The results show that certain universal metrics, such as bandwidth, can be meaningfully aligned with application performance, enabling more consistent and interpretable evaluation. Our findings also identify metrics that remain missing or cannot be reconciled, highlighting the need for standardization to align metrics produced by benchmarks and performance analysis tools.

The decision to optimize I/O in scientific applications often depends on the proportion of I/O time within an application's runtime. For AI workloads, such as machine learning (ML), we observe that different configurations with the same data size can lead to varying I/O impacts. In this work, we analyze common tunable parameters—batch size, number of samples, and number of files—typically adjusted when adapting ML models to new datasets or use cases. Using the XGBoost model, we predict the I/O percentage for ResNet50 and UNet3D workloads and identify feature combinations that influence I/O behavior. Our results provide practical guidance for optimizing I/O performance in AI workloads.

Providing a high-quality performance prediction has the potential to enhance various aspects of a cluster, such as devising scheduling and provisioning policies, guiding procurement decisions, suggesting candidate applications for tuning, and identifying probable scaling and porting challenges. Creating such a prediction for the I/O metrics is still challenging, however, due to the intricate interplay of multiple cluster components, making this an ideal case for machine learning. Nevertheless, achieving the required accuracy level with machine learning calls for a substantial amount of high-quality data, which is often a difficult challenge for most HPC clusters. In this work we explore the use of transfer learning to predict the applications’ I/O bandwidth based on a public dataset. As a result, our experiment can provide an I/O bandwidth prediction for a different cluster comparable to the current state-of-the-art result while employing 100 times less data than needed to construct the base model. Furthermore, we evaluate potential future improvements of the proposed workflow.

Modern software engineering is getting increasingly complicated. Especially in the HPC field, we are dealing with cutting edge infrastructure and a novel problem with unprecedented scale. The ability to monitor and analyze the performance of such applications and infrastructure is imperative for the future of improvement, design, and maintenance. In the current era, the writing and maintenance of these applications have ceased to be the job solely of computer scientists and has grown to encompass a wide variety of experts in mathematics, science, and other engineering disciplines. The fact that many developers from these disciplines have not received a formal education in computer science and rely increasingly on the tools created by computer scientists to analyze and optimize their code shows that there's a need for a forum to work together.