This paper presents a comprehensive and quantitative study of bugs related to Data Path in PyTorch with a focus on tensor management in memory. The bugs were reported from 2017 to 2024. Analyzing 3,089 closed issues, we identified 11 distinct bug types affecting the data storage, allocation, and loading, including memory bugs, indexing errors, and tensor contiguity violations. Our analysis reveals that data-path bugs have more occurrences than bugs related to computation in PyTorch in recent years. Among the memory bugs, non-contiguity bugs account for 30.2% of the total number of bugs and they have the most significant impact, leading to both crashes and silent correctness failures. One of the common solutions to addressing non-contiguity bugs is transforming from non-contiguous data to contiguous data in memory before machine-learning computation. To assess the impact of memory layout transformation, we conducted experiments involving tensor augmentation and non-contiguous tensor conversion. Our findings demonstrate that maintaining tensor contiguity throughout the augmentation process can improve performance by up to 49.6%, while the time required for non-contiguous tensor conversion varies significantly based on the number and order of dimensions. Our research provides valuable insights for developers and researchers working with PyTorch, helping them to identify and address potential bugs in data paths and tensor memory management.

This study presents an in-depth manual analysis of memory-related bugs within the PyTorch deep learning framework, leveraging a filtered dataset of 1,678 closed issues from the official PyTorch GitHub repository. The selected issues span a three-year period from January 1, 2020, to March 23, 2023, allowing for a comprehensive examination of trends, patterns, and solutions. This study aims to understand the correlations between the characteristics of PyTorch bugs and also the composition of the root causes behind memory bugs. The findings reveal that Correctness and Runtime Error bugs occur most frequently, with a lack of a correlation between Affected Components and Bug Symptoms. Our results highlight the need for more integrated inter-component debugging tools. Furthermore, the findings show that indexing errors occur most frequently among memory bugs. We determine that, to address the severe impact of such memory bugs, there exists a need for more comprehensive and redundant test cases. Through this analysis, this work aims to provide actionable insights for developers to improve the robustness of PyTorch, improving its reliability in machine learning applications.