API recommendation has been widely used to enhance developers’ efficiency in software development. However, existing API recommendation methods for dynamic languages such as Python usually suffer from the limitations of incorrect type inference and lack of rich contextual semantics. To address these issues, we propose in this paper a novel approach, PyBartRec, to recommend APIs for Python programs concerning their rich semantics. Instead of analyzing the data flow information only, our approach utilizes a Transformer-based pre-trained model to extract the semantic features of Python code snippets. Such contextual information allows our approach to recommend correct APIs even when the APIs are not included in the local data flow information. We also use such information to perform a post-processing type inference. By narrowing the range of candidate types, our approach can recommend APIs accurately even in the failure scenarios of type inference. We evaluated PyBartRec in eight popular Python projects and the experimental results show that our approach significantly outperforms the state-of-the-art solutions. In particular, the average top-1 accuracy and average top-10 accuracy of PyBartRec within the same project are over 40%, and 60%, respectively. In cross-project recommendation, the MRR of PyBartRec is also over 40%.

Deep learning Transformer architectures play a critical role in developing advanced code prediction models, which are essential in modern Integrated Development Environments (IDEs). Nevertheless, these architectures encounter a significant challenge in effectively capturing and utilizing the structural information present in Abstract Syntax Trees (ASTs). To tackle this challenge, we propose an innovative approach that leverages AST structural matrices to enhance Transformers for source code prediction. Specifically, we integrate three types of AST structural matrices - the R matrix, A&S matrix, and MVG matrix - into the attention module of the Transformer to effectively capture the structural information within ASTs. To ensure optimal utilization, we have devised a range of strategies and integration methods tailored specifically for the attention module. To assess the effectiveness of our proposal, we conduct empirical studies using a standard Python dataset. The results demonstrate that incorporating AST structural matrices significantly enhances the accuracy of code prediction models, leading to an overall improvement from 73.18% to 75.10%. Furthermore, we conduct an in-depth analysis of the impact of each matrix type, offering a comprehensive understanding of their application scenarios. This insightful analysis provides valuable guidance on how to effectively leverage each matrix type in various contexts.