ChatGPT presents emerging opportunities in software development, yet its capabilities for understanding code remain largely understudied. This study aims to focus on the logical aspect of code comprehension of ChatGPT by examining its performance in detecting and fixing bugs. Our preliminary results suggest that ChatGPT seems to correct code in a different way than human logical steps.

Pre-trained language models have achieved many prominent results in natural language processing. Since software engineering widely includes many natural language documents, the application of pre-trained language models have received much attention in software engineering tasks. However, pretraining a large volume of source code requires a huge amount of computational resources and time. In this study, we propose an additional pre-training approach to a well-trained language model. Our initial results on mT5, multilingual T5 with an additional pretraining of Python code shows improved performance on multiple software engineering tasks including code generation, code summarisation, code repair, and error diagnosis.