Large language models like ChatGPT can generate human-like code, posing challenges for programming education as students may be tempted to misuse them on assignments. However, there are currently no robust detectors designed specifically to identify AI-generated code. This is an issue that needs to be addressed to maintain academic integrity while allowing proper utilization of language models. Previous work has explored different approaches to detect AI-generated text, including watermarks, feature analysis, and fine-tuning language models. In this paper, we address the challenge of determining whether a student's code assignment was generated by a language model. First, our proposed method identifies AI-generated code by leveraging targeted masking perturbation paired with comperhesive scoring. Rather than applying a random mask, areas of the code with higher perplexity are more intensely masked. Second, we utilize a fine-tuned CodeBERT to fill in the masked portions, producing subtle modified samples. Then, we integrate the overall perplexity, variation of code line perplexity, and burstiness into a unified score. In this scoring scheme, a higher rank for the original code suggests it's more likely to be AI-generated. This approach stems from the observation that AI-generated codes typically have lower perplexity. Therefore, perturbations often exert minimal influence on them. Conversely, sections of human-composed codes that the model struggles to understand can see their perplexity reduced by such perturbations. Our method outperforms current open-source and commercial text detectors. Specifically, it improves detection of code submissions generated by OpenAI's text-davinci-003, raising average AUC from 0.56 (GPTZero baseline) to 0.87 for our detector.

Reverse engineering involves analyzing the design, architecture, and functionality of systems, and is crucial for legacy systems. Legacy systems are outdated software systems that are still in use and often lack proper documentation, which makes their maintenance and evolution challenging. To address this, we introduce SC2Req, utilizing the Generative Pre-trained Transformer (GPT) for automated code analysis and requirement generation. This approach aims to convert source code into understandable requirements and bridge the gap between those two. Through experiments on diverse software projects, SC2Req shows the potential to enhance the accuracy and efficiency of the translation process. This approach not only facilitates faster software development and easier maintenance of legacy systems but also lays a strong foundation for future research, promoting better understanding and communication in software development.

In the era of large language models like Chatgpt, maintaining academic integrity in programming education has become challenging due to potential misuse. There's a pressing need for reliable detectors to identify Chatgpt-generated code. While previous studies have tackled model-generated text detection, identifying such code remains uncharted territory. In this paper, we introduce a novel method to discern Chatgpt-generated code. We employ targeted masking perturbation, emphasizing code sections with high perplexity. Fine-tuned CodeBERT is utilized to replace these masked sections, generating subtly perturbed samples. Our scoring system amalgamates overall perplexity, variations in code line perplexity, and burstiness. In this scoring scheme, a higher rank for the original code suggests it's more likely to be chatgpt-generated. The underlying principle is that code generated by models typically exhibits consistent, low perplexity and reduced burstiness, with its ranking remaining relatively stable even after subtle modifications. In contrast, human-written code, when perturbed, is more likely to produce samples that the model prefers. Our approach significantly outperforms current detectors, especially against OpenAI's text-davinci-003 model, with the average AUC rising from 0.56 (GPTZero baseline) to 0.87.

The rapid evolution of Large Language Models (LLMs) like OpenAI’s ChatGPT has significantly influenced the landscape of natural language processing, heralding breakthroughs in text generation capabilities. These advancements, while beneficial, have raised concerns regarding the authenticity and origin of LLM-generated content, particularly in the context of misuse such as misinformation dissemination and academic dishonesty. Addressing these concerns, this paper introduces a novel, customizable text watermarking technique that embeds a spectrum of information within LLM-generated texts. Unlike traditional binary classifiers that only differentiate between human and model-generated texts, our method embeds a watermark containing detailed information like the originating model, generation date, and user identifiers. This approach not only aids in distinguishing between human and LLM-generated texts but also carries crucial metadata for traceability and authenticity verification. Utilizing Quadrature Phase Shift Keying (QPSK) for embedding and decoding watermarks, our watermark method demonstrates effectiveness, secrecy, robustness, and customizability. Our text watermarking method maintains high-quality text while achieving a detection performance with an Area Under the Curve (AUC) score of 0.98, on par with advanced watermarking techniques on LLMs. Additionally, it exhibits a near-perfect success rate of almost 100% in decoding single-letter watermark information. Furthermore, it ensures robustness against common textual attacks, effectively balancing the need for traceability and authenticity verification with minimal impact on text quality. This method’s efficacy demonstrates its potential as a valuable tool in addressing the challenges posed by the misuse of generative AI technologies.

We aim to propose a system repairing programs with logic errors to be functionally correct among different programming languages. Logic error program repair has always been a thorny problem: First, a logic error is usually harder to repair than a syntax error in a program because it has no diagnostic feedback from compilers. Second, it requires inferring in different ranges (i.e., the distance of related code lines) and tracking symbols across its pseudocode, source code, and test cases. Third, the logic error datasets are scarce, since an ideal logic error dataset should contain lots of components during the development procedure of a program, including a program specification, pseudocode, source code, test cases, and test reports (i.e., test case failure report). In our work, we propose novel solutions to these challenges. First, we introduce pseudocode information to assist logic error localization and correction. We construct a code-pseudocode graph to connect symbols across a source code and its pseudocode and then apply a graph neural network to localize and correct logic errors. Second, we collect logic errors generated in the process of syntax error repairing via DrRepair from 500 programs in the SPoC dataset and reconstruct them to our single logic error dataset, which we leverage to train and evaluate our models. Our experimental results show that we achieve 99.39% localization accuracy and 19.20% full repair accuracy on logic errors with five-fold cross-validation. Based on our current work, we will replenish and construct more complete public logic error datasets and propose a novel system to comprehend different programming languages from several perspectives and correct logic errors to be functionally correct.

This research introduces a system that combines Graph Neural Networks and Machine Learning to accurately pinpoint logic errors in programming. Unlike syntax errors which compilers can easily identify, logic errors lack clear diagnostic feedback, making them harder to locate. To enhance error detection, our system incorporates pseudocode, enabling a structured connection between source code and its pseudocode through a code-pseudocode graph. By using a Graph Neural Network model, we can localize these logic errors with higher precision. To train and validate our model, we gather logically erroneous programs from the syntax error repair process, forming a unique dataset. Experiments reveal an impressive localization accuracy of 99.42% on this dataset with five-fold cross-validation. Future directions include expanding our dataset and devising a system to interpret diverse programming languages for efficient error localization.