Commit messages are important for developers to understand the content and the reason for code changes. However, poor and even empty commit messages widely exist. To improve the quality of commit messages and development efficiency, many commit message generation methods have been proposed. Nevertheless, previous methods mainly focus on a brief generation problem, where both the input code change and the output commit messages are restricted to short. This may initiate a debate on the performance of these methods in practice. In this article, we attempt to remove the restrictions and move the needle forward to a holistic commit message generation problem. In particular, we conduct experiments to evaluate the performance of existing commit message generation methods in holistic commit message generation. In the experiments, we choose seven state-of-the-art commit generation methods and focus on two important scenarios in commit message generation (i.e., the within-project scenario and the cross-project scenario). To conduct our experiments, we publish a holistic commit message dataset HORDA with test data manually labeled. In our evaluations, we find that in generating holistic commit messages, the IR-based method has a better performance than non-pre-trained generation-based methods in the within-project scenario, contradicting previous research findings. Further, while the pre-trained generation-based methods are better than non-pre-trained generation-based methods, they are still constrained by the limitations of generation models.

Code pre-trained models have shown promising effectiveness in various software engineering tasks. Among these tasks, many tasks are related to software evolution and/or code editing. However, existing code pre-trained models often overlook the real-world code editing data and the evolutionary nature of the editing process. In this paper, to simulate the step-by-step code editing process of human developers, we propose DivoT5, a pre-trained model based on directional diffusion at the data level. In DivoT5, we adopt two categories of pre-training tasks. The first category is mask and denoising tasks augmented with a diffusion direction representing code evolution. That is, we first apply a noising process to the code snippets before evolution, and then ask the pre-training process to restore the snippets with noise into the code snippets after evolution. The second category is tasks aiming to reinforce the evolutionary direction. That is, we first generate various intermediate versions for each pair of snippets before and after evolution, and then ask the pre-training process to transform the intermediate versions into the snippet after evolution for each pair. We evaluate DivoT5 for two code-editing scenarios (including a number of tasks) and one non-editing scenario using four downstream tasks. For each downstream task, we fine-tune the pre-trained DivoT5 on multiple corresponding datasets and evaluate its effectiveness across diverse scenarios Our experimental results show that ivoT5 achieves state-of-the-art (SOTA) performance on most tasks in comparison to models of the same scale (220M), large-scale (770M, 6.7B) models in fine-tuning, and billion-scale (6.7B, 8B, ChatGPT) instruct models in few-shot settings. For one code-editing task (i.e., CodeReview in NL-based CodeRefinement task), DivoT5 pre-trained on top of CodeT5-small (60M) can even outperform CodeT5-base (220M) and other pre-trained models with 220M parameters except for DivoT5 pre-trained on top of CodeT5-base (220M).

evaluation metrics, addressing both self-model and inter-model perspectives.It supports automated batch experiments and allows for easy customization of agents and environments, enabling seamless integration for rapid MARL robustness research.Our experiments further validate the platform's efficacy in evaluating and improving MARL robustness.

Compilers are one of the most fundamental software systems. A large number of software systems rely on compilers for execution. Compiler bugs can significantly hinder software developers from diagnosing issues within their software. Therefore, it is essential to ensure the correctness of compilers and to isolate and fix compiler bugs. Isolating bugs within compilers is challenging due to compilers' complexity and large codebase. The prior studies on compiler bug isolation struggle to generate sufficient test cases for bug isolation and are not effective enough.

Widely used compilers like GCC and LLVM usually have hundreds of optimizations controlled by optimization flags, which are enabled or disabled during compilation to improve the runtime performance (e.g., small execution time) of the compiler program. Due to the large number of optimization flags and their combination, it is difficult for compiler users to manually tune compiler optimization flags. In the literature, a number of autotuning techniques have been proposed, which tune optimization flags for a compiled program by comparing its actual runtime performance with different optimization flag combinations. Due to the huge search space and heavy actual runtime cost, these techniques suffer from the widely recognized efficiency problem. To reduce the heavy runtime cost, in this article we propose a lightweight learning approach that uses a small number of actual runtime performance data to predict the runtime performance of a compiled program with various optimization flag combinations. Furthermore, to reduce the search space, we design a novel particle swarm algorithm that tunes compiler optimization flags with the prediction model. To evaluate the performance of the proposed approach, CompTuner, we conduct an extensive experimental study on two popular C compilers, GCC and LLVM, with two widely used benchmarks, cBench and PolyBench. The experimental results show that CompTuner significantly outperforms the six compared techniques, including the state-of-the-art technique BOCA.

Widely used compilers like GCC usually have hundreds of optimizations controlled by optimization flags, which can be enabled or disabled during compilation to improve the runtime performance of a compiled program. Due to the large number of optimization flags and their combination, it is difficult for compiler users to tune compiler optimization flags manually. In the literature, many auto-tuning techniques have been proposed, which find a desired setting on all optimization flags (i.e., an optimization sequence) by designing different search strategies in the entire optimization space. Due to the huge search space, these techniques suffer from the widely-recognized efficiency problem. To reduce the search space, in this paper, we propose a critical-flag selection based approach CFSCA which first finds flags potentially relevant to the target program by analyzing program structure and compiler documentation, and then identifies critical flags through statistical analysis on the program's predicted runtime performance with various optimization sequences. With the reduced search space, CFSCA selects a desired optimization sequence. To evaluate the performance of the proposed approach CFSCA, we conduct an extensive experimental study on the latest version of the compiler GCC with a widely used benchmark cBench. The experimental results show that CFSCA significantly outperforms the four compared techniques, including the state-of-art technique BOCA.