Continuous automated testing throughout each cycle can ensure the security of the continuous integration (CI) development lifecycle. Test case prioritization (TCP) is a critical factor in optimizing automated testing, which prioritizes potentially failed test cases and improves the efficiency of automated testing. In CI automated testing, the TCP is a continuous decision-making process that can be solved with reinforcement learning (RL). RL-based CITCP can continuously generate a TCP strategy for each CI development lifecycle, with the reward mechanism as the core. The reward mechanism consists of the reward function and the reward strategy. However, there are new challenges to RL-based CITCP in real-industry CI testing. With high-frequency iteration, the reward function is often calculated with a fixed length of historical information, ignoring the spatial characteristics of the current cycle. Therefore, the dynamic time window (DTW)-based reward function is proposed to perform the reward calculation, which adaptively adjusts the recent historical information range based on the integration cycle. Moreover, with low-failure testing, the reward strategy usually only rewards failure test cases, which creates a sparse reward problem in RL. To address this issue, the similarity-based reward strategy is proposed, which increases the reward objects of some passed test cases, similar to the failure test cases. The DTW-based reward function and the similarity-based reward strategy together constitute the proposed adaptive reward mechanism in RL-based CITCP. To validate the effectiveness of the adaptive reward mechanism, experimental verification is carried out on 13 industrial data sets. The experimental results show that the adaptive reward mechanism can improve the TCP effect, where the average NAPFD is maximally improved by 7.29%, the average Recall is maximally improved by 6.04% and the average TTF is improved by 6.81 positions with a maximum of 63.77.

Abstract Reinforcement learning (RL) has been used to optimize the continuous integration (CI) testing, where the reward plays a key role in directing the adjustment of the test case prioritization (TCP) strategy. In CI testing, the frequency of integration is usually very high, while the failure rate of test cases is low. Consequently, RL will get scarce rewards in CI testing, which may lead to low learning efficiency of RL and even difficulty in convergence. This paper introduces three rewards to tackle the issue of sparse rewards of RL in CI testing. First, the historical failure density‐based reward (HFD) is defined, which objectively represents the sparse reward problem. Second, the average failure position‐based reward (AFP) is proposed to increase the reward value and reduce the impact of sparse rewards. Furthermore, a technique based on additional reward is proposed, which extracts the test occurrence frequency of passed test cases for additional rewards. Empirical studies are conducted on 14 real industry data sets. The experiment results are promising, especially the reward with additional reward can improve NAPFD (Normalized Average Percentage of Faults Detected) by up to 21.97%, enhance Recall with a maximum of 21.87%, and increase TTF (Test to Fail) by an average of 9.99 positions.

In software regression testing, newly added test cases are more likely to fail, and therefore, should be prioritized for execution. In software regression testing for continuous integration, reinforcement learning-based approaches are promising and the RETECS (Reinforced Test Case Prioritization and Selection) framework is a successful application case. RETECS uses an agent composed of a neural network to predict the priority of test cases, and the agent needs to learn from historical information to make improvements. However, the newly added test cases have no historical execution information, thus using RETECS to predict their priority is more like ‘random’. In this paper, we focus on new test cases for continuous integration testing, and on the basis of the RETECS framework, we first propose a priority assignment method for new test cases to ensure that they can be executed first. Secondly, continuous integration is a fast iterative integration method where new test cases have strong fault detection capability within the latest periods. Therefore, we further propose an additional reward method for new test cases. Finally, based on the full lifecycle management, the ‘new’ additional rewards need to be terminated within a certain period, and this paper implements an empirical study. We conducted 30 iterations of the experiment on 12 datasets and our best results were 19.24%, 10.67%, and 34.05 positions better compared to the best parameter combination in RETECS for the NAPFD (Normalized Average Percentage of Faults Detected), RECALL and TTF (Test to Fail) metrics, respectively.

Test Case Prioritization (TCP) based on the continuous decision of Reinforcement Learning (RL) has achieved a successful application for test cases optimization in Continuous Integration (CI). The reward functions of RL describe how a test case "ought" to be executed in next integration, of which the design is usually based on the historical executions of the test case. The Average Percentage of Historical Failure (APHF) had been considered as one of the best reward function which has a strong correlation with the recent failure executions of a test case. However, for a test case with many historical failures but passes in recent cycles, the APHF value may be low. In this paper, two novel reward functions are proposed focusing on the impact of failure position in test case history execution sequence, which are the Average Position Exponential Weight (APEW) reward function and the Average Position Quadratic Weight (APQW) reward function, respectively. Both APEW and APQW carry out weight design of failure position but with different weights. We theoretically prove the issue of the only strong correlation with recent failure executions, and also prove that both proposed reward functions can reflect the quantity of historical failures and the distribution of these failures. Experimental verification on 10 industrial-level data sets show that the proposed reward functions can effectively improve the fault detection capability of test cases.

In the continuous integration, test case prioritization can effectively alleviate the resource-intensive problems associated with frequent integration commits. Test case prioritization in continuous integration is a sequential decision problem from which reinforcement learning is applied and can effectively adapt and learn from a changing environment. However, continuous integration testing brings new problems of sparse rewards to reinforcement learning because of frequent integration with low test failure and this problem can be addressed by increasing the number of rewarded test cases. In this paper, we propose a reinforcement learning reward object selection strategy based on Test Case Synchronization and Diversity (TCSD) that rewards failed test cases and with an additional selection of passed test cases with potential failure ability. The experiments on six real-world industrial data sets show that TCSD improves the learning efficiency and fault detection ability of reinforcement learning 6.35% in average NAPFD compared with the traditional strategies.

Continuous Integration (CI) testing is an expensive, time-consuming, and resource-intensive process. Test case prioritization (TCP) can effectively reduce the workload of regression testing in the CI environment, where Reinforcement Learning (RL) is adopted to prioritize test cases, since the TCP in CI testing can be formulated as a sequential decision-making problem, which can be solved by RL effectively. A useful reward function is a crucial component in the construction of the CI system and a critical factor in determining RL’s learning performance in CI testing. This paper focused on the validity of the execution history information of the test cases on the TCP performance in the existing CI testing optimization methods based on RL, and a Dynamic Time Window based reward function are proposed by using partial information dynamically for fast feedback and cost reduction. Experimental studies are carried out on six industrial datasets. The experimental results showed that using dynamic time window based reward function can significantly improve the learning efficiency of RL and the fault detection ability when comparing with the reward function based on fixed time window.

In continuous integration (CI) environments, the program is rapidly and frequently modified and integrated. This feature introduces significant challenges to testing processes conducted in these environments. Based on existing technology, a test case that fails frequently is likely to fail in future tests. Therefore, the historical execution results of test cases are essential to guide the test case prioritization (TCP) in the CI environment. Reinforcement learning involves solving sequential decision-making problems and is suitable for TCP in the CI environment. At present, most of the TCP techniques based on reinforcement learning rely on the current cycle historical failure information of test cases. They rarely consider more historical cycle information, as well as other influencing factors. In this paper, we discussed the occurrence frequency of test cases for the first time. We also considered all historical information of each test case and proposed three new reward function, which employs the percentage of historical failure and the failure distribution of test cases, which can guide the reinforcement learning process. We evaluate our method on five industrial data sets. The experimental results show that our method can effectively prioritize test cases and improve the cost-effectiveness of the CI process.

Continuous integration refers to the practice of merging the working copies of all developers into the mainline frequently. Regression testing for each mergence is characterized by continually changing test suite, limited execution time, and fast feedback, which demands new test optimization techniques. Reinforcement learning is introduced for test case prioritization to save computing resources in continuous integration environment, where a reasonable reward function is highly important for learning strategy, since the process of reinforcement learning is a reward-guided behavior. In this paper, APHFW, a novel reward function is proposed by using partial historical information of test cases effectively for fast feedback and cost reduction. The experiments are based on three open-source data sets, and the results show that the proposed reward function is more cost-effect than other reinforcement learning rewards in continuous integration environment.