Continuous integration and delivery (CI/CD) have transformed software development by reducing delivery time, improving product quality, and giving enterprises a competitive advantage. However, large-scale projects confront difficulties in giving fast feedback to developers due to large test suites, resulting in longer testing cycles and lower productivity. Traditional regression testing methods struggle to find a balance between efficacy and efficiency, demanding advanced approaches. This study investigates the use of machine learning (ML), specifically Neural Network and Random Forest models, to choose test cases based on source code changes, commit messages, and change file path in order to offer developers with faster feedback. The study investigates the predicted accuracy of ML models using a large industrial dataset from a telecom company, which included 15 million test executions over 15 months. The results show that Random Forest outperforms Neural Network models in test case selection, with up to 97% accuracy achieved. Real-time evaluations conducted over a month show significant savings in test executions (88 % -90 %) and testing time (44 % -74%) across multiple regression testing activities, illustrating the potential of ML-driven techniques to optimize CI/CD pipelines and increase developer productivity.

Many test prioritisation techniques have been proposed in order to improve test effectiveness of Continuous Integration (CI) pipelines. Particularly, diversity-based testing (DBT) has shown promising and competitive results to improve test effectiveness. However, the technical and practical challenges of introducing test prioritisation in CI pipelines are rarely discussed, thus hindering the applicability and adoption of those proposed techniques. This research builds on our prior work in which we evaluated diversity-based techniques in an industrial setting. This work investigates the factors that influence the adoption of DBT both in connection to improvements in test cost-effectiveness, as well as the process and human related challenges to transfer and use DBT prioritisation in CI pipelines. We report on a case study considering the CI pipeline of Axis Communications in Sweden. We performed a thematic analysis of a focus group interview with senior practitioners at the company to identify the challenges and perceived benefits of using test prioritisation in their test process. Our thematic analysis reveals a list of ten challenges and seven perceived effects of introducing test prioritisation in CI cycles. For instance, our participants emphasized the importance of introducing comprehensible and transparent techniques that instill trust in its users. Moreover, practitioners prefer techniques compatible with their current test infrastructure (e.g., test framework and environments) in order to reduce instrumentation efforts and avoid disrupting their current setup. In conclusion, we have identified tradeoffs between different test prioritisation techniques pertaining to the technical, process and human aspects of regression testing in CI. We summarize those findings in a list of seven advantages that refer to specific stakeholder interests and describe the effects of adopting DBT in CI pipelines.

Abstract Several operations, ranging from regular code updates to compiling, building, testing, and distribution to customers, are consolidated in continuous integration and delivery. Professionals seek additional information to complete the mission at hand during these tasks. Developers who devote a large amount of time and effort to finding such information may become distracted from their work. We will better understand the processes, procedures, and resources used to deliver a quality product on time by defining the types of information that software professionals seek. A deeper understanding of software practitioners' information needs has many advantages, including remaining competitive, growing knowledge of issues that can stymie a timely update, and creating a visualisation tool to assist practitioners in addressing their information needs. This is an extension of a previous work done by the authors. The authors conducted a multiple‐case holistic study with six different companies (38 unique participants) to identify information needs in continuous integration and delivery. This study attempts to capture the importance, frequency, required effort (e.g. sequence of actions required to collect information), current approach to handling, and associated stakeholders with respect to identified needs. 27 information needs associated with different stakeholders (i.e. developers, testers, project managers, release team, and compliance authority) were identified. The identified needs were categorised as testing, code & commit, confidence, bug, and artefacts. Apart from identifying information needs, practitioners face several challenges in developing visualisation tools. Thus, 8 challenges that were faced by the practitioners to develop/maintain visualisation tools for the software team were identified. The recommendations from practitioners who are experts in developing, maintaining, and providing visualisation services to the software team were listed.

Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase, hence reducing the confidence of developers during maintenance as well as in the quality of a product. Re-running test cases to reveal flakiness is resource-consuming, unreliable and does not reveal the root causes of test flakiness. Our paper evaluates a multi-factor approach to identify flaky test executions implemented in a tool named MDF laker. The four factors are: trace-back coverage, flaky frequency, number of test smells, and test size. Based on the extracted factors, MDFlaker uses k-Nearest Neighbor (KNN) to determine whether failed test executions are flaky. We investigate MDFlaker in a case study with 2166 test executions from different open-source repositories. We evaluate the effectiveness of our flaky detection tool. We illustrate how the multi-factor approach can be used to reveal root causes for flakiness, and we conduct a qualitative comparison between MDF laker and other tools proposed in literature. Our results show that the combination of different factors can be used to identify flaky tests. Each factor has its own trade-off, e.g., trace-back leads to many true positives, while flaky frequency yields more true negatives. Therefore, specific combinations of factors enable classification for testers with limited information (e.g., not enough test history information).

Summary Identifying the root causes of test flakiness is one of the challenges faced by practitioners during software testing. In other words, the testing of the software is hampered by test flakiness. Since the research about test flakiness in large‐scale software engineering is scarce, the need for an empirical case‐study where we can build a common and grounded understanding of the problem as well as relevant remedies that can later be evaluated in a large‐scale context is a necessity. This study reports the findings from a multiple‐case study. The authors conducted an online survey to investigate and catalogue the root causes of test flakiness and mitigation strategies. We attempted to understand how practitioners perceive test flakiness in closed‐source development, such as how they define test flakiness and what practitioners perceive can affect test flakiness. The perceptions of practitioners were compared with the available literature. We investigated whether practitioners' perceptions are reflected in the test artefacts such as what is the relationship between the perceived factors and properties of test artefacts. This study reported 19 factors that are perceived by professionals to affect test flakiness. These perceived factors are categorized astest code,system under test,CI/test infrastructure, andorganization‐related. The authors concluded that some of the perceived factors in test flakiness in closed‐source development are directly related to non‐determinism, whereas other perceived factors concern different aspects, for example, lack of good properties of a test case, deviations from the established processes, and ad hoc decisions. Given a data set from investigated cases, the authors concluded that two of the perceived factors (i.e., test case size and test case simplicity) have a strong effect on test flakiness.

Requirements engineering activities act as a backbone of software development. The more efforts devoted during requirements engineering activities guarantee a better software product. Appropriate selection of requirements has been a challenge for software industry. This selection will increase the probability of success of the software product. Each year many cases are registered against companies for not fulfilling product requirements appropriately. The product failure mostly depends on, either by missing important requirements or capturing irrelevant requirements. SDLC consists of stages where software starts from scratch to a refined product. Requirements Development Life cycle (RDLC) consists of stages where requirements gets initiated, raised, refined, forcefully changed, implemented and validated. The processes to capture requirements vary industry to industry. This paper presents several requirements engineering processes used during the development of requirements, in industry. These processes will identify appropriate requirements and develop a quality product within budget on time. These practices are captured within the Pakistan software industry. This paper also explains the motivations for selecting particular methods, within company, during requirements development and the results associated with it. The processes captured in this paper, from different companies, can be an education for software industry.