This study assesses software testing education in Europe and the Americas through a literature review, dividing selected studies into two categories: 1) the state of software testing education and 2) empirical interventions in universities. The review reveals a limited number of studies, primarily from European and American institutions, and highlights the scarcity of dedicated software testing courses. Although interventions, such as problem-based learning and gamification, have demonstrated improved student engagement and performance, challenges related to time restrictions, instructors' skills, and content management restrict the introduction of such interventions. The study underscores the need for dedicated software testing courses at both undergraduate and master's levels, as these can address issues of time and content management, enable instructors to refine their skills, and better prepare students for industry demands.

Practitioners working in large-scale software development face many challenges in regression testing activities. One of the reasons is the lack of a structured regression testing process. In this regard, checklists can help practitioners keep track of essential regression testing activities and add structure to the regression testing process to a certain extent. This study aims to introduce regression testing checklists so test managers/teams can use them: (1) to assess whether test teams/members are ready to begin regression testing, and (2) to keep track of essential regression testing activities while planning and executing regression tests. We used interviews, workshops, and questionnaires to design, evolve, and evaluate regression testing checklists. In total, 25 practitioners from 12 companies participated in creating the checklist. Twenty-three of them participated in checklists evolution and evaluation. We identified activities practitioners consider significant while planning, performing, and analyzing regression testing. We designed regression testing checklists based on these activities to help practitioners make informed decisions during regression testing. With the help of practitioners, we evolved these checklists into two iterations. Finally, the practitioners provided feedback on the proposed checklists. All respondents think the proposed checklists are useful and customizable for their environments, and 80% think checklists cover aspects essential for regression testing. The proposed regression testing checklists can be useful for test managers to assess their team/team members’ readiness and decide when to start and stop regression testing. The checklists can be used to record the steps required while planning and executing regression testing. Further, these checklists can provide a basis for structuring the regression testing process in varying contexts.

Abstract Regression testing is challenging because of its complexity and the amount of effort and time it requires, especially in large‐scale environments with continuous integration and delivery. Regression test selection and prioritization techniques have been proposed in the literature to address the regression testing challenges, but adoption rates of these techniques in industry are not encouraging. One of the possible reasons could be the disparity in the regression testing goals in industry and literature. This work compares the research perspective to industry practice on regression testing goals, corresponding information needs, and metrics required to evaluate these goals. We have conducted a literature review of 44 research papers and a survey with 56 testing practitioners. The survey comprises 11 interviews and 45 responses to an online questionnaire. We identified that industry and research accentuate different regression testing goals. For instance, the literature emphasizes increasing the fault detection rates of test suites and early identification of critical faults. In contrast, the practitioners' focus is on test suite maintenance, controlled fault slippage, and awareness of changes. Similarly, the literature suggests maintaining information needs from test case execution histories to evaluate regression testing techniques based on various metrics, whereas, at large, the practitioners do not use the metrics suggested in the literature. To bridge the research and practice gap, based on the literature and survey findings, we have created a goal–question–metric (GQM) model that maps the regression testing goals, associated information needs, and metrics from both perspectives. The GQM model can guide researchers in proposing new techniques closer to industry contexts. Practitioners can benefit from information needs and metrics presented in the literature and can use GQM as a tool to follow their regression testing goals.

Abstract Replication studies help solidify and extend knowledge by evaluating previous studies’ findings. Software engineering literature showed that too few replications are conducted focusing on software artifacts without the involvement of humans. This study aims to replicate an artifact-based study on software testing to address the gap related to replications. In this investigation, we focus on (i) providing a step-by-step guide of the replication, reflecting on challenges when replicating artifact-based testing research and (ii) evaluating the replicated study concerning the validity and robustness of the findings. We replicate a test case prioritization technique proposed by Kwon et al. We replicated the original study using six software programs, four from the original study and two additional software programs. We automated the steps of the original study using a Jupyter notebook to support future replications. Various general factors facilitating replications are identified, such as (1) the importance of documentation; (2) the need for assistance from the original authors; (3) issues in the maintenance of open-source repositories (e.g., concerning needed software dependencies, versioning); and (4) availability of scripts. We also noted observations specific to the study and its context, such as insights from using different mutation tools and strategies for mutant generation. We conclude that the study by Kwon et al. is partially replicable for small software programs and could be automated to facilitate software practitioners, given the availability of required information. However, it is hard to implement the technique for large software programs with the current guidelines. Based on lessons learned, we suggest that the authors of original studies need to publish their data and experimental setup to support the external replications.

Abstract Model‐based test case generation techniques provide a mechanism to derive tests systematically. This study provides a systematic mapping of test case generation techniques based on UML interaction diagrams. The study compares the test case generation techniques regarding their capabilities and limitations, and it also assesses the reporting quality of the primary studies. We can conclude that the studies presenting test case generation techniques using UML interaction diagrams were not following the guidelines for research methods (eg, case studies or experiments). Solutions were not empirically evaluated in industrial contexts. Our study revealed that better tool support is needed to introduce the UML interaction diagram–based test case generation techniques in the industry.

There has been a lot of research on personality and its impact on software engineering practice. These studies use different psychological tests to identify personality types of software practitioners. The administration of these tests requires expertise. As the humans are involved, other ethical issues, such as consent, also become important. In this study, we evaluated a small sample of 15 studies that used a psychological test Myers-Briggs Type Indicator (MBTI) in a software engineering context with respect to different ethical issues related to informed consent, qualification of the test administrators and the use of appropriate tests. The results show that most of the studies in our sample seriously lack with respect to various ethical issues.

Requirements prioritization plays a vital role in software release planning, and the process requires good interaction between stakeholders. Requirement prioritization process is challenging in global software development (GSD) context, where communication and coordination are core bottlenecks. Based on the issues found in the literature and industry survey, this paper proposes a requirements prioritization technique for GSD settings based on stakeholders' weight, vote, and priority (SWVP). The technique has two phases: (1) a calculation of stakeholders' profile weights and setting up communication parameters and (2) a voting process supporting the actual prioritization, based on the weights from phase (1). The proposed technique was applied to three industrial projects. The feedback of the practitioners after using SWVP was encouraging, and the feedback results indicate that SWVP could be a practical solution for requirements prioritization in GSD contexts.

Regression testing is a means to assure that a change in the software, or its execution environment, does not introduce new defects. It involves the expensive undertaking of rerunning test cases. Several techniques have been proposed to reduce the number of test cases to execute in regression testing, however, there is no research on how to assess industrial relevance and applicability of such techniques. We conducted a systematic literature review with the following two goals: firstly, to enable researchers to design and present regression testing research with a focus on industrial relevance and applicability and secondly, to facilitate the industrial adoption of such research by addressing the attributes of concern from the practitioners’ perspective. Using a reference-based search approach, we identified 1068 papers on regression testing. We then reduced the scope to only include papers with explicit discussions about relevance and applicability (i.e. mainly studies involving industrial stakeholders). Uniquely in this literature review, practitioners were consulted at several steps to increase the likelihood of achieving our aim of identifying factors important for relevance and applicability. We have summarised the results of these consultations and an analysis of the literature in three taxonomies, which capture aspects of industrial-relevance regarding the regression testing techniques. Based on these taxonomies, we mapped 38 papers reporting the evaluation of 26 regression testing techniques in industrial settings.

Context: Existing empirical evidence indicates that loss of alertness associated with mental fatigue is highly correlated with fluctuations in the performance of people carrying out auditory tasks. In software engineering research, mental fatigue may affect the results of study selection (an auditory task) when conducting secondary studies such as systematic literature reviews or systematic mapping studies. However, to date there is no empirical study that reports an in-depth investigation about the relationship between mental fatigue and researchers' selection decisions during study selection process.