API testing has increasing demands for software companies. Prior API testing tools were aware of certain types of dependencies that needed to be concise between operations and parameters. However, their approaches, which are mostly done manually or using heuristic-based algorithms, have limitations due to the complexity of these dependencies. In this paper, we present KAT (Katalon API Testing), a novel AI -driven approach that leverages the large language model GPT in conjunction with advanced prompting techniques to autonomously generate test cases to validate RESTful APIs. Our comprehensive strategy encompasses various processes to construct an operation depen-dency graph from an OpenAPI specification and to generate test scripts, constraint validation scripts, test cases, and test data. Our evaluation of KAT using 12 real-world RESTful services shows that it can improve test coverage, detect more undocumented status codes, and reduce false positives in these services in comparison with a state-of-the-art automated test generation tool. These results indicate the effectiveness of using the large language model for generating test scripts and data for API testing.

Regression testing of software is a crucial but time-consuming task, especially in the context of user interface (UI) testing where multiple microservices must be validated simultaneously. Test case prioritization (TCP) is a cost-efficient solution to address this by scheduling test cases in an execution order that maximizes an objective function, generally aimed at increasing the fault detection rate. While several techniques have been proposed for TCP, most rely on source code information which is usually not available for UI testing. In this paper, we introduce a multi-objective optimization approach to prioritize UI test cases, using evolutionary search algorithms and four coverage criteria focusing on web page elements as objectives for the optimization problem. Our method, which does not require source code information, is evaluated using two evolutionary algorithms (AGE-MOEA and NSGA-II) and compared with other TCP methods on a self-collected dataset of 11 test suites. The results show that our approach significantly outperforms other methods in terms of Average Percentage of Faults Detected (APFD) and APFD with Cost (APFDc), achieving the highest scores of 87.8% and 79.2%, respectively. We also introduce a new dataset and demonstrate the significant improvement of our approach over existing ones via empirical experiments. The paper’s contributions include the application of web page segmentation in TCP, the construction of a new dataset for UI TCP, and empirical comparisons that demonstrate the improvement of our approach.

Testing Representational State Transfer (REST) APIs is crucial for ensuring the reliability and performance of APIs, which are essential to modern web services. This testing process helps identify and resolve issues related to data exchange and integration with other systems. Among the various API testing techniques, black-box testing relies on the OpenAPI Specification (OAS) to generate test cases and data. However, current API test automation methods are primarily focused on status code [10] and schema validation [1]. Status code validation involves ensuring that each HTTP request returns a response with a status code, a three-digit integer that indicates the outcome of the request. Schema validation verifies the correctness of the response data by comparing it to the schema. This includes checking that all required properties are present and that data types of these properties align with the schema specified.

Building an autonomous intelligent agent capable of carrying out web automation tasks from descriptions in natural language offers a wide range of applications, including software testing, virtual assistants, and task automation in general. However, recent studies addressing this problem often require manually constructing of prior human demonstrations. In this paper, we approach the problem by leveraging the idea of reinforcement learning (RL) with the two-phase mechanism to form an agent using LLMs for automating computer tasks without relying on human demonstrations. We evaluate our LLM-based agent using the MiniWob++ dataset of web-based application tasks, showing that our approach achieves 85% success rate without prior demonstrations. The results also demonstrate the agent's capability of self-improvement through training.

Test flakiness has been a common problem in the software automation testing, affecting the effectiveness and productivity of test automation. Many studies have been published to tackle test flakiness in the software research community, and existing test automation tools provide capabilities to address issues related to test flakiness. This paper describes our review of recent approaches in the research community and popular industrial tools with capabilities to tackle test flakiness. Our review indicates that while many studies focus on test flakiness in unit testing, few address the problem in the end-to-end system testing. On the contrary, industrial test automation tools tend to be more concerned with test flakiness at the system level.

Software testing is a process of evaluating and verifying whether a software product still works as expected, and it is repetitive, laborious, and time-consuming. To address this problem, automation tools have been developed to automate testing activities and enhance quality and delivery time. However, automation tools become less effective with continuous integration and continuous delivery (CI/CD) pipelines when the system under test is constantly changing. Recent advances in artificial intelligence and machine learning (AI/ML) present the potential for addressing important challenges in test automation. AI/ML can be applied to automate various testing activities such as detecting bugs and errors, maintaining existing test cases, or generating new test cases much faster than humans.

The process of creating test cases from requirements written in natural language (NL) requires intensive human efforts and can be tedious, repetitive, and error-prone. Thus, many studies have attempted to automate that process by utilizing Natural Language Processing (NLP) approaches. Furthermore, with the advent of massive language models and transfer learning techniques, people have introduced various advancements in NLP-assisted software testing with promising results. More notably, in recent years, not only have researchers been engrossed in solving the above task, but many companies have also embedded the feature to translate from human language to test cases their products. This paper presents an overview of NLP-assisted solutions being used in both the literature and the software testing industry.

Test case prioritization, which focuses on ranking test cases, is an important activity in software engineering given a large number of test cases to be executed within a short period of time.Recent approaches use test execution history and test coverage as the key information for ranking prediction while reinforcement learning has the potential for improving the accuracy of prioritization.Still, each approach has its own advantages and limitations.This paper proposes a ensemble method to take advantages of several existing models by combining different them into a single one.We evaluate our ensemble models on the data sets, including sixteen projects.The results show that one of our proposed models outperforms all single models on 12 over 16 data sets.

Summary Web user interface (UI) test automation strategies have been dominated by programmable and record–playback approaches. Of these, record–playback allows creating automation tests easily and reduces the cost of test generation. However, this approach increases the cost of test maintenance due to its unstable generated locators for identifying UI objects during playback. In this paper, we propose a new approach to generating and selecting resilient and maintainable locators. Our approach consists of two parts, a new XPath construction method and selecting the best XPath to locate the target element. Our XPath construction method relies on semantic structures of Web pages to locate the target element using its neighbors. We conducted an experiment on 15 popular websites. The results show that our approach outperforms the state‐of‐the‐practice/art Selenium IDE and Robula+ in locating target elements by effectively avoiding wrong locators. It also produces more readable XPaths (hence more maintainable tests) than do these approaches.

Abstract In model‐based software estimation, using the right training data is a key contributor for making accurate predictions, which is crucial for the success of software projects. This study investigates the use of duration‐based windows and estimation by analogy to calibrate COCOMO and assess their estimation performance. We compare these approaches as well as the use of all available historical data using the COCOMO data set of 341 projects and NASA data set of 93 projects. The results show that timing information exists in the data sets affecting estimation accuracy. Given sufficient data for calibration, using recently completed projects within short durations generates more accurate estimates than retaining all historical data or using k‐nearest neighbors based on estimation by analogy. More training data spanning a long period of time may not lead to improved estimation accuracy. This study offers evidence to support the use of projects completed within recent years for training estimation models.

Collaborative programming is an effective approach to software development, improving software quality, programmer's satisfaction and shortening delivery time This study examines the application of a collaborative Web-based IDE named IDEOL to execute a four-week multi-submission programming assignment in an introductory object-oriented programming class. Forty eight students forming 24 two-member groups in class used the IDE to interact and write source code required by the project. All collaborative and programming activities performed by students were recorded by IDEOL. The results of the study shows that students tend to postpone their programming work until the submission dates. This study also provides an approach to designing and executing an extended programming exercises, which receives high student satisfaction. Our results imply that IDEOL is a useful environment for students to collaborate, learn, and practice programming to improve their learning satisfaction. In addition, as students tend to procrastinate, IDEOL is a useful tool to facilitate, monitor, and report student progress in extended programming exercises.

There have been studies suggesting that collaboration and cooperation can deliver higher performance than competition or individual work. The Web does not only provide ubiquitous access to resources and computation power but also can be an open structure for better and continuous collaboration. In this study, we introduce our vision and construction of an integrated social environment called EduCo to assist teaching and learning software engineering courses. EduCo is a Web environment for instructors to teach and for students to learn and practice designing, programming, and managing software in software engineering courses. It is also a social network platform that helps stimulate participation, interaction, sharing, awareness, accountability, and teamwork. This paper describes the initial construction of the system with many core capabilities realized. The paper also presents our case studies from applying the system to several programming language classes. The results from the case studies suggest that the system has the potential to encourage students' participation and satisfaction. In addition, this paper presents our vision for future enhancements of the system with core capabilities such as feeds, dashboards, notifications, tracking, and reporting.

This paper provides a brief description of our study proposing improvements to the COCOMO models for estimating maintenance size and effort. The proposed size and effort models take into account characteristics of software maintenance that have not been addressed in the current COCOMO models. We found that the proposed models potentially improve the estimation accuracies of software maintenance projects.

This paper describes a controlled experiment of student programmers performing maintenance tasks on a C++ program. The goal of the study is to assess the maintenance size, effort, and effort distribution of three different maintenance types and to describe estimation models to predict the programmer's effort on maintenance tasks. The results of our study suggest that corrective maintenance is much less productive than enhancive and reductive maintenance. Our study also confirms the previous results which conclude that corrective and reductive maintenance requires large proportions of effort on program comprehension activity. Moreover, the best effort model we obtained from fitting the experiment data can estimate the time of 79% of the programmers with the error of 30% or less.

Building cost estimation models is often considered a search problem in which the solver should return an optimal solution satisfying an objective function. This solution also needs to meet certain constraints. For example, a solution for the estimates coefficients of COCOMO models must be non-negative. In this research, we introduce a constrained regression technique that uses objective functions and constraints to estimate the coefficients of the COCOMO models. To access the performance of the proposed technique, we run a cross-validation procedure and compare the prediction accuracy from different approaches such as least squares, stepwise, Lasso, and Ridge regression. Our result suggests that the regression model that minimizes the sum of relative errors and imposes non-negative coefficients is a favorable technique for calibrating the COCOMO model parameters.

The problem of channel estimation for spatially correlated fading multiple-input multiple-output (MIMO) systems is considered. Based on the channel's second order statistic, the minimum mean-square error (MMSE) channel estimator that works with the superimposed training signal is first developed. The problem of designing the optimal superimposed signal is then addressed and solved with an iterative optimization algorithm. Results show that under the constraint of equal training power and bandwidth efficiency, our optimal design of the superimposed training signal leads to a significant reduction in channel estimation error when compared to the conventional design of time-multiplexing training, especially for slowly time-varying channels with a large coherence time. The issue of power allocation between the information-bearing and training signals for detection enhancement is also investigated. Simulation results demonstrate excellent bit-error-rate performance of orthogonal space-time block codes with our proposed channel estimation.