Context: Selenium is a widely used tool for end-to-end (E2E) web testing. However, it is often criticized for brittleness, slowness, and flakiness. In parallel, newer frameworks and Artificial Intelligence (AI) are reshaping the test automation landscape. Objectives: This study aims to investigate current practices, challenges, and emerging trends in Selenium-based test automation. Methods: We designed and executed a large-scale online survey targeting software professionals who use Selenium. The questionnaire covered technical practices, tooling, AI usage, perceived challenges, and competing tools. Results: A total of 88 complete responses were analyzed using descriptive statistics and thematic coding. The results show that Selenium remains the dominant tool for regression and functional testing, primarily using the Page Object Model (POM) pattern. The most reported challenges are related to assertability, asynchrony, and brittleness. AI tools like ChatGPT are gaining traction for test generation. Playwright is the most prominent alternative. Conclusion: While Selenium is recognized as a cornerstone in many automation workflows, its limited native test-specific features present a significant drawback. The findings indicate an increasing demand for testing-focused improvements within the Selenium ecosystem, as well as for enhanced integration with AI-driven development tools.

Web applications are essential in modern society and require thorough testing to ensure reliability and dependability. End-to-End (E2E) testing is key to evaluating both front-end and back-end components of complex web applications. Recent research has focused on improving E2E test suites by reducing maintenance costs, mitigating flakiness, and enhancing test resilience. Yet, a major challenge remains the lack of a publicly available benchmark for comparing techniques. Developing test suites for research is complex and prone to bias, highlighting the need for a shared benchmark to facilitate comparisons and experimental validation. We address this gap by providing a set of test suites designed as a benchmark for E2E testing studies. They come with scripts and automated installers for seamless deployment of the application under test in Docker containers, enhancing their usability. Our benchmark consists of 36 Java Selenium WebDriver-based E2E test suites for 8 different web applications. Each application includes multiple test suites with varying characteristics, such as the use of the Page Object pattern and advanced waiting mechanisms. Additionally, for 4 out of the 8 web applications, we provide test suites for two different versions, enabling studies on test suite evolution. Our benchmark includes 1166 test scripts, featuring 492 Page Objects and 8284 locators. To manage asynchronous behavior and reduce flakiness, the test suites incorporate 718 thread sleeps and 815 explicit waits. With a total of 51838 lines of code (LOC) this dataset can potentially be used as a resource, for example, to evaluate test automation strategies, study test suite evolution, or explore flakiness mitigation techniques.

In the medical field, the use of sensors and wearable devices is now widely regarded as a routine support technique for clinical evaluations. Given the advancements in activity recognition from wearable devices in recent years, it is reasonable to explore the potential of similar data as clinical assessment tools for monitoring the progression of chronic diseases. In the current state of the art, datasets collected using wearable devices for subjects with diseases are relatively rare, and their availability becomes even more limited when focusing on pediatric subjects. Therefore, we decided to record a new dataset in collaboration with the Istituto Giannina Gaslini (Genoa, Italy), a center of excellence in pediatric rheumatology. In this article, we present and describe a dataset collected using accelerometers in FDA-approved wearable devices, positioned on the wrist and ankle of the subjects. This dataset included patients between the ages of 2 and 18 years with chronic diseases and age-matched healthy children as a control group. The activities of daily living to be recorded were selected in collaboration with medical specialists, as the diseases considered may potentially affect functional abilities. This shared dataset could enable the scientific community to develop machine learning-based methods to assess severity and monitor the progression of these diseases in a remote, objective, and non-intrusive manner. The recorded dataset is published and fully accessible on the Harvard Dataverse web portal.

End-to-end (E2E) testing is an approach in which an application is automatically tested through scripts that simulate the actions a user would perform. Properly managing asynchronous interactions is crucial in this approach to avoid test failures and flakiness. In the Selenium WebDriver framework, this is typically addressed by using thread sleeps (which pause the test for a fixed time) or explicit waits (function calls that pause the test execution until a specified condition is met). Explicit waits require the selection of both a condition to wait for (e.g., element visibility, element clickability) and an element on which that condition applies. Since thread sleeps are unreliable and replacing them with appropriate explicit waits is a time consuming task, in this work, we leverage a Large Language Model (LLM) to assist testers in selecting the most appropriate explicit waits. We defined a structured procedure (a series of prompts) for engaging with the LLM and validated this approach empirically on three test suites affected by asynchronous waiting issues, as well as on 12 synthetic examples. Additionally, we compared our approach with SleepReplacer, the current state-of-the-art tool for replacing thread sleeps with explicit waits in E2E web test suites. The results show that the LLM-based approach can automatically replace the majority of thread sleeps in a test suite on the first attempt, outperforming SleepReplacer.

Context: While test automation offers numerous benefits, it also introduces significant challenges. Two challenges that developers and testers face on a daily basis, particularly when using Selenium WebDriver to test web applications, are driver management (involving tasks such as version identification, download, installation, and maintenance) and management of test lifecycle phases (using specific test libraries, as for example JUnit, and inserting annotations into the code). These manual tasks make test suite development particularly tedious, error-prone, and expensive. Recently, to ease the burden on developers and testers, some Java libraries have been proposed, called WebDriverManager and Selenium-Jupiter, capable of automatically carrying out the driver management process for Selenium WebDriver and simplifying the development of test suites. These libraries appear to be very promising but until now no one has experimentally evaluated their effectiveness. Objective: To investigate the effectiveness of WebDriverManager and Selenium-Jupiter in reducing driver management times and boilerplate code. Method: We designed and conducted a family of experiments (three for WebDriverManager and two for Selenium-Jupiter) with 104 master student participants from the University of Genoa, Italy (across academic years 2021/2022 and 2022/2023) and nine professional participants. Results: Results indicate that the adoption of Selenium WebDriver with WebDriverManager significantly reduces setup time for multi-browser test suites from 33% to 50% (depending on the tester experience). Additionally, Selenium-Jupiter reduces test suite development time significantly (20% on average). Although it also decreases total code length, the reduction is relatively small compared to overall code length. Conclusion: WebDriverManager and Selenium-Jupiter can be seen as valuable solutions for enhancing testers’ productivity by shortening the time needed to develop test suites and minimizing the amount of code to write.

Web applications quality is commonly assessed by executing End-to-End (E2E) test scripts interacting with those systems as a human tester would. To avoid setting up the web application state for each test script, testers usually create test scripts that may depend on others previously executed. However, the presence of dependencies prevents parallelization, a fundamental technique for speedup the execution of large test suites. In this paper, we present Stile , a tool for parallelizing the execution of E2E web test scripts that generates and executes a set of test schedules satisfying two important constraints: (1) every schedule respects existing test dependencies, and (2) all test scripts in the test suite are executed at least once. Moreover, Stile optimizes the execution by running only once the test scripts that are shared among the schedules. We empirically evaluated Stile on eight E2E test suites by comparing the execution time of Stile both with the sequential execution and with the parallel execution based on Selenium Grid. Our results show that Stile can reduce the execution time up to 80% w.r.t. the sequential execution and up to 50% w.r.t. Grid. Moreover, Stile provides a reduction in the CPUs usage (i.e., overall CPU-time) up to 75%.

Automated testing is vital for ensuring the reliability of web applications. This paper presents a preliminary study on leveraging artificial intelligence (AI) models, specifically ChatGPT and Github Copilot, to generate test scripts for web end-to-end testing. Through experimentation, we evaluated the feasibility and effectiveness of AI language models in generating test scripts based on natural language descriptions of user interactions with web applications.

Abstract A new advancement in test automation is the use of natural language processing (NLP) to generate test cases (or test scripts) from natural language text. NLP is innovative in this context and promises of reducing test cases creation time and simplifying understanding for “non‐developer” software testers as well. Recently, many vendors have launched on the market many proposals of NLP‐based tools and testing frameworks but their superiority has never been empirically validated. This paper investigates the adoption of NLP‐based test automation in the web context with a series of case studies conducted to compare the costs of the NLP testing approach—measured in terms of test cases development and test cases evolution—with respect to more consolidated approaches, that is, programmable (or script‐based) testing and capture&replay testing. The results of our study show that NLP‐based test automation appears to be competitive for small‐ to medium‐sized test suites such as those considered in our empirical study. It minimizes the total cumulative cost (development and evolution) and does not require software testers with programming skills.

Abstract Mutation testing is an important technique able to evaluate the bug-detection effectiveness of existing software test suites. Mutation testing tools exist for several languages, e.g., Java and JavaScript, but no solutions are available for managing the mutation testing process for entire web applications, in the context of end-to-end (E2E) web testing. In this paper, we propose Mutta, a novel tool able to automate the entire mutation testing process. Mutta mutates the various server source files of the target web application, runs the E2E test suite against the mutated web applications, and finally collects the test outcomes. To evaluate Mutta, we designed a case study using the mutated versions of the target web application with the aim of comparing the effectiveness of two different approaches to E2E web testing: (1) test cases based on classical assertions and (2) test cases relying on differential testing. In detail, Mutta has been executed on two web applications, each equipped with different test suites to compare assertions with differential testing. In this scenario, Mutta generated a large number of mutants (more than 15k overall), took into account the coverage information to consider only the mutants actually executed, deployed the mutated web app, ran the entire E2E test suites (about 87k tests runs overall), and finally, it correctly saved the test suite results. Thus, results of the case study show that Mutta can be successfully employed to automate the entire mutation testing process of E2E web test suites and, therefore, can be used in practice to evaluate the effectiveness of different test suites (e.g., based on different techniques, E2E frameworks, or composed by a different number of test scripts).

Modern web applications are complex and used for tasks of primary importance, so their quality must be guaranteed at the highest levels. For this reason, testing techniques (e.g., end-to-end) are required to validate the overall behavior of web applications. One of the most popular tools for testing web applications is Selenium WebDriver. Selenium WebDriver automates the browser to mimic real user actions on the web.While Selenium has made testing easier for many Teams worldwide, it still has its share of challenges. To better understand the challenges and the corresponding solutions adopted we decided to undertake a personal opinion survey from the industry (in total with 78 highly skilled participants) with a focus on the Selenium ecosystem.The results allow understanding which challenges are consid-ered more relevant by professionals in their daily practice and which are the techniques, approaches, and tools they adopt to face them. Therefore, this study is useful to (1) practitioners interested in understanding how to solve the problems they face every day and (2) researchers interested in proposing innovative solutions to problems having a solid industrial impact.

In software engineering, observability is the ability to determine the current state of a software system based on its external outputs or signals such as metrics, logs, or traces. Web engineers rely on the web browser console as the primary tool to monitor the client-side of web applications during end-to-end tests. However, this is a manual and time-consuming task due to the different browsers available. This paper presents BrowserWatcher, an open-source browser extension providing cross-browser capabilities to observe web applications and automatically gather browser console logs in different browsers (e.g., Chrome, Firefox, or Edge). We have leveraged this extension to conduct an empirical study analyzing the browser console of the top-50 public websites manually and automatically. The results show that BrowserWatcher gathers all the well-known log categories such as console or error traces. It also reveals that each web browser additionally includes other types of logs, which differ among browsers, thus providing distinct pieces of information for the same website.

Abstract Tests, when not correctly implemented, can pass on incorrect system implementations rather than fail. In this case, they are named silent horrors or false‐negative tests. They make releasing low‐quality (buggy) versions of the software system more probable. Furthermore, faithfully implementing test specifications is crucial when they play the role of documentation, like when documenting components or services or driving legacy systems' re‐engineering. This paper presents TestWizard, a novel approach and tool for automatically assessing individual tests' quality from the point of view of their coherence to specifications. TestWizard automatically assesses the quality of each individual test case w.r.t. its specification, providing detailed reports on why a single test is a false negative, hence helping testers fix them. Thus, TestWizard can help to automate the test code review process, which is still mainly manual today. The analysis of 1012 test implementations, developed by 123 students in three experiments, shows that TestWizard is (1) by far more accurate than code review performed by multiple students, (2) slightly better than code review performed by three senior experts, and (3) always able to detect a significant percentage of false‐negative test methods (up to 21.22%).

Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This article proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be 4 milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.

End-to-end automated test scripts are a great way to ensure the quality of web applications, but are often perceived as expensive both during their initial development and subsequent maintenance activities. However, maintenance costs can be re-duced when test scripts adopt the Page Object (PO) pattern, a sort of web page facade exposing methods to the test scripts. In this work, we proposed ASSESSOR, a novel tool capable of reducing the effort needed for building PO-based Selenium WebDriver test suites. ASSESSOR allows to simply record the test cases, with only a few additional steps compared to Selenium IDE, and then to automatically generate PO-based WebDriver test suites. The in-depth evaluation performed with four web applications shows that ASSESSOR's adoption allows to reduce the development effort of PO-based web test scripts compared to the classic manual approach: 59% time reduction overall, corresponding to a 2.44 increment in productivity.

Reproducibility of proposed approaches is a crucial element in scientific fields, in order to let other researchers trust published works. Moreover, in order to let authors compare the effectiveness of a novel method to the state of the art, benchmark datasets should be commonly used. Concentrating on the task of activity recognition using data coming from wearable devices with inertial sensors, we have analyzed the reproducibility of proposed approaches with a focus on used datasets. In this work, with a literature review, we have measured what percentage of works in the literature verified their approach using public datasets or sharing the ones created on purpose. At the same time, we have also examined the characteristics of considered datasets, with attention to the amount of data recorded, involved population, and studied activities. Starting from 1289 works retrieved on Scopus, we analyzed in detail 146 of them and found out that approximately one out of three (~33%) used public datasets and that less than one out of three (~28%) of the specially made datasets were shared with the public. Moreover, considering all the examined datasets, 13% of them had restricted access (e.g. requiring requests to authors or subscriptions to websites for a fee) or were offline.

Abstract In the last few years, Internet of Things (IoT) systems have drastically increased their relevance in many fundamental sectors. For this reason, assuring their quality is of paramount importance, especially in safety-critical contexts. Unfortunately, few quality assurance proposals for assuring the quality of these complex systems are present in the literature. In this paper, we extended and improved our previous approach for semi-automated model-based generation of executable test scripts. Our proposal is oriented to system-level acceptance testing of IoT systems. We have implemented a prototype tool taking in input a UML model of the system under test and some additional artefacts, and producing in output a test suite that checks if the system’s behaviour is compliant with such a model. We empirically evaluated our tool employing two IoT systems: a mobile health IoT system for diabetic patients and a smart park management system part of a smart city project. Both systems involve sensors or actuators, smartphones, and a remote cloud server. Results show that the test suites generated with our tool have been able to kill 91% of the overall 260 generated mutants (i.e. artificial bugged versions of the two considered systems). Moreover, the optimisation introduced in this novel version of our prototype, based on a minimisation post-processing step, allowed to reduce the time required for executing the entire test suites (about -20/25%) with no adverse effect on the bug-detection capability.

Abstract Assuring quality of web applications is fundamental, given their relevance in the today’s world. A possible way to reach this goal is through end-to-end (E2E) testing, an approach in which a web application is automatically tested by performing the actions that a user would do. With modern web applications (for example, single-page applications), it is of great importance to properly handle asynchronous calls in the test suite. In E2E Selenium WebDriver test suites, asynchronous calls are usually managed in two ways: using thread sleeps or explicit waits. The first is easier to use, but is inefficient and can lead to instability (also called flakiness, a problem often present in test suites that makes us lose confidence in the testing phase), while the second is usually more efficient but harder to use because, if the correct kind of wait is not carefully selected, it can introduce flakiness too. To help Testers, who often opt for the first strategy, we present in this work a tool-based approach to automatically replace thread sleeps with explicit waits in an E2E Selenium WebDriver test suite without introducing new flakiness. We empirically validated our tool named SleepReplacer on four different test suites, and we found that it can correctly replace in an automatic way from 81 to 100% of thread sleeps, leading to a significant reduction of the total execution time of the test suite (i.e., from 13 to 71%).

Automated end-to-end (E2E) Web testing relying on frameworks such as Selenium Web Driver is commonly used to assess the quality of web applications. However, the resulting test scripts may require long execution times, due to their interaction with the browser GUI and backend services. To avoid repeated and costly setup of the Web application state, testers tend to build test suites whose test scripts depend on each other (i.e., one test case sets up the application state expected by another test case). In this paper we present Stile, a tool for the parallel execution of Web test scripts that ensures the compliance of all execution schedules with the dependencies among the involved test scripts, while at the same time minimizing the execution time and the computation time required for such parallel execution. Experimental results show that execution times can be approximately halved thanks to Stile.

Automated recognition of Activities of daily living is a significant task since it allows monitoring patients remotely using wearable devices. It could also help doctors and specialists to easily track the health status of elderly people and of patients suffering from a variety of geriatric diseases. Most of the literature regarding activity recognition does not face the problem of classifying activities of interest performed among activities that are not, i.e., unknown activities for the classifier. In this paper, we propose a novel method, based on the acceleration data recorded by wearable smartwatches, aimed at improving the activity recognition accuracy in presence of unknown activities (i.e., in real-world settings). The approach is based on an ensemble of different classifiers, a filtering procedure, and a final voting mechanism. The empirical evaluation, carried out on 8 different subjects, shows that our approach improves the overall F1 score by 5.5%, increases the recognition of unknown activities by 11.1%, and decreases the amount of data wrongly classified as an unknown activity by 15.5%. The first two observed differences are respectively statistically significant (Wilcoxon test p-value<0.01).

Abstract In this paper, we present Precise SOM (Precise Service Oriented Modelling) —a novel lightweight method for integrated domain and business process modelling—which follows the service‐oriented paradigm, uses a UML profile as notation and provides detailed workflows to guide the production of the models. In our method, the UML models are precisely defined by means of a metamodel and of a set of constraints, and by restricting UML to the essential language constructs, to help modellers to avoid common mistakes and to guarantee, by construction, a good quality. Precise SOM has been validated by detailing how it can be used in various modelling tasks, some of them illustrated by a (industrial) case study.

Summary By ensuring adequate functional coverage, End‐to‐End (E2E) testing is a key enabling factor of continuous integration. This is even more true for web applications, where automated E2E testing is the only way to exercise the full stack used to create a modern application. The test code used for web testing usually relies on DOM locators, often expressed as XPath expressions, to identify the web elements and to extract the data checked in assertions. When applications evolve, the most dominant cost for the evolution of test code is due to broken locators, which fail to locate the target element in the novel versions and must be repaired. In this paper, we formulate the robust XPath locator generation problem as a graph exploration problem, instead of relying on ad‐hoc heuristics as the one implemented by the state of the art tool robula+. Our approach is based on a statistical adaptive algorithm implemented by the tool sidereal, which outperforms robula+'s heuristics in terms of robustness by learning the potential fragility of HTML properties from previous versions of the application under test. sidereal was applied to six applications and to a total of 611 locators and was compared against two baseline algorithms, robula+ and Montoto. The adoption of sidereal results in a significant reduction of the number of broken locators (respectively ‐55% and ‐70%). The time for generating such robust locators was deemed acceptable being in the order of hundredths of second.

Automated web testing is an appealing option, especially when continuous testing practices are adopted. However, web test cases are known to be fragile and to break easily when a web application evolves. The Page Object (PO) design pattern addresses such problem by providing a layer of indirection that decouples test cases from the internals of the web page, where web page elements are located and triggered by the web tests. However, PO development could potentially introduce an additional burden to the already strictly constrained testing activities. This paper reports an empirical investigation of costs and benefits due to the introduction of the PO pattern in web test suite development. In particular, we conducted a family of controlled experiments in which test cases were developed with and without the PO pattern. While the benefits of POs did not compensate for the extra development effort they require in the limited experimental setting of our study, results indicate that when the test suite to be developed is at least 10× larger, test development becomes more efficient with than without POs.

Assuring the quality of Internet of Things (IoT) systems is of paramount importance, and guaranteeing their reliability and compliance with the requirements is mandatory, but few attempts have been made so far. In previous works, we proposed two approaches for acceptance testing and runtime verification of IoT systems. Both works rely on a UML state machine to specify the system expected behaviour. In the acceptance testing approach, the interesting paths to exercise are identified and translated into executable test scripts. In the runtime verification approach, the relevant events during the system execution are monitored and compared against a formal specification derived from the UML state machine. In this paper, we compare the effectiveness of our two approaches, by applying them to a mobile health IoT system for the management of diabetic patients, employing over 100 mutated versions of the original system and analysing more than 1000 different executions. Results show that both approaches are effective in different ways in detecting bugs. While the acceptance testing approach is more effective to detect the bugs affecting the user interface, the runtime verification approach tracks better the subtle deviations from the system expected behaviour, in particular those concerning network issues.

When modellers have to describe a business process, they can decide to rely on one of the several notations currently available such as for instance BPMN or UML. However, it is not easy to select the most appropriate one. Indeed, even though in the literature several comparisons among notations have been proposed, often they focus on specific properties of the notations or on their constructs. On the contrary, the real concern of the modeller is to select the best modelling method, based on a notation they know for a specific modelling case. For this reasons, in this work we further raise the level of comparison by proposing an approach based on matching the features, required by the modelling case that the modeller has to face, with the ones supported by available modelling methods based on those notations. In this way the modeller can really select the best method (based on a modelling notation) according to the needs of the specific modelling case.

Currently, several notations can be used for modelling business processes, and various researchers tried to compare them focusing on their specific properties or constructs. We propose an approach that, instead of resulting in selecting a notation, drives to pair specific business process modelling cases with suitable modelling methods, by matching the features required by the former with the ones supported by the latter.

Internet of things (IoT) systems are becoming ubiquitous and assuring their quality is fundamental. Unfortunately, a few proposals for testing these complex, and often safety‐critical, systems are present in the literature. The authors propose an approach for acceptance testing of IoT systems adopting graphical user interfaces as a principal way of interaction. Acceptance testing is a type of black box testing based on test scenarios, i.e. sequences of steps/actions performed by the user or the system. In their approach, test scenarios are derived from a state machine that expresses the behaviour of the system under test, and test cases are derived from them by specifying the actual data and assertions and made executable by implementing the corresponding test scripts. As a case study, they selected a mobile health IoT system for diabetes management composed of local sensors/actuators, smartphones, and a remote cloud‐based system. The effectiveness of the approach has been evaluated by measuring the capability of two test suites implemented using different localisation strategies (visual and structure‐based) in detecting mutants of the original m‐health system. Results show the effectiveness of the test suites implemented by following the proposed approach since 93% of the generated mutants have been detected.

Summary Test automation tools are widely adopted for testing complex Web applications. Three generations of tools exist: first, based on screen coordinates; second, based on DOM–based commands; and third, based on visual image recognition. In our previous work, we proposed Pesto, a tool able to migrate second‐generation Selenium WebDriver test suites towards third‐generation Sikuli ones. In this work, we extend Pesto to manage Web elements having (1) complex visual interactions and (2) multiple visual appearances. Pesto relies on aspect‐oriented programming, computer vision, and code transformations. Our new improved tool has been evaluated on two Web test suites developed by an independent tester. Experimental results show that Pesto manages and transforms correctly test suites with Web elements having complex visual interactions and multistate elements. By using Pesto, the migration of existing DOM–based test suites to the visual approach requires a low manual effort, since our approach proved to be very accurate.

Automated test scripts are used with success in many web development projects, so as to automatically verify key functionalities of the web application under test, reveal possible regressions and run a large number of tests in short time. However, the adoption of automated web testing brings advantages but also novel problems, among which the test code fragility problem. During the evolution of the web application, existing test code may easily break and testers have to correct it. In the context of automated DOM-based web testing, one of the major costs for evolving the test code is the manual effort necessary to repair broken web page element locators – lines of source code identifying the web elements (e.g. form fields and buttons) to interact with. In this work, we present Robula+, a novel algorithm able to generate robust XPath-based locators – locators that are likely to work correctly on new releases of the web application. We compared Robula+ with several state of the practice/art XPath locator generator tools/algorithms. Results show that XPath locators produced by Robula+ are by far the most robust. Indeed, Robula+ reduces the locators' fragility on average by 90% w.r.t. absolute locators and by 63% w.r.t. Selenium IDE locators. Copyright © 2016 John Wiley & Sons, Ltd.

The main reason for the fragility of web test cases is the inability of web element locators to work correctly when the web page DOM evolves. Web elements locators are used in web test cases to identify all the GUI objects to operate upon and eventually to retrieve web page content that is compared against some oracle in order to decide whether the test case has passed or not. Hence, web element locators play an extremely important role in web testing and when a web element locator gets broken developers have to spend substantial time and effort to repair it. While algorithms exist to produce robust web element locators to be used in web test scripts, no algorithm is perfect and different algorithms are exposed to different fragilities when the software evolves. Based on such observation, we propose a new type of locator, named multi-locator, which selects the best locator among a candidate set of locators produced by different algorithms. Such selection is based on a voting procedure that assigns different voting weights to different locator generation algorithms. Experimental results obtained on six web applications, for which a subsequent release was available, show that the multi-locator is more robust than the single locators (about -30% of broken locators w.r.t. the most robust kind of single locator) and that the execution overhead required by the multiple queries done with different locators is negligible (2-3% at most).

UML is a large notation offering many diagrams and a large set of constructs for each of them covering any possible modelling need. As a result its specification is a huge book, its metamodel is large, and defining/understanding its static and dynamic semantics is difficult. These features have a negative impact on the perception of the UML and lead in some cases to replace it by ad-hoc lean and simple DSLs. On the other hand, people naturally tend to downsize UML considering only a part of its constructs. Thus, the following question arises: which are the most/less used UML diagrams/constructs? We would like to answer to this question by means of a survey, trying to detect which parts of the UML are the most used. In this work, we focus our attention on the usage of the Activity Diagram constructs. To see how much a construct is used we preliminarily investigate books, ourses/tutorials, and tools covering UML. As future work, we will conduct a personal opinion survey on the same topic.

Automated testing of web applications reduces the effort needed in manual testing. Old 1st generation tools, based on screen coordinates, produce quite fragile test suites, tightly coupled with the specific screen resolution, window position and size experienced during test case recording. These tools have been replaced by a 2nd generation of tools, which offer easy selection and interaction with the web elements, based on DOM-oriented commands. Recently, a new 3rd generation of tools came up based on visual image recognition, bringing the promise of wider applicability and simplicity. A tester might ask if the migration towards such new technology is worthwhile, since the manual effort to rewrite a test suite might be overwhelming. In this paper, we propose PESTO, a tool facing the problem of the automated migration of 2nd generation test suites to the 3rd generation. PESTO determines automatically the screen position of each web element located on the DOM by a 2nd generation test case. It then calculates a screenshot image centred around the web element so as to ensure unique visual matching. Then, the entire source code of the DOM-based test suite is transformed into a visual test suite, based on such automatically extracted images and using specific visual commands.

This paper reports the results and some challenges we discovered during the design and execution of a pilot experiment with 21 bachelor students aimed at investigating the effect of documentation accuracy during software maintenance and evolution activities. As documentation we considered: a high level system functionality description and UML documents. Preliminary results indicate a benefit of +15% in terms of efficiency (computed as number of correct tasks per minute) when a more accurate documentation is used. The discovered challenging aspects to carefully consider in future executions of the experiment are as follows: selecting "the right" documentation artefacts, maintenance tasks and documentation versions, verifying that the subjects really used the documentation during the experiment and measuring documentation-code alignment.

This poster presents an industrial case study about test automation and test suite maintenance in the context of Web applications. The Web application under test is a Learning Content Management System (eXact learning LCMS). We analysed the costs associated with the realignment of four equivalent Selenium WebDriver test suites, implemented using the page object pattern and different methods to locate web page elements, to a subsequent release of eXact learning LCMS. In our study, the two ID-based test suites required significantly less maintenance effort than the XPath-based ones.

In this poster, we sketch our research plan about a “massive” empirical evaluation of model-driven techniques following the first two already conducted steps in that respect (an exploratory survey and a series of controlled experiments concerning maintainability). We intend to experiment UML-based model-driven techniques in several contexts (e.g., desktop and Web applications), focusing on several software characteristics (e.g., maintainability and productivity) and employing empirical methods such as controlled experiments, surveys, case studies.

We conducted a personal opinion survey in two rounds - years 2008 and 2011 - with the aim of investigating the level of knowledge and adoption of SOA in the Italian industry. We are also interested in understanding what is the trend of SOA (positive or negative?) and what are the methods, technologies and tools really used in the industry. The main findings of this survey are the following: (1) SOA is a relevant phenomenon in Italy, (2) Web services and RESTFul services are well-known/used and (3) orchestration languages and UDDI are little known and used. These results suggest that in Italy SOA is interpreted in a more simplistic way with respect to the current/real definition (i.e., without the concepts of orchestration/choreography and registry). Currently, the adoption of SOA is medium/low with a stable/positive trend of pervasiveness.

One of the claimed advantages of Model-driven development is the improvement in maintainability. However, few studies consider this aspect from an empirical point of view. This paper reports the results of a controlled experiment with 21 bachelor students aimed at investigating the effectiveness of Model-driven development during software maintenance and evolution activities. The tool used in the experiment is UniMod, a specific implementation of executable UML. Preliminary results indicate a relevant shortening of time with no significant impact on correctness, gained through the use of UniMod instead of conventional programming (i.e., code-centric programming).