Ensuring software quality in embedded firmware is critical, especially in safety-critical domains such as automotive systems, where compliance with functional safety standards like ISO 26262 requires strong guarantees of software reliability. While machine learning-based fault prediction models have demonstrated high accuracy, their lack of transparency and interpretability limits their adoption in industrial settings. Developers need actionable insights that can be directly employed in software quality assurance (SQA) processes and guide defect mitigation strategies. In this paper, we present a structured process for defining context-specific software metric thresholds suitable for integration into fault detection workflows in industrial settings. Our approach supports cross-project fault prediction by deriving thresholds from one set of projects and applying them to independently developed firmware, thereby enabling reuse across similar software systems without retraining or domain-specific tuning. We analyze three real-world C-embedded firmware projects provided by an industrial partner, using Coverity and Understand static analysis tools to extract software metrics. Through statistical analysis and hypothesis testing, we identify discriminative metrics and derived empirical threshold values capable of distinguishing faulty from non-faulty functions. The derived thresholds are then validated through an experimental evaluation, demonstrating their effectiveness in identifying fault-prone functions with high precision. The results confirm that statistically derived thresholds can serve as a practical and interpretable solution for fault prediction, aligning with industry standards and SQA practices. This approach provides a practical alternative to black-box AI models, allowing developers to systematically assess software quality, take preventive actions, and integrate metric-based fault prediction into industrial development workflows to mitigate software faults.

Software testing is an important part of the software development lifecycle. As it is a highly sought-after skill in the industry, it is not surprising that there has been a great deal of research into the teaching of software testing in higher education. Most of this research proposes or evaluates pedagogical approaches or software testing tools to assist teachers in educating the next generation of software engineers. These evaluations are often limited to measuring teachers’ opinions about the use of a novel pedagogical approach or an educational tool and students’ acceptance and performance in terms of desired software testing skills. While tools and pedagogical approaches address specific aspects of a course, to date, little attention has been paid to the opinions of the students about all the individual aspects of a software testing course. This paper aims to address this missing student perspective by taking a holistic view of software testing course designs. To address this gap, an exploratory study was performed by distributing a questionnaire to 103 students from ten different courses to gauge their opinions on a software testing course they are enrolled in. The results show that students generally have a positive perception of the different aspects of their software testing course. However, several areas for improvement were suggested based on the gathered data.

When teaching Programming and Software Engineering in Bachelor’s Degree programs, the emphasis on creating functional software projects often overshadows the focus on software quality, a trend consistent with ACM curricula recommendations. Dedicated Software Engineering courses take typically place in the later stages of the curriculum, and allocate only limited time to software quality, leaving educators with the difficult task of deciding which quality aspects to prioritize. To educate students on the importance of developing high-quality code, it is important to introduce these skills as part of the assessment criteria. To this end, we have implemented a pipeline based on advanced frameworks such as ArchUnit and SonarQube. It was successfully tested on a class of students engaged in the Object Oriented Programming course, demonstrating its usefulness as a resource for educators and providing some concrete evidence of quality problems in student projects.

Software testing is an indispensable component of software development, yet it often receives insufficient attention. The lack of a robust testing culture within computer science and informatics curricula contributes to a shortage of testing expertise in the software industry. Addressing this problem at its root -education- is paramount. In this paper, we conduct a comprehensive mapping review of software testing courses, elucidating their core attributes and shedding light on prevalent subjects and instructional methodologies. We mapped 117 courses offered by Computer Science (and related) degrees in 49 academic institutions from four Western European countries, namely Belgium, Italy, Portugal and Spain. The testing subjects were mapped against the conceptual framework provided by the ISO/IEC/IEEE 29119 standard on software testing. Among the results, the study showed that dedicated software testing courses are offered by only 39% of the analysed universities, whereas the basics of software testing are taught in at least one course at every university. The analysis of the software testing topics highlights the gaps that need to be filled in order to better align the current academic offerings with the real industry needs.

GUI-based test-cases generated by Capture and Replay tools suffer from the well-known fragility problem: they may break even if small layout changes are operated in a Web application, without modifying the app functionality. An approach based on the automatic injection of HTML tag attributes named hooks in the source code of Web templates has been recently proposed to solve this problem. Such hooks allow the unique identification of GUI items to be located during test case execution. This technique showed its effectiveness in a preliminary validation study, where it allowed to significantly reduce the number of test case locator breakages in regression testing of student-made Web applications. This paper presents a further validation study where we compared the robustness of hook-based test cases against state-of-the-art and state-of-the-practice techniques for locating GUI objects. We proposed a three dimensional model for classifying different types of layout changes and used it to define a benchmark of realistic changes. Thanks to the model, we systematically compared the robustness of test cases generated by different techniques with respect to specific types of changes and studied the relationship between fragility issues and types of changes in different test case generation techniques.

Abstract Graphical user interface (GUI) testing plays a pivotal role in ensuring the quality and functionality of mobile apps. In this context, exploratory testing (ET), a distinctive methodology in which individual testers pursue a creative, and experience‐based approach to test design, is often used as an alternative or in addition to traditional scripted testing. Managing the exploratory testing process is a challenging task that can easily result either in wasteful spending or in inadequate software quality, due to the relative unpredictability of exploratory testing activities, which depend on the skills and abilities of individual testers. A number of works have investigated the diversity of testers' performance when using ET strategies, often in a crowdtesting setting. These works, however, investigated ET effectiveness in detecting bugs, and not in scenarios in which the goal is to generate a re‐executable test suite, as well. Moreover, less work has been conducted on evaluating the impact of adopting different exploratory testing strategies. As a first step toward filling this gap in the literature, in this work, we conduct an empirical evaluation involving four open‐source Android apps and 20 masters students that we believe can be representative of practitioners partaking in exploratory testing activities. The students were asked to generate test suites for the apps using a capture and replay tool and different exploratory testing strategies. We then compare the effectiveness, in terms of aggregate code coverage that different‐sized groups of students using different exploratory testing strategies may achieve. Results provide deeper insights into code coverage dynamics to project managers interested in using exploratory approaches to test simple Android apps, on which they can make more informed decisions.

Capture and Replay techniques provide a well-known solution for End-To-End (E2E) testing of Web applications. They allow a tester to generate test scripts without requiring advanced programming skills. For this reason, they are very popular in acceptance and regression testing activities. These techniques are affected by the issue of fragility of the produced test cases, which may break even if small changes are operated in the user interface, without modifications of the app functionality. To overcome this issue, several approaches for either generating robust test cases or automatically repairing broken test cases have been proposed. In this paper we propose an alternative solution that aims at improving the testability of Web applications for generating robust test cases. This solution applies to Web applications developed with template-based technologies. It is based on the template source code automatic injection of additional hook attributes and on the proposal of a new type of locators based on such hooks. These locators aid the unique retrieval of the user interface items involved in test cases. We validated our technique in the context of a continuous integration and delivery processes of template-based web applications that was developed from scratch. The study showed that the use of hook-based locators can improve the robustness of test cases generated by a Capture & Replay testing tool, introducing relevant savings in the regression test case repairing activity.

Summary Exploratory testing and fully automated testing tools represent two viable and cheap alternatives to traditional test‐case‐based approaches for graphical user interface (GUI) testing of Android apps. The former can be executed by capture and replay tools that directly translate execution scenarios registered by testers in test cases, without requiring preliminary test‐case design and advanced programming/testing skills. The latter tools are able to test Android GUIs without tester intervention. Even if these two strategies are widely employed, to the best of our knowledge, no empirical investigation has been performed to compare their performance and obtain useful insights for a project manager to establish an effective testing strategy. In this paper, we present two experiments we carried out to compare the effectiveness of exploratory testing approaches using a capture and replay tool (Robotium Recorder) against three freely available automatic testing tools (AndroidRipper, Sapienz, and Google Robo). The first experiment involved 20 computer engineering students who were asked to record testing executions, under strict temporal limits and no access to the source code. Results were slightly better than those of fully automated tools, but not in a conclusive way. In the second experiment, the same students were asked to improve the achieved testing coverage by exploiting the source code and the coverage obtained in the previous tests, without strict temporal constraints. The results of this second experiment showed that students outperformed the automated tools especially for long/complex execution scenarios. The obtained findings provide useful indications for deciding testing strategies that combine manual exploratory testing and automated testing.

Random testing is a software testing technique through which programs are tested by generating and executing random inputs. Because of its unstructured nature, it is difficult to determine when to stop a random testing process. Faults may be missed if the process is stopped prematurely, and resources may be wasted if the process is run too long. In this article, we propose two promising termination criteria, “All Equivalent” (AEQ) and “All Included in One” (AIO), applicable to random testing. These criteria stop random testing once the process has reached a code-coverage-based saturation point after which additional testing effort is unlikely to provide additional effectiveness. We model and implement them in the context of a general random testing process composed of independent random testing sessions. Thirty-six experiments involving GUI testing and unit testing of Java applications have demonstrated that the AEQ criteria is generally able to stop the process when a code coverage equal or very near to the saturation level is reached, while AIO is able to stop the process earlier in cases it reaches the saturation level of coverage. In addition, the performance of the two criteria has been compared against other termination criteria adopted in the literature.

We carried out a family of controlled experiments to investigate whether the use of abbreviated identifier names, with respect to full-word identifier names, affects fault fixing in C and Java source code. This family consists of an original (or baseline) controlled experiment and three replications. We involved 100 participants with different backgrounds and experiences in total. Overall results suggested that there is no difference in terms of effort, effectiveness, and efficiency to fix faults, when source code contains either only abbreviated or only full-word identifier names. We also conducted a qualitative study to understand the values, beliefs, and assumptions that inform and shape fault fixing when identifier names are either abbreviated or full-word. We involved in this qualitative study six professional developers with 1--3 years of work experience. A number of insights emerged from this qualitative study and can be considered a useful complement to the quantitative results from our family of experiments. One of the most interesting insights is that developers, when working on source code with abbreviated identifier names, adopt a more methodical approach to identify and fix faults by extending their focus point and only in a few cases do they expand abbreviated identifiers.

Spreadsheet applications are widely adopted by millions of end users from several application domains and provide strategic support to many business, scientific, industrial, and organizational processes. These applications are usually developed by rapid application development processes, exploiting host scripting languages allowing the basic spreadsheets to provide complex functionality, business rules, and user interfaces. Several factors complicate the comprehension of these applications because they are usually developed and maintained by end users without specific software engineering skills, grow over time, are not adequately documented, and do not present explicit separation between data, business logic, and user interface layers. This paper presents a reverse engineering tool intended to support the comprehension of Excel spreadsheet applications developed using the Visual Basic for Application programming language. The tool has been implemented as an add-in that extends the Excel working environment by providing analysis and visualization features. It is able to extract information about the elements composing the analyzed Excel spreadsheet application, the functionality it exposes through its user interface, and the dependencies among its cells. This information is provided by means of interactive views. The validity of the tool has been assessed by a qualitative case study performed with professional end users from an automotive industrial domain. Copyright © 2016 John Wiley & Sons, Ltd.

Nevertheless spreadsheets were originally designed for computing purposes and for commercial applications, they are often used in industry to implement Information Systems, thanks to the functionalities offered by integrated scripting languages and ad-hoc frameworks (e.g., Visual Basic for Applications). This technological solution allows the adoption of Rapid Application Development processes for the quickly development of Spreadsheets-based Information Systems, but the resulting systems are quite difficult to be maintained and very difficult to be migrated to other architectures such as Database-oriented Informative Systems or Web applications. In this paper we present an approach for reverse engineering the data model from an Excel spreadsheet-based system in the context of a process of migration to a Web based application based on a MVC architecture. The proposed approach was successfully applied in a real context of a company operating in the automotive industry. The main contribution of this paper is represented by the Data Model Reverse Engineering activity that is the basis of the Migration process.

Context: abbreviated and full-word identifier names in dealing with faults in source code. Goal: investigating whether the use of abbreviated identifier names affects the ability of novice professional software developers in identifying and fixing faults in Java code. Method: external replication. Results: the results of the original experiment (conducted on C code) were confirmed. Conclusions: the difference in using abbreviated and full-word identifiers is not statistically significant with respect to the time to complete a task and the number of faults identified and fixed.

This paper presents a toolset for GUI testing of Android applications. The toolset is centered on a GUI ripper that systematically explores the GUI structure of an application under test with the aim of firing sequences of user events and exposing failures of the application. The toolset supports the execution of a testing procedure that automatically performs crash testing of subject applications and provides test results made of several artifacts. The paper illustrates some examples of using the toolset for testing real Android applications.

We present AndroidRipper, an automated technique that tests Android apps via their Graphical User Interface (GUI). AndroidRipper is based on a user-interface driven ripper that automatically explores the app’s GUI with the aim of exercising the application in a structured manner. We evaluate AndroidRipper on an open-source Android app. Our results show that our GUI-based test cases are able to detect severe, previously unknown, faults in the underlying code, and the structured exploration outperforms a random approach.

The exchange of information between heterogeneous and distributed health information systems preserving the semantics is an important open issue for the health care sector. In this paper, we propose an approach that, exploiting the Semantic Web technologies, has the objective of allowing semantic interoperability among software agents for e-Health applications that preserves, not only the semantic of transmitted messages, but also the subjectivity of agent's world vision in the communication The proposed approach exploits the Semantic Triangle Model to differentiate the roles of referents (real world objects) and concepts (mental image or impression about a real object) in the communication process and ensures an effective semantic interoperability without the need of a unique shared conceptualization.

In this paper we propose a novel communication approach and a possible implementation of it that, exploiting the Semantic Web technologies, allows semantic interoperability among software agents in the Web, preserving not only the semantics but also the subjectivity of the agent's world vision in the communication. Such an approach takes advantage of a particular communication model called Semantic Triangle in which communication agents share the referents (real world objects) and not the concepts (mental image or impression of a real object from the sender agents point of view), thus ensuring an effective semantic interoperability in the information exchange process. The proposed approach has been submitted to an experiment that involved an instantiation of the communication process based on semantic machines and showed the approach feasibility for the semantic information exchange and its effectiveness.

Thanks to Rich Internet Applications (RIAs) with their enhanced interactivity, responsiveness and dynamicity, the user experience in the Web 2.0 is becoming more and more appealing and user-friendly. At the same time, the dynamic nature of RIAs, and the heterogeneous technologies, frameworks, communication models used for implementing them negatively affect their analyzability and understandability, so that specific software techniques and tools are needed for supporting their comprehension. This paper presents DynaRIA, a tool for the comprehension of RIAs implemented in Ajax that is based on dynamic analysis and provides functionalities for recording and analyzing user sessions from several perspectives, and producing various types of abstractions and visualizations about the run-time behaviour of the application.

While the rapid and growing diffusion of rich Internet applications (RIAs) with their enhanced interactive, responsive and dynamic behaviour is sharpening the distance between Web applications and desktop applications, at the same time, the maintenance community is experiencing the need for effective analysis approaches for understanding and modelling this behaviour adequately. This paper presents a reverse engineering technique based on dynamic analysis and supported by a tool that reconstructs a model of the RIA behaviour based on finite state machines. The technique is based on the analysis of the RIA user interface evolution shown in user sessions, and exploits user interface equivalence criteria for abstracting relevant states and state transitions to be included in the model. For assessing the effectiveness and the cost of this technique, an experiment involving four distinct RIAs implemented with AJAX technique was carried out.

In this paper, we propose an architecture that, exploiting the Semantic Web technologies, has the objective of allowing semantic interoperability among software agents in the Web. Such an architecture takes advantage by the Semantic Triangle model in which communication agents share the referents (real world objects) and not the references (mental image of a real object of the sender agent), thus ensuring an effective semantic interoperability in the information exchange process. We have carried a case study in order to assess the appropriateness and the feasibility of the process for the semantic information exchange by realizing and testing an instantiation of the related architecture.

Analysis and classification of Web application user interfaces is a relevant problem in Web maintenance processes. This paper presents an approach for the reliable classification of HTML pages of a dynamic Web application. The approach is based on the assumption that groups of semantically equivalent built pages are characterized by the same key features which can be used for discriminating the pages. These features are obtained by an iterative process that exploits formal concept analysis for finding features that are specific for each class of pages. The process is supported by a toolkit that allows an effective definition of the discriminating features. The approach has been preliminarily validated with an experiment that produced encouraging results.

In dynamic cooperative architectures that are based on services (SOA), customers are not only interested in service functionalities, but also in their quality, such as performance, cost, reliability, security and so on. In this scenario, models, techniques and tools supporting the selection of the best service are needed. In this paper, we propose an evaluation framework that includes a flexible quality meta-model for formalising customer and provider views of quality, and a decisional model defining a systematic approach for comparing offered and requested quality of services. We also illustrate the applicability of the framework in a Web service (WS) scenario.

The heterogeneous and dynamic nature of components making up a Web application, the lack of effective programming mechanisms for implementing basic software engineering principles in it, and undisciplined development processes induced by the high pressure of a very short time-to-market, make Web application maintenance a challenging problem. A relevant issue consists of reusing the methodological and technological experience in the sector of traditional software maintenance, and exploring the opportunity of using reverse engineering to support effective Web application maintenance. This article presents an approach for reverse engineering Web applications. The approach include the definition of reverse engineering methods and supporting software tools, that help to understand existing undocumented Web applications to be maintained or evolved, through the reconstruction of UML diagrams. Some validation experiments have been carried out and they showed the usefulness of the proposed approach and highlighted possible areas for improvement of its effectiveness.

An approach providing automatic support in the assignment of concepts to documents recovered by reverse engineering Web applications is presented. Web pages composing Web applications usually include relevant textual information from the domain of the application, while different editing formats are used for emphasising to the end users the various concepts provided by the pages. The proposed concept assignment approach exploits both the textual information contained in the Web pages, and the editing formal used to display it in order to identify automatically a set of candidate concepts describing a Web page or a set of pages. These concepts can be used by maintainers involved in the task of assigning a meaning to software artefacts recovered by reverse engineering the Web applications. Validation experiments carried out with Web applications selected from the real world showed the validity of the proposed approach. The experimental results are presented in the paper.

Abstract The rapid, progressive diffusion of Web applications in several productive contexts of our modern society is laying the foundations of a renewed scenario of software development, where one of the emerging problems is that of defining and validating cost‐effective approaches for maintaining and evolving these software systems. Due to several factors, the solution to this problem is not straightforward. The heterogeneous and dynamic nature of components making up a Web application, the lack of effective programming mechanisms for implementing basic software engineering principles in it, and undisciplined development processes induced by the high pressure of a very short time‐to‐market, make Web application maintenance a challenging problem. A relevant issue consists of reusing the methodological and technological experience in the sector of traditional software maintenance, and exploring the opportunity of using reverse engineering to support effective Web application maintenance. This paper presents an approach for defining reverse engineering processes involving Web applications. The approach has been used to implement a process, including reverse engineering methods and a supporting software tool, that helps to understand existing undocumented Web applications to be maintained or evolved, through the reconstruction of UML diagrams. The proposed reverse engineering process has been submitted to a validation experiment, the results of which showed the usability of the process for reverse engineering Web applications with different characteristics, and highlighted possible areas for improvement of its effectiveness. The experiment and the lessons learned from it are presented in the paper. Copyright © 2004 John Wiley & Sons, Ltd.

The growing market request for Web applications is forcing software industries to produce applications under the pressure of a short time-to-market and a strong competition, with the consequence that low quality and poor documented software is often produced. Maintaining, evolving or comprehending these applications are not straightforward tasks, and reverse engineering processes should be defined and validated to support them. In this paper a reverse engineering approach for reconstructing an object-oriented conceptual model of the application domain of a Web application is presented. The proposed approach defines a process that reconstructs the model in three steps. In each step, heuristic criteria exploiting source code analysis are used for the identification of objects and their relationships. Tools for implementing this method have been produced, and experiments for validating it have been carried out with the support of case studies. Experimental results showed the feasibility and the effectiveness of the proposed approach.