Context: Deep learning, and in particular graph-based models, has advanced software vulnerability detection by effectively capturing structural code features. Nonetheless, most existing approaches treat source code as independent components, overlooking inter-function relationships and thereby missing potential vulnerability propagation across function boundaries. Objective: To address this limitation, we propose SCVdet , a Graph Attention Network (GAT) based method that integrates inter-function dependencies into the vulnerability detection process. Method: These dependencies are approximated by using a clustering technique over learned code functions’ embeddings. This enables scalable and dynamic modeling of function relationships without incurring the high computational cost of the static code analysis required to recover the full set of real and complete inter-function dependencies. We enhance representation learning by complementing the analysis of the functions’ code, local code semantics, with the inter-function dependency analysis, global project-level analysis. We then adopt two strategies: concatenation and attention, to fuse such local and global types of information. Results: Extensive experiments on multi-language datasets ( FFmpeg+QEMU , ProjectKB , Big-Vul , and CVEFixes ) demonstrate that SCVdet consistently outperforms both sequence-based and graph-based baselines, achieving up to a 16% improvement in F1-score at the function level and 7% at the statement level. Conclusion: The results indicate that SCVdet improves vulnerability detection by combining both local and global information. This approach increases detection accuracy and outperforms the capabilities of state-of-the-art tools while maintaining scalability.

Deep learning, in particular, graph-based models, has advanced software vulnerability detection by capturing structural code features. However, existing approaches often rely solely on source code and focus mainly on C/C++ at the function level, limiting their ability to detect fine-grained vulnerabilities in diverse languages like Java and Python. To overcome these limitations, we propose VVulDet , an enhanced Graph Neural Network model. VVulDet enriches code representations with complex graph representations through random walks feature update and incorporates domain knowledge from CVE and CWE descriptions, along with expert-provided reference code fragments. This integration enhances the model’s understanding of vulnerabilities beyond pure code structure. We evaluate VVulDet on four datasets covering Java, Python, and C/C++, demonstrating consistent improvements at both statement and function level detection. Notably, VVulDet achieves, on average, the highest overall performance across all datasets, demonstrating F1-score improvements of up to 8.6% and 9.6% at the statement and function levels on ProjectKB, 6.8% and 15.5% on MegaVul, 1.4% and 4.5% on CVEFixes, and 2.4% and 23.7% on BigVul, respectively, compared to the model version that does not incorporate domain knowledge. These results confirm that integrating domain knowledge into graph-based models significantly boosts vulnerability detection performance across multiple programming languages and granularity levels.

In recent years, security testing and vulnerability detection in source code have experienced a significant transformation with the adoption of data-driven techniques. This shift has reduced reliance on manual analysis, addressed the high false-positive rates of static analyzers, and accelerated the early detection of software bugs, ultimately mitigating the risk of cyberattacks. Among these advancements, graph-based approaches have shown promising results by capturing structural and contextual patterns within source code. However, such methods often rely solely on the code under analysis, limiting their ability to comprehensively learn vulnerable patterns.This study explores the integration of domain-specific knowledge into a Graph Neural Network (GNN)–based model to enhance its understanding and detection of vulnerabilities. By incorporating resources such as Common Vulnerability Exposure (CVE) descriptions, Common Weakness Enumeration (CWE) definitions, and sample functions provided by security experts at the MITRE Corporation, we aim to enrich the model’s knowledge base. Our approach demonstrates significant improvements on a Java vulnerability dataset across all considerable metrics. This finding underscores the value of domain-specific augmentation in advancing vulnerability detection capabilities.

Security vulnerabilities are weaknesses of software due for instance to design flaws or implementation bugs that can be exploited and lead to potentially devastating security breaches. Traditionally, static code analysis is recognized as effective in the detection of software security vulnerabilities but at the expense of a high human effort required for checking a large number of produced false positive cases. Deep-learning methods have been recently proposed to overcome such a limitation of static code analysis and detect the vulnerable code by using vulnerability-related patterns learned from large source code datasets. However, the use of these methods for localizing the causes of the vulnerability in the source code, i.e., localize the statements that contain the bugs, has not been extensively explored.

Peopleware, which includes anything related to the role of people in Software Development (SD), has been arousing an increasing interest from both the software industry and research community. This interest is due to the current economic system that demands high-quality software products with a short time to market, staying on the budget. This exposes software developers to the risk of experiencing stress, burnout, and reduced motivation, leading, in turn, to reduced job performance, low-quality SD-related artifacts, and increased turnover. Mindfulness represents a promising intervention that might let developers do their best at work, limiting or even preventing the previously mentioned negative outcomes. This paper presents MOOD (Mindfulness fOr sOftware Developers), a research project whose overarching goal is to customize a well-known and validated group-based intervention program, Mindfulness-Based Stress Reduction (MBSR), in the context of SD-related tasks and assess whether it helps developers to improve their well-being and performance, as well as the quality of the SD-related artifacts they produce.

Abstract A new advancement in test automation is the use of natural language processing (NLP) to generate test cases (or test scripts) from natural language text. NLP is innovative in this context and promises of reducing test cases creation time and simplifying understanding for “non‐developer” software testers as well. Recently, many vendors have launched on the market many proposals of NLP‐based tools and testing frameworks but their superiority has never been empirically validated. This paper investigates the adoption of NLP‐based test automation in the web context with a series of case studies conducted to compare the costs of the NLP testing approach—measured in terms of test cases development and test cases evolution—with respect to more consolidated approaches, that is, programmable (or script‐based) testing and capture&replay testing. The results of our study show that NLP‐based test automation appears to be competitive for small‐ to medium‐sized test suites such as those considered in our empirical study. It minimizes the total cumulative cost (development and evolution) and does not require software testers with programming skills.

A Connected Vehicle System (CVS) is a cyberphysical system of highly-equipped infrastructure-connected vehicles interconnected with road-side units and cloud-based services to offer safer driving. Despite the ever increasing relevant research, the testing of CVS functionalities and communication features remains problematic; computer-based simulation requires detailed system models while field-testing is expensive, focusing on few components and may raise safety concerns. In this paper, we present a full CVS prototype testbed enabled to realize a broad set of timely Vehicle-to-Infrastructure (V2I) use-cases and serve as the basis for automotive cybersecurity testing. The testbed designed and built in the context of the H2020 SAFERtec project, is described in terms of its hardware requirements, software design and implementation. The conducted testing activities on its capability to realize the considered V2I use-cases and support cybersecurity testing is explained. More importantly, the paper details take-home lessons derived from the CVS implementation experiences aiming to assist future development of automotive prototype testbeds.

Test suites tend to become large and complex after software evolution iterations, thus increasing effort and cost to execute regression testing. In this context, test suite reduction approaches could be applied to identify subsets of original test suites that preserve the capability of satisfying testing requirements and revealing faults. In this paper, we propose Multi-Objective test suites REduction (named MORE+): a three-dimension approach for test suite reduction. The first dimension is the structural one and concerns the information on how test cases in a suite exercise the under-test application. The second dimension is functional and concerns how test cases exercise business application requirements. The third dimension is the cost and concerns the time to execute test cases. We define MORE+ as a multi-objective approach that reduces test suites so maximizing their capability in revealing faults according to the three considered dimensions. We have compared MORE+ with seven baseline approaches on 20 Java applications. Results showed, in particular, the effectiveness of MORE+ in reducing test suites with respect to these baselines, i.e., significantly more faults are revealed with test suites reduced by applying MORE+.

Regression testing is an important activity that can be expensive (e.g., for large test suites). Test suite reduction approaches speed up regression testing by removing redundant test cases. These approaches can be classified as adequate or inadequate. Adequate approaches reduce test suites so that they completely preserve the test requirements (e.g., code coverage) of the original test suites. Inadequate approaches produce reduced test suites that only partially preserve the test requirements. An inadequate approach is appealing when it leads to a greater reduction in test suite size at the expense of a small loss in fault-detection capability. We investigate a clustering-based approach for inadequate test suite reduction and compare it with well-known adequate approaches. Our investigation is founded on a public dataset and allows an exploration of trade-offs in test suite reduction. Results help a more informed decision, using guidelines defined in this research, to balance size, coverage, and fault-detection loss of reduced test suites when using clustering.

While performing regression testing, an appropriate choice for test case ordering allows the tester to early discover faults in source code. To this end, test case prioritization techniques can be used. Several existing test case prioritization techniques leave out the execution cost of test cases and exploit a single objective function (e.g., code or requirements coverage). In this paper, we present a multi-objective test case prioritization technique that determines the ordering of test cases that maximize the number of discovered faults that are both technical and business critical. In other words, our new technique aims at both early discovering faults and reducing the execution cost of test cases. To this end, we automatically recover links among software artifacts (i.e., requirements specifications, test cases, and source code) and apply a metric-based approach to automatically identify critical and fault-prone portions of software artifacts, thus becoming able to give them more importance during test case prioritization. We experimentally evaluated our technique on 21 Java applications. The obtained results support our hypotheses on efficiency and effectiveness of our new technique and on the use of automatic artifacts analysis and weighting in test case prioritization.

Several techniques and tools have been proposed for the automatic generation of test cases. Usually, these tools are evaluated in terms of fault-revealing or coverage capability, but their impact on the manual debugging activity is not considered. The question is whether automatically generated test cases are equally effective in supporting debugging as manually written tests. We conducted a family of three experiments (five replications) with humans (in total, 55 subjects) to assess whether the features of automatically generated test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on the effectiveness and efficiency of debugging. The first two experiments compare different test case generation tools (Randoop vs. EvoSuite). The third experiment investigates the role of code identifiers in test cases (obfuscated vs. original identifiers), since a major difference between manual and automatically generated test cases is that the latter contain meaningless (obfuscated) identifiers. We show that automatically generated test cases are as useful for debugging as manual test cases. Furthermore, we find that, for less experienced developers, automatic tests are more useful on average due to their lower static and dynamic complexity.

Graphical representations are used to visualise, specify, and document software artifacts in all stages of software development process. In contrast with text, graphical representations are presented in two-dimensional form, which seems easy to process. However, few empirical studies investigated the efficiency of graphical representations vs. textual ones in modelling and presenting software requirements. Therefore, in this paper, we report the results of an eye-tracking experiment involving 28 participants to study the impact of structured textual vs. graphical representations on subjects' efficiency while performing requirement comprehension tasks. We measure subjects' efficiency in terms of the percentage of correct answers (accuracy) and of the time and effort spend to perform the tasks. We observe no statistically-significant difference in term of accuracy. However, our subjects spent more time and effort while working with the graphical representation although this extra time and effort does not affect accuracy. Our findings challenge the general assumption that graphical representations are more efficient than the textual ones at least in the case of developers not familiar with the graphical representation. Indeed, our results emphasise that training can significantly improve the efficiency of our subjects working with graphical representations. Moreover, by comparing the visual paths of our subjects, we observe that the spatial structure of the graphical representation leads our subjects to follow two different strategies (top-down vs. bottomup) and subsequently this hierarchical structure helps developers to ease the difficulty of model comprehension tasks.

Software testing is an effective, yet expensive, method to improve software quality. Test automation, a potential way to reduce testing cost, has received enormous research attention recently, but the so-called “oracle problem” (how to decide the PASS/FAIL outcome of a test execution) is still a major obstacle to such cost reduction. We have extensively investigated state-of-the-art works that contribute to address this problem, from areas such as specification mining and model inference. In this paper, we compare three types of automated oracles: Data invariants, Temporal invariants, and Finite State Automata. More specifically, we study the training cost and the false positive rate; we evaluate also their fault detection capability. Seven medium to large, industrial application subjects and real faults have been used in our empirical investigation.

SUMMARY Web applications are often used to expose business processes implemented as software systems. This paper describes a technique for recovering business processes based on a dynamic analysis of the applications behavior. The technique described here does not require any access to internal software artifacts of the application, such as source code or documentation. An initial process is inferred to by means of the analysis of execution traces, in which the execution of GUI elements such as forms and links is recorded. The recovered process is then abstracted by clustering its elements according to four different criteria: structural, page‐based, dependency‐based and semantical. A case study has been conducted with the aim of evaluating understandability and readability of the reverse engineered processes as well as the clustering techniques used in refining them. Copyright © 2010 John Wiley & Sons, Ltd.

Automatically generated test cases are usually evaluated in terms of their fault revealing or coverage capability. Beside these two aspects, test cases are also the major source of information for fault localization and fixing. The impact of automatically generated test cases on the debugging activity, compared to the use of manually written test cases, has never been studied before. In this paper we report the results obtained from two controlled experiments with human subjects performing debugging tasks using automatically generated or manually written test cases. We investigate whether the features of the former type of test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on accuracy and efficiency of debugging. The empirical study is aimed at investigating whether, despite the lack of readability in automatically generated test cases, subjects can still take advantage of them during debugging.

Test prioritization techniques can be used to determine test case ordering and early discover faults in source code. Several of these techniques exploit a single objective function, e.g., code or requirements coverage. In this tool demo paper, we present MOTCP, a software tool that implements a multi-objective test prioritization technique based on the information related to the code and requirements coverage, as well as the execution cost of each test case. To establish users' and system requirements coverage, the MOTCP uses Latent Semantic Indexing to recover traceability links among application source code and requirements specifications. The test case ordering is then obtained by applying a non-dominated sorting genetic algorithm.

Specification models recovered from existing software applications can support developers in comprehending and checking the applications during maintenance and evolution operations. Often, in fact, a huge amount of business knowledge is embedded in the application implementation while documentation is not available or not aligned with the actual software implementation. In order to (re)acquire and preserve the business knowledge, specifications recovery techniques are adopted. In this paper we present reBPMN, a tool that recovers business process models from execution traces of target applications. It recovers the process exposed by means of Web interfaces and it applies a multi-objective process reduction technique, which minimizes at the same time process complexity, non-conformances, and loss of business content. This allows us to obtain processes having high readability by decreasing their structural complexity, while preserving the completeness of the described business and domain-specific information. A case study shows the effectiveness of reBPMN in recovering readable and business-meaningful processes.

Automated web crawlers can be used to explore and exercise portions of a web application under test. However, the possibility to achieve full exploration of a web application through automated crawling is severely limited by the choice of the input values submitted with forms. Depending on the crawler's capabilities, a larger or smaller portion of web application will be automatically explored. In this paper, we introduce web crawl ability metrics to quantify properties of application pages and forms that affect crawl ability. Moreover, we show that our metrics can be used to identify the boundaries between those parts of the application that can be successfully crawled automatically and those parts that will require manual intervention or other crawl ability support. We have validated our crawl ability metrics on real web applications, for which low crawl ability was indeed associated with the existence of pages never exercised during automated crawling.

Models inferred from execution traces (logs) may admit more behaviours than those possible in the real system (over-approximation) or may exclude behaviours that can indeed occur in the real system (under-approximation). Both problems negatively affect model based testing. In fact, over-approximation results in infeasible test cases, i.e., test cases that cannot be activated by any input data. Under-approximation results in missing test cases, i.e., system behaviours that are not represented in the model are also never tested. In this paper we balance over- and under-approximation of inferred models by resorting to multi-objective optimization achieved by means of two search-based algorithms: A multi-objective Genetic Algorithm (GA) and the NSGA-II. We report the results on two open-source web applications and compare the multi-objective optimization to the state-of-the-art KLFA tool. We show that it is possible to identify regions in the Pareto front that contain models which violate fewer application constraints and have a higher bug detection ratio. The Pareto fronts generated by the multi-objective GA contain a region where models violate on average 2% of an application's constraints, compared to 2.8% for NSGA-II and 28.3% for the KLFA models. Similarly, it is possible to identify a region on the Pareto front where the multi-objective GA inferred models have an average bug detection ratio of 110 : 3 and the NSGA-II inferred models have an average bug detection ratio of 101 : 6. This compares to a bug detection ratio of 310928 : 13 for the KLFA tool.

Specifications mined from execution traces are largely used to support testing and analysis of software applications with little runtime variability. However, when models are mined from applications that evolve at runtime, the resulting models become quickly obsolete, and thus of little support for any testing and analysis activity. To cope with such systems, mined specifications must be consistently updated every time the software changes. In principle, models can be periodically mined from scratch, but in many cases this solution is too expensive or even impossible. In this paper we describe Revolution, an approach for the automatic evolution of specifications mined by applying state abstraction techniques. Revolution produces models that are continuously updated and thus remain aligned with the actual implementation. Empirical results show that Revolution can suitably address run-time evolving applications.

Model-based testing relies on the assumption that effective adequacy criteria can be defined in terms of model coverage achieved by a set of test paths. However, such test paths are only abstract test cases and input test data must be specified to make them concrete. We propose a novel approach that combines model-based and combinatorial testing in order to generate executable and effective test cases from a model. Our approach starts from a finite state model and applies model-based testing to generate test paths that represent sequences of events to be executed against the system under test. Such paths are transformed to classification trees, enriched with domain input specifications such as data types and partitions. Finally, executable test cases are generated from those trees using t-way combinatorial criteria.

Web services evolve frequently to meet new business demands and opportunities. However, service changes may affect service compositions that are currently consuming the services. Hence, audit testing (a form of regression testing in charge of checking for compatibility issues) is needed. As service compositions are often in continuous operation and the external services have limited (expensive) access when invoked for testing, audit testing has severe time and resources constraints, which make test prioritization a crucial technique (only the highest priority test cases will be executed).This paper presents a novel approach to the prioritization of audit test cases using information retrieval. This approach matches a service change description with the code portions exercised by the relevant test cases. So, test cases are prioritized based on their relevance to the service change. We evaluate the proposed approach on a system that composes services from eBay and Google.

Several projects rely on one or more Heroes who are the only ones who understand and know certain critical parts of a system. Often Heroes are very useful in the economy of a project but, their presence can increase the risk of project failure if they decide to leave the project. For this reason, tools for measuring the amount of spread of knowledge within a team (i.e. the Truck factor) and identifying possible Heroes are welcomed.

Ajax supports the development of rich-client Web applications, by providing primitives for the execution of asynchronous requests and for the dynamic update of the page structure and content. Often, Ajax Web applications consist of a single page whose elements are updated in response to callbacks activated asynchronously by the user or by a server message. These features give rise to new kinds of faults that are hardly revealed by existing Web testing approaches. In this paper, we propose a novel state-based testing approach, specifically designed to exercise Ajax Web applications. The document object model (DOM) of the page manipulated by the Ajax code is abstracted into a state model. Callback executions triggered by asynchronous messages received from the Web server are associated with state transitions. Test cases are derived from the state model based on the notion of semantically interacting events. We evaluate the approach on a case study in terms of fault revealing capability. We also measure the amount of manual interventions involved in constructing and refining the model required by this approach.

One of the most well known and used approach to dynamically analyze a Web application requires to the user to analyze code and requirements of the application to extract its scenarios and the needed inputs. This information is used to exercise the application behavior and so build its model. Hence, high knowledge and effort are required to apply that kind of analysis. Moreover, it is well recognized that a dynamic approach builds partial models since it is strictly related to the application execution. In this paper, we talk about an approach that uses code mutation to reverse engineer a Web application and build its model then used for testing. We document an experiment done to evaluate feasibility and effectiveness of that approach comparing it with other traditional ones. The results show that the approach automatically builds models containing a limited degree of inaccuracy that can be pruned during the application testing.