Quantum computing has become an active research field in recent years, as its applications in fields such as cryptography, optimization, and materials science are promising. Along with these developments, challenges and opportunities exist in the field of Quantum Software Engineering, as the development of frameworks and higher-level abstractions has attracted practitioners from diverse backgrounds. Unlike initial quantum frameworks based on the circuit model, recent frameworks and libraries leverage higher-level abstractions for creating quantum programs. This paper presents an empirical study of 985 Jupyter Notebooks from 80 open-source projects to investigate how quantum patterns are applied in practice. Our work involved two main stages. First, we built a knowledge base from three quantum computing frameworks (Qiskit, PennyLane, and Classiq). This process led us to identify and document nine new patterns that refine and extend the existing quantum computing pattern catalog. Second, we developed a reusable semantic search tool to automatically detect these patterns across our large-scale dataset, providing a practitioner-focused analysis. Our results show that developers use patterns in three levels: from foundational circuit utilities, to common algorithmic primitives (e.g., Amplitude Amplification), up to domain-specific applications for finance and optimization. This indicates a maturing field where developers are increasingly using high-level building blocks to solve real-world problems.

Although classical computing has excelled in a wide range of applications, there remain problems that push the limits of its capabilities, especially in fields like cryptography, optimization, and materials science. Quantum computing introduces a new computational paradigm, based on principles of superposition and entanglement to explore solutions beyond the capabilities of classical computation. With the increasing interest in the field, there are challenges and opportunities for academics and practitioners in terms of software engineering practices, particularly in testing quantum programs. This paper presents an empirical study of testing patterns in quantum algorithms. We analyzed all the tests handling quantum aspects of the implementations in the Qiskit Algorithms library and identified seven distinct patterns that make use of (1) fixed seeds for algorithms based on random elements; (2) deterministic oracles; (3) precise and approximate assertions; (4) Data-Driven Testing (DDT); (5) functional testing; (6) testing for intermediate parts of the algorithms being tested; and (7) equivalence checking for quantum circuits. Our results show a prevalence of classical testing techniques to test the quantum-related elements of the library, while recent advances from the research community have yet to achieve wide adoption among practitioners.

Quantum computing has existed in the theoretical realm for several decades. Recently, quantum computing has re-emerged as a promising technology to solve problems that a classical computer could take hundreds of years to solve. However, there are challenges and opportunities for academics and practitioners regarding software engineering practices for testing and debugging quantum programs. This article presents a roadmap for addressing these challenges, pointing out the existing gaps in the literature and suggesting research directions. We discuss the limitations caused by noise, the no-cloning theorem, the lack of a standard architecture for quantum computers, among others. Regarding testing, we highlight gaps and opportunities related to transpilation, mutation analysis, input states with hybrid interfaces, program analysis, and coverage. For debugging, we present the current strategies, including classical techniques applied to quantum programs, quantum-specific assertions, and quantum-related bug patterns. We introduce a conceptual model to illustrate concepts regarding the testing and debugging of quantum programs and the relationship between them. Those concepts are used to identify and discuss research challenges to cope with quantum programs through 2030, focusing on the interfaces between classical and quantum computing and on creating testing and debugging techniques that take advantage of the unique quantum computing characteristics.

Summary Developers spend significant time locating and fixing bugs, which is often performed manually. Although spectrum‐based fault localization (SFL) techniques aim at helping developers to locate faults, they are not yet used in practice. Recent studies have investigated how developers use SFL, presenting different conclusions about their effectiveness and usefulness. We carried out a user study to further enhance the understanding of SFL. We assessed whether SFL improves the developers' performance and to what extent SFL leads developers to inspect faulty code excerpts. We also investigated the intention of the developers to use SFL and how they interact with SFL. Twenty‐six participants performed debugging tasks using real programs, with and without using the Jaguar SFL tool. Using SFL, more developers located and fixed the bugs. SFL also led more developers to inspect the faulty code and locate the faulty method. However, they did not spend less time locating the faults. SFL was well‐accepted by the participants, who showed intention to use it in their daily activities. Our results indicate that SFL is useful even when the fault is not ranked among the first positions, leading developers to reach faulty code regions and find the bugs.

Summary Data flow testing creates test requirements as definition‐use (DU) associations, where adefinitionis a program location that assigns a value to a variable and auseis a location where that value is accessed. Data flow testing is expensive, largely because of the number of test requirements. Luckily, many DU‐associations are redundant in the sense that if one test requirement (e.g. node, edge and DU‐association) is covered, other DU‐associations are guaranteed to also be covered. This relationship is calledsubsumption. Thus, testers can save resources by only covering DU‐associations that are not subsumed by other testing requirements. Although this has the potential to significantly decrease the cost of data flow testing, there are roadblocks to its application. Finding data flow subsumptions correctly and efficiently has been an elusive goal; the savings provided by data flow subsumptions and the cost to find them need to be assessed; and the fault detection ability of a reduced set of DU‐associations and the advantages of data flow testing over node and edge coverage need to be verified. This paper presents novel solutions to these problems. We present algorithms that correctly find data flow subsumptions and are asymptotically less costly than previous algorithms. We present empirical data that show that data flow subsumption is effective at reducing the number of DU‐associations to be tested and can be found at scale. Furthermore, we found that using reduced DU‐associations decreased the fault detection ability by less than 2%, and data flow testing adds testing value beyond node and edge coverage.

Data flow testing helps testers design effective tests by requiring the tests to execute sequences of statements from definitions of variables to one or more subsequent uses. These def-use associations are derived from graphs that model software behavior. A "flow graph" that only includes paths that cover defuse associations, and not other control flows, has been defined elsewhere. Although these flow graphs have several advantages over previous graphs, as computed, they omit some valid paths, which are needed to use the graphs to discover subsumption relationships and generate test data. These omissions lead to errors in the results. This paper extends previous solutions by presenting a graph that represents all paths that cover def-use associations. The paper presents empirical data showing that this graph can be generated at reasonable cost and efficiently applied for data flow subsumption discovery.

Data flow testing creates test requirements as definition-use (DU) associations, where a definition is a program location that assigns a value to a variable and a use is a location where that value is accessed. Data flow testing is expensive, largely because of the number of test requirements. Luckily, many DU-associations are redundant in the sense that if one test requirement (e.g., node, edge, DU-association) is covered, other DU-associations are guaranteed to also be covered. This relationship is called subsumption. Thus, testers can save resources by only covering DU-associations that are not subsumed by other testing requirements. Although this has the potential to significantly decrease the cost of data flow testing, finding subsumption among DU-associations is quite difficult. Previous solutions are costly and contain subtle flaws that sometimes lead to incorrect results. We model the data flow testing subsumption as a data flow analysis framework, allowing us to use efficient algorithms that quickly discover data flow subsumption relationships. Experimental data suggest that the framework and algorithm can reduce the cost of data flow testing and will work at scale.

Abstract As software systems have increased in complexity, manual testing has become harder or even infeasible. In addition, each test phase and application domain may have its idiosyncrasies in relation to testing automation. Techniques and tools to automate test oracles in domains such as graphical user interfaces are available; nevertheless, they are scarce in the virtual reality (VR) realm. We present an approach to automate software testing in VR-based systems with haptic interfaces—interfaces that allow bidirectional communication during human–computer interaction, capturing movements and providing touch feedback. It deals with the complexity and characteristics of haptic interfaces to apply the record and playback technique. Our approach also provides inference rules to identify possible faulty modules of the system under testing. A case study was performed with three systems: a system with primitive virtual objects, a dental anesthesia simulator and a game. Faulty versions of the systems were created by seeding faults manually and by using mutation operators. The results showed that 100% of the manually seeded faults and 93% of mutants were detected. Moreover, the inference rules helped identify the faulty modules of the systems, suggesting that the approach improves the test activity in VR-based systems with haptic interfaces.

Background: Debugging is a key task during the software development cycle. Spectrum-based Fault Localization (SFL) is a promising technique to improve and automate debugging. SFL techniques use control-flow spectra to pinpoint the most suspicious program elements. However, data-flow spectra provide more detailed information about the program execution, which may be useful for fault localization. Aims: We evaluate the effectiveness and efficiency of ten SFL ranking metrics using data-flow spectra. Method: We compare the performance of data- and control-flow spectra for SFL using 163 faults from 5 real-world open source programs, which contain from 468 to 4130 test cases. The data- and control-flow spectra types used in our evaluation are definition-use associations (DUAs) and lines, respectively. Results: Using data-flow spectra, up to 50% more faults are ranked in the top-15 positions compared to control-flow spectra. Also, most SFL ranking metrics present better effectiveness using data-flow to inspect up to the top-40 positions. The execution cost of data-flow spectra is higher than control-flow, taking from 22 seconds to less than 9 minutes. Data-flow has an average overhead of 353% for all programs, while the average overhead for control-flow is of 102%. Conclusions: The results suggest that SFL techniques can benefit from using data-flow spectra to classify faults in better positions, which may lead developers to inspect less code to find bugs. The execution cost to gather data-flow is higher compared to control-flow, but it is not prohibitive. Moreover, data-flow spectra also provide information about suspicious variables for fault localization, which may improve the developers' performance using SFL.

Although many studies investigate the automatic adaptation in serious games with the goal to improve the users motivation, the majority of Affective Computing approaches requires a high development cost and usually does not consider the intervention of health professionals in adapting the game. This paper describes an approach to enable affective adaptation in serious games for motor rehabilitation with physiotherapists aid. Our approach consists of the definition and implementation of a framework. Its architecture reduces the development cost of a game with affective adaptation whilist enabling physiotherapists to configure adaptations in it according to patients profile. The results of an experiment with physiotherapists show that the system presents a high level of acceptance.

Data-flow (DF) testing was introduced more than thirty years ago aiming at extensively evaluating a program structure. It requires tests that traverse a path in which the definition of a variable and its subsequent use, i.e., a definition-use association (dua), is exercised. While control-flow testing tools have being able to tackle big systems-large and long running programs, DF testing tools have failed to do so. This situation is in part due to the costs associated with tracking duas at run-time. Recently, an algorithm, called Bitwise Algorithm (BA), which uses bit vectors and bitwise operations for tracking intra-procedural duas at run-time, was proposed. This paper presents the implementation of BA for programs compiled into bytecodes. Previous approaches were able to deal with small to medium size programs with high penalties in terms of execution and memory. Our experimental results show that by using BA we are able to tackle large systems with more than 200 KLOCs and 300K required duas. Furthermore, for several programs the execution penalty was comparable with that imposed by a popular control-flow testing tool.

Fault localization is a costly task in the debugging process. Several techniques to automate fault localization have been proposed aiming at reducing effort and time spent. Some techniques use heuristics based on code coverage data. The goal is to indicate program code excerpts more likely to contain faults. The coverage data mostly used in automated debugging is based on white-box unit testing (e.g., statements, basic blocks, predicates). This paper presents a technique which uses integration coverage data to guide the fault localization process. By ranking most suspicious pairs of method invocations, roadmaps-sorted lists of methods to be investigated-are created. At each method, unit coverage (e.g., basic blocks) is used to locate the fault site. Fifty-five bugs of four programs containing 2K to 80K lines of code (LOC) were analyzed. The results indicate that, by using the roadmaps, the effectiveness of the fault localization process is improved: 78% of all the faults are reached within a fixed amount of basic blocks; 40% more than an approach based on the Tarantula technique. Furthermore, fewer blocks have to be investigated until reaching the fault.

Web services are an emerging Service-Oriented Architecture technology to integrate applications using open standards based on XML. Software Engineering tools integration is a promising area since companies adopt different software processes and need different tools on each activity. Software engineers could take advantage of software engineering tools available as web services and create their own workflow for integrating the required tools. In this paper, we propose the development of testing tools designed as web services and discuss the pros and cons of this idea. We developed a web service for structural testing of Java programs called JaBUTiService, which is based on the stand-alone tool JaBUTi. We also present an usage example of this service with the support of a desktop front-end and pre prepared scripts. A set of 62 classes of the library Apache-Commons-BeanUtils was used for this test and the results are discussed.

Abstract Testing and debugging activities consume a significant amount of the software development and maintenance budget. To reduce this cost, the use of testing information for debugging purposes has been advocated. In general, heuristics are used to select structural testing requirements (nodes, branches and definition–use associations) more closely related to the manifestation of a failure, which are then mapped into a piece of code. The intuition is that the selected piece of code is likely to contain the fault. However, this approach has its drawbacks. Heuristics that select a manageable piece of code are less likely to hit the fault and the piece of code itself does not provide enough guidance for program understanding—a major factor in program debugging. These problems occur because this approach relies only on static information—a fragment of code. We introduce a strategy for fault localization that addresses these problems. The strategy—called the debugging strategy based on the requirements of testing (DRT)—is based on the investigation of indications (or hints) provided at run‐time by data‐flow testing requirements (definition–use associations). Our claim is that the selected definition–use associations may fail to hit the fault site, but still provide information useful for fault localization. The strategy's novelty and attractiveness are threefold: (i) the focus on dynamic information related to testing data; (ii) implementation in state‐of‐the‐practice symbolic debuggers with a low overhead; and (iii) the use of algorithms which consume constant memory and are linear on the number of branches in the program. A case study shows that our claim is valid (for the subject program) and a prototype tool implements the strategy. Copyright © 2004 John Wiley & Sons, Ltd.