With the massive adoption of mobile devices, it became more mandatory for developers to provide high-quality applications. Nowadays, mobile devices are used for different purposes: entertainment, shopping, banking, and communication. Moreover, mobile devices can communicate and exchange information with various IoT devices distributed across the city. However, mobile application testing has different challenges when compared to other types of applications (i.e., desktop and client-server applications). First, we must consider mobile devices' different characteristics and limitations, such as connectivity, screen size, density, sensors, and limited battery. Second, there is a wide range of mobile devices from diverse vendors and models. Hence, there is a need to consider different device configurations to reduce compatibility issues that may occur in a high-fragmented ecosystem. In this case, several tools and services with various features and business models aim to run tests on multiple devices. In this practical experience report, we present the initial results of implementing a testing tool/service at Von Braun Labs to support the execution of tests across multiple Android devices. The stakeholders stated the need to (i) execute the tests on physical devices; and (ii) the tool/service must support tests that interact with a specialized IoT device. We start the study by comparing different tools/services to select the most suitable one for Von Braun Labs. We propose a comparison framework to help evaluate six tools/services based on their technical, usability, and customization features. Then, we present a case study with an app from Von Braun Labs to validate the selected testing environment. Finally, we discuss the lessons learned, contributions, and future directions, pinpointing the need for a testing process since the beginning of the development project and the importance of lessening the gap between academia and industry.

There is an increasing interest in systematically assessing Online Social Learning (OSL) in informal environments. However, there is no agreement on how their performance and impact can be measured. Social Network Analysis (SNA) has been used to analyze OSL; however, limitations have been pointed out such as many studies apply SNA superficially, the sample size of the majority of them is small, and many studies are not grounded in any theoretical framework. This paper presents an approach to assess large Virtual Communities of Practice (VCoP) used in OSL. Our approach aims to systematize the process of assessing social interactions in VCoP based on the value creation theoretical framework. It combines SNA measures and analysis of the discussions' structure in order to identifying behavioral patterns associated to learning. The results point out a set of quantitative measures which shows that participation and ongoing collaboration have a fundamental role for VCoP knowledge-sharing in informal settings.

This Innovate Practice Full Paper reports a Industry/Academia partnership to improve the qualification of both students and professionals. Consider the following scenario: a city located in a third world country, with one of the lowest countrywide educational performances, hosting an industrial pole based on Federal tax incentives aiming to develop a R&D cluster. The city has basic services issues, from health to urban violence, making the attraction of professionals from other locations unfeasible. As a consequence, the lack of professionals with R&D profile is now considered a relevant risk to the continuation of the city's industrial policies. Therefore, initiatives that shorten the qualification life-cycle of local professionals, while preparing senior students with real-world experience, are welcome. An example of such initiative, reported in this paper, was the software test residence, formed by professionals from the industry, and senior undergraduate and graduate students of a local university. The proposal consisted on defining a set of learning activities where students and professionals would be partners, but still being aware that they belong to different realities (with respect to timing, accountability and accreditation). The challenge can be summarized as follows: how to jointly qualify students and professionals, while respecting the peculiarities of both audiences. In order to address the issue in a short time period, representatives from a global company with R&D site in the city partnered with a local university to define a learning program inspired by the software residence approach. The university's course syllabus in Software Test was reviewed and customized to the on-the-job training subject of interest. Moreover, the approach was based on three nonnegotiable principles: Accountability, Excellence and Sustainability. The experience was successful, reaching 25 participants from four R&D organizations and the university. Results obtained include qualification of 23 participants (only two dropped off), the development of an open source educational material composed by a set of slides for testing automation classes, theory-practice of software testing targeting Test Maturity Model integration (TMMi) level 2, and the improvement in the collaboration between academia and industry for other projects. The paper will detail the experience, and list some lessons learned, including how to synchronize the schedules.

This Research Full Paper presents an analysis of the challenges and advantages on applying MOOCs in software engineering teaching and training contexts. Software engineering is a constantly evolving discipline in which educators are involved with a constant flow of new tools, resources and techniques in software development. This scenario makes the act of teaching and contributing to the students' academic education more complex. The insertion of educational technologies brings contributions in this context, causing a transformation in the current scenario of teaching. An example of these new technologies are the MOOCs (Massive Open Online Courses) - open and online courses that are available in providers in partnership with reputable universities. Considering this scenario, this paper aims at identifying the challenges and trends of MOOCs application in software engineering domain, by means of a systematic mapping of the literature (SML). During the SML it was performed an analysis of 5100 papers and selection of 96 primary studies. The outcomes indicate that there is a potential in using MOOCs on teaching or training, as they are an alternative to offer courses applying traditional methods of learning. In addition, the challenges of this direction are analyzed and discussed.

Software testing is a relevant activity to provide evidence of qualifty of software products. However, there is a lack of qualified professionals in this area. This can be caused due to difficulty in teaching software testing through approaches that use only theoretical classes and traditional tools. In addition, there is a lack of motivation due to the work environment and the strategies of allocation and responsibility of these professionals in development and testing teams. To mitigate these problems, approaches have been used to support software testing education, such as: educational games, integrated teaching of software testing with programming, educational modules, among others. The objective of this paper was to develop an educational game named Testing Game, addressing the following topics: functional testing, structural testing and mutation testing. To support the development of the Testing Game, we performed a systematic mapping aiming at selecting a game engine. To evaluate the efficiency of the game, we conducted a feasibility study to evaluate the quality regarding motivation, user experience and learning from the point of view of the students. We also evaluate the usability of the Testing Game. Approximately 85.64% of people who participated in the study assessed the quality of the game in a positive perspective regarding motivation, user experience and learning from the point of view of the students. Regarding the usability of the game, students identified minor problems. Through this work, we realize that the game Testing Game can constitute as a complementary resource to support software testing education.

Mutation testing is a program-transformation technique that injects artificial bugs to check whether the existing test suite can detect them. However, the costs of using mutation testing are usually high, hindering its use in industry. Useless mutants (equivalent and duplicated) contribute to increase costs. Previous research has focused mainly on detecting useless mutants only after they are generated and compiled. In this paper, we introduce a strategy to help developers with deriving rules to avoid the generation of useless mutants. To use our strategy, we pass as input a set of programs. For each program, we also need a passing test suite and a set of mutants. As output, our strategy yields a set of useless mutants candidates. After manually confirming that the mutants classified by our strategy as "useless" are indeed useless, we derive rules that can avoid their generation and thus decrease costs. To the best of our knowledge, we introduce 37 new rules that can avoid useless mutants right before their generation. We then implement a subset of these rules in the MUJAVA mutation testing tool. Since our rules have been derived based on artificial and small Java programs, we take our MUJAVA version embedded with our rules and execute it in industrial-scale projects. Our rules reduced the number of mutants by almost 13% on average. Our results are promising because (i) we avoid useless mutants generation; (ii) our strategy can help with identifying more rules in case we set it to use more complex Java programs; and (iii) our MUJAVA version has only a subset of the rules we derived.

This project is inserted in the research and practice areas that support the development of educational resources, such as, serious games and training simulation, to understand how they should be planned, developed, evaluated, validated and used for the learning and assessment purposes. In this study, we have proposed and specified an agile, integrative and open method for educational resources development, named AIMED, in a holistic and multidisciplinary approach. This approach integrates the areas of simulation modeling, game design, instructional design, principles of agile development methods, project management practices and content domain. AIMED is based on integration of DevJSTA methodology, for creation of serious games, and AM-OER method, for development of open educational resources. A feasibility study was done with serious game development, which should be used and evaluated by domain experts, as well as, the methodology will be used and validated in new case studies.

Extended finite state machines (EFSMs) provide a rigorous model for the derivation of functional tests for software systems and protocols. Various types of data-flow, control-flow, graph-based, and state machine based test selection criteria can be used for deriving tests from a given EFSM specification. Also, traditional types of state machine based notions of faults, such as transfer and output parameter faults, and common types of assignment faults can be used to describe the fault domains of EFSMs. We present an assessment of the most known types of EFSM test selection criteria such as test suites that cover single transfer faults, double transfer faults, single output parameter faults, and many types of single assignment faults of a given EFSM specification. Also, test suites that cover edge-pair, prime path, prime path with side trip, and all-uses criterion are derived from the graph and flow-graph representations of the specification. We also consider transition tour and random test suites. The assessment ranks the considered test suites in terms of their length and their coverage of single transfer, double transfer, and different type of single assignment faults. Dispersion of the obtained results is assessed and results are summarized.

Systems-of-Systems (SoS) refer to large, complex, and software-intensive systems, resulted from the interoperability among heterogeneous, independent constituent systems. The main purpose of SoS is to perform tasks that could not be achieved by these constituents separately; besides, unique SoS characteristics impose new challenges to their development processes. We conducted a Systematic Mapping to identify initiatives, trends, and challenges in the SoS development and, as a result, 32 initiatives were identified. We also evaluated these initiatives with regard to the adherence to the technical processes of IEEE/ISO/IEC 15288:2015 and to the software implementation essential processes of IEEE/ISO/IEC 12207:2008. In general, these initiatives emphasize artifacts in the early stages of the development processes, addressing mainly requirements and architecture. Moreover, 17 of them emphasize the testing activity. We also summarized the SoS characteristics addressed in the studies and discuss the main trends and challenges.

Systems-of-Systems (SoS) refer to a new class of software-intensive systems, where their constituent systems work cooperatively in order to fulfill specific missions. Characterized by managerial and operational independence, geographic distribution, evolutionary development, and emergent behavior, SoS bring substantial challenges to the software engineering area. SESoS 2015, held in Florence, Italy, on May 17, 2015, as a joint workshop of the 37th International Conference on Software Engineering (ICSE), provided a forum to exchange ideas and experiences, analyze current research and development issues, discuss promising solutions, and to explore inspiring visions for the future of Software Engineering (SE) for SoS.

Usability is related to software quality, improving its ability to be understood, operated and attractive to users.We proposed the Design Usability Evaluation (DUE) technologies to allow identifying usability problems earlier in the development of Web applications, through the inspection of mockups.While we found that the DUE technique and tool were effective and efficient in the identification of usability problems, we saw the need to investigate their acceptance in practitioners' work environment.This paper reports the results from a study evaluating the acceptance of the DUE technologies from the point of view of software engineers.We asked questions based on the indicators from the Technology Acceptance Model and identified that a majority of the software engineers who participated in the study: (a) found the DUE technologies useful and easy to use for supporting the usability inspection process; and (b) would regularly use the DUE technologies for future inspections in their job.Nevertheless, the practitioners indicated that the technique should be refined in order to reduce the ambiguity and repetition of some of its items, while the tool should become more intuitive.

One of the activities associated with the Systematic Literature Review (SLR) process is the selection review of primary studies. When the researcher faces large volumes of primary studies to be analyzed, the process used to select studies can be arduous. In a previous experiment, we conducted a pilot test to compare the performance and accuracy of PhD students in conducting the selection review activity manually and using Visual Text Mining (VTM) techniques. The goal of this paper is to describe a replication study involving PhD and Master students. The replication study uses the same experimental design and materials of the original experiment. This study also aims to investigate whether the researcher's level of experience with conducting SLRs and research in general impacts the outcome of the primary study selection step of the SLR process. The replication results have confirmed the outcomes of the original experiment, i.e., VTM is promising and can improve the performance of the selection review of primary studies. We also observed that both accuracy and performance increase in function of the researcher's experience level in conducting SLRs. The use of VTM can indeed be beneficial during the selection review activity.

Context: In order to preserve the value of Systematic Reviews (SRs), they should be frequently updated considering new evidence that has been produced since the completion of the previous version of the reviews. However, the update of an SR is a time consuming, manual task. Thus, many SRs have not been updated as they should be and, therefore, they are currently outdated. Objective: The main contribution of this paper is to support the update of SRs. Method: We propose USR-VTM, an approach based on Visual Text Mining (VTM) techniques, to support selection of new evidence in the form of primary studies. We then present a tool, named Revis, which supports our approach. Finally, we evaluate our approach through a comparison of outcomes achieved using USR-VTM versus the traditional (manual) approach. Results: Our results show that USR-VTM increases the number of studies correctly included compared to the traditional approach. Conclusions: USR-VTM effectively supports the update of SRs.

Both software testing and programming foundations are difficult subjects to be taught. In case of programming, experiences have suggested that the mastering of testing can contribute to enhance the students' skills of comprehension and analysis. So, automatic test-based assessment systems have been developed and adopted in programming courses. Aiming at investigating the adoption of assessment systems also in the teaching of software testing, in this paper we discuss the use of ProgTest - a tool for the submission and automatic evaluation of programming assignments based on testing activities. As most of the automatic assessment systems, ProgTest offers benefits such as immediate feedback, objectivity and consistency of the evaluation, and a substantial time saving in the evaluation of the programming assignments. Additionally, since it performs the assessments based on testing activities, ProgTest also offers benefits for the teaching of software testing. Results from two experiments involving the use of ProgTest in testing courses are presented, providing evidences on the feasibility of its adoption to support the teaching of testing concepts and criteria.

Reference architectures have emerged as a special type of software architecture that achieves well-recognized understanding of specific domains, promoting reuse of design expertise and facilitating the development, standardization, and evolution of software systems. Because of their advantages, several reference architectures have been proposed and have been also successfully used, including in the industry. However, the most of these architectures are still built using an ad-hoc approach, lacking of a systematization to their construction. If existing, these approaches could motivate and promote the building of new architectures and also support evolution of existing ones. In this scenario, the main contribution of this paper is to present the evolution of ProSA-RA, a process that systematizes the design, representation, and evaluation of reference architectures. ProSA-RA has been already applied in the establishment of reference architectures for different domains and this experience was used to evolve our process. In this paper, we illustrate an application of ProSA-RA in the robotics domain. Results achieved through the use of ProSA-RA have showed us that it is a viable, efficient process and, as a consequence, it could contribute to the reuse of knowledge in several applications domains, by promoting the establishment of new reference architectures.

Software Product Line (SPL) has arisen as an approach for developing a family of software-intensive systems at lower costs, within shorter time, and with higher quality. In particular, SPL is supported by a product line architecture (sometimes also referred to as reference architecture) that captures the architectures of a product family. From another perspective, a special type of architecture that contains knowledge about a specific domain has been increasingly investigated, resulting in the research area of Reference Architecture. In spite of the positive impact of this type of architecture on reuse and productivity, the use of the knowledge contained in existing reference architectures in order to develop SPL has not been widely explored yet. The main contribution of this paper is to present a process, named ProSA-RA2PLA, that systematizes the use of reference architectures for building product line architectures. To illustrate the application of this process, we have built a product line architecture for an SPL of software testing tools using a reference architecture of that domain. Based on initial results, we have observed that benefits can be achieved, mainly regarding improvement in reuse and productivity to develop SPL.

Programming foundations is not an easy subject to be taught - many students have difficulties understanding the abstract concepts of programming and have a wrong view about the programming activity. In order to address these problems, experiences have suggested the integrated teaching of programming concepts and software testing in introductory CS courses. Shortly, the idea is that testing can contribute to enhance the students' capabilities of understanding and analysis. However, such perspective requires tools to provide an adequate feedback to evaluate the students' performance concerning programming and testing activities. In this paper we describe ProgTest - a web-based tool for the submission and automatic evaluation of practical programming assignments based on testing activities. Results from a preliminary validation of ProgTest are also presented. Such results provide evidences on the practical use of ProgTest as a supporting mechanism for the integrated teaching of programming foundations and software testing.

Background: A systematic literature review (SLR) is a methodology used to aggregate all relevant existing evidence to answer a research question of interest. Although crucial, the process used to select primary studies can be arduous, time consuming, and must often be conducted manually. Objective: We propose a novel approach, known as 'Systematic Literature Review based on Visual Text Mining' or simply SLR-VTM, to support the primary study selection activity using visual text mining (VTM) techniques. Method: We conducted a case study to compare the performance and effectiveness of four doctoral students in selecting primary studies manually and using the SLR-VTM approach. To enable the comparison, we also developed a VTM tool that implemented our approach. We hypothesized that students using SLR-VTM would present improved selection performance and effectiveness. Results: Our results show that incorporating VTM in the SLR study selection activity reduced the time spent in this activity and also increased the number of studies correctly included. Conclusions: Our pilot case study presents promising results suggesting that the use of VTM may indeed be beneficial during the study selection activity when performing an SLR.

The development of effective learning objects that explores blended learning, collaborative, and open development requires a laborious process. Information created and used at each phase of the process must be manually translated and augmented until achieving a proper product. The issue of systematic development of learning objects can be addressed using model-driven development. In fact, each phase in the development of learning objects requires a specific type of user/developer profile and generates different models. This article defines a model-driven approach for the open and collaborative development of learning objects. The approach herein described uses concept maps, represented as CXL documents, and statechart-based models, which are represented as UML models in XMI documents. Finally, the later model is used to generate learning objects, given a specific platform description. The current implementation supports the generation of slides in LaTeX/Beamer format. The feasibility of the approach is demonstrated using a course on software testing for undergraduate students, with learning objects generated for a context that comprises both traditional classroom and blended learning.

This paper presents the results of an exploratory study on the fault-proneness of aspect-oriented programs. We analysed the faults collected from three evolving aspect-oriented systems, all from different application domains. The analysis develops from two different angles. Firstly, we measured the impact of the obliviousness property on the fault-proneness of the evaluated systems. The results show that 40% of reported faults were due to the lack of awareness among base code and aspects. The second analysis regarded the fault-proneness of the main aspect-oriented programming (AOP) mechanisms, namely pointcuts, advices and intertype declarations. The results indicate that these mechanisms present similar fault-proneness when we consider both the overall system and concern-specific implementations. Our findings are reinforced by means of statistical tests. In general, this result contradicts the common intuition stating that the use of pointcut languages is the main source of faults in AOP.

Web services are an emerging Service-Oriented Architecture technology to integrate applications using open standards based on XML. Software Engineering tools integration is a promising area since companies adopt different software processes and need different tools on each activity. Software engineers could take advantage of software engineering tools available as web services and create their own workflow for integrating the required tools. In this paper, we propose the development of testing tools designed as web services and discuss the pros and cons of this idea. We developed a web service for structural testing of Java programs called JaBUTiService, which is based on the stand-alone tool JaBUTi. We also present an usage example of this service with the support of a desktop front-end and pre prepared scripts. A set of 62 classes of the library Apache-Commons-BeanUtils was used for this test and the results are discussed.

Software architecture has received increasing attention of practitioners and researchers, since it has played a significant role in determining the success and quality of software systems. At the same time, the success of Open Source Software (OSS) has also sparked interest of researchers in the universities and in the software industry. OSS has been largely used and developed and, as a consequence, the OSS quality has been a concern and an interesting subject for researchers. However, in spite of narrow relation between software architecture and software quality, there is lack of more detailed works that investigate how software architecture can influence OSS quality. In this paper, we present a case study reporting how software architecture is directly related to OSS quality. We have hence proposed architecture refactoring activity in order to repair software architectures, aiming at improving mainly maintainability, functionality and usability of these systems.

Educational modules, concise units of study capable of integrating theoretical/practical content and supporting tools, are relevant mechanisms to improve learning processes. In this paper we briefly discuss the establishment of mechanisms to ease the development ofeducational modules - a Standard Process for Developing Educational Modules and an Integrated Modeling Approach for structuring their learning content. The proposed mechanisms have been investigated in the development of the ITonCode module - an educational module for teaching inspection and testing techniques. Aiming at evaluating the module we have replicated an extended version of the Basili & Selby experiment, originally used for comparing V&V techniques, now considering the educational context. The obtained results were mainly analyzed in terms of the student's uniformity in detecting existent faults, giving us very preliminar evidences on the learning effectiveness provided by the module produced.

Experimental replications are very important to the advancement of empirical software engineering. Replications are one of the key mechanisms to confirm previous experimental findings. They are also used to transfer experimental knowledge, to train people, and to expand a base of experimental evidence. Unfortunately, experimental replications are difficult endeavors. It is not easy to transfer experimental know-how and experimental findings. Based on our experience, this paper discusses this problem and proposes a Framework for Improving the Replication of Experiments (FIRE). The FIRE addresses knowledge sharing issues both at the intra-group (internal replications) and inter-group (external replications) levels. It encourages coordination of replications in order to facilitate knowledge transfer for lower cost, higher quality replications and more generalizable results.

Mutation testing has been shown to be one of the strongest testing criteria for the evaluation of both programs and test suites. Comprehensive sets of mutants require strong test sets to achieve acceptable testing coverage. Moreover, mutation operators are valuable for the evaluation of other testing approaches. Although its importance has been highlighted for aspect-oriented (AO) programs, there is still a need for a suitable set of mutation operators for AO languages. The quality of the mutation testing itself relies on the quality of such operators. This paper presents the design of a set of mutation operators for AspectJ-based programs. These operators model instances of fault types identified in an extensive survey. The fault types and respective operators are grouped according to the related language features. We also discuss the generalisation of the fault types to AO approaches other than AspectJ and the coverage that may be achieved with the application of the proposed operators. In addition, a cost analysis based on two case studies involving real-world applications has provided us feedback on the most expensive operators, which will support the definition of further testing strategies.

The software engineering research community has been adopting systematic reviews as an unbiased and fair way to assess a research topic. Despite encouraging early results, a systematic review process can be time consuming and hard to conduct. Thus, tools that help on its planning or execution are needed. This article suggests the use of Visual Text Mining (VTM) to aid systematic reviews. A feasibility study was conducted comparing the proposed approach with a manual process. We observed that VTM can contribute to Systematic Review and we propose a new strategy called VTM-Based Systematic Review.

Abstract This paper describes intra‐method control‐flow and data‐flow testing criteria for the Java bytecode language. Six testing criteria are considered for the generation of testing requirements: four control‐flow and two data‐flow based. The main reason to work at a lower level is that, even when there is no source code, structural testing requirements can still be derived and used to assess the quality of a given test set. It can be used, for instance, to perform structural testing on third‐party Java components. In addition, the bytecode can be seen as an intermediate language, so the analysis performed at this level can be mapped back to the original high‐level language that generated the bytecode. To support the application of the testing criteria, we have implemented a tool named JaBUTi (Java Bytecode Understanding and Testing). JaBUTi is used to illustrate the application of the ideas developed in this paper. Copyright © 2006 John Wiley & Sons, Ltd.

Abstract Testing and debugging activities consume a significant amount of the software development and maintenance budget. To reduce this cost, the use of testing information for debugging purposes has been advocated. In general, heuristics are used to select structural testing requirements (nodes, branches and definition–use associations) more closely related to the manifestation of a failure, which are then mapped into a piece of code. The intuition is that the selected piece of code is likely to contain the fault. However, this approach has its drawbacks. Heuristics that select a manageable piece of code are less likely to hit the fault and the piece of code itself does not provide enough guidance for program understanding—a major factor in program debugging. These problems occur because this approach relies only on static information—a fragment of code. We introduce a strategy for fault localization that addresses these problems. The strategy—called the debugging strategy based on the requirements of testing (DRT)—is based on the investigation of indications (or hints) provided at run‐time by data‐flow testing requirements (definition–use associations). Our claim is that the selected definition–use associations may fail to hit the fault site, but still provide information useful for fault localization. The strategy's novelty and attractiveness are threefold: (i) the focus on dynamic information related to testing data; (ii) implementation in state‐of‐the‐practice symbolic debuggers with a low overhead; and (iii) the use of algorithms which consume constant memory and are linear on the number of branches in the program. A case study shows that our claim is valid (for the subject program) and a prototype tool implements the strategy. Copyright © 2004 John Wiley & Sons, Ltd.

Statistical data show that it is much less expensive to correct software bugs at the early design stage rather than the late stage of the development process when the final system has already been implemented and integrated together. The use of slicing and execution histories as an aid in software debugging is well established for programming languages like C and C++; however, it is rarely applied in the field of software design specification. We propose a solution by applying the source code level technologies to debugging software designs represented in a high-level specification and description language such as SDL. More specifically, we extend execution slice-based heuristics from source code-based debugging to the software design specification level. Suspicious locations in an SDL specification are prioritized by their likelihood of containing faults. Locations with a higher priority should be examined first rather than those with a lower priority as the former are more likely to contain the faults. A debugging tool, SmartD/sub DSL/, with user-friendly interfaces was developed to support our method. An illustration is provided to demonstrate the feasibility of using our method to effectively debug an architectural design.

Though software testing courses are commonly taught as part of software engineering curricula, software testing is still a challenging issue in software engineering education. Students frequently see testing only as something that happens at the end of the development process. Two challenges can be recognized: "how to make the students recognize the relevance of the testing activity?" and "how to motivate the students on using testing ideas in their projects?". In an attempt to explore the impact of introducing testing practices throughout development, during the past Fall semester we modified the project requirements in a course on object-oriented analysis and design offered to the undergraduate students in computer science at Georgia Institute of Technology. Our idea was to require the students to start thinking about testing as early as possible, by including testing-related practices in all phases of the development process. This paper presents the details of the testing approach used in the course and discusses the results we obtained, in terms of the students' attitudes and learning.

Mutation testing (Mutation Analysis), although powerful in revealing faults, is considered a computationally expensive criterion, due to the high number of mutants created and the effort to determine the equivalent mutants. Using mutation-based alternative testing criteria it is possible to reduce the number of mutants but it is still necessary to determine the equivalent ones. In this paper the Bayesian Learning(one of the Artificial Intelligence techniques used in machine learning) is investigated to define the Bayesian Learning-Based Equivalent Detection Technique (BaLBEDeT), which provides guidelines to help the tester to analyze the live mutants in order to determine the equivalent ones.

Abstract Mutation testing, originally proposed for unit testing, has been extended to integration testing with the proposition of the Interface Mutation criterion. This paper presents the results of an experiment using two mutation‐based testing criteria for unit and integration testing phases: the Mutation Analysis and the Interface Mutation adequacy criteria, respectively. The aim is to investigate how they can be used in a complementary way during the testing activity, establishing an incremental testing strategy comprising the unit and integration testing phases and guidelines on how to obtain a high mutation score with respect to mutation testing with a low cost, in terms of the number of mutants generated. Copyright © 2001 John Wiley & Sons, Ltd.

The need for test adequacy criteria is widely recognized. Several criteria have been proposed for the assessment of adequacy of tests at the unit level. However, there remains a lack of criteria for the assessment of the adequacy of tests generated during integration testing. We present a mutation based interprocedural criterion, named Interface Mutation (IM), suitable for use during integration testing. A case study to evaluate the proposed criterion is reported. In the study, the UNIX sort utility was seeded with errors and Interface Mutation evaluated by measuring the cost of its application and its error revealing effectiveness. Alternative IM criteria using different sets of Interface Mutation operators were also evaluated. While comparing the error revealing effectiveness of these Interface Mutation-based test sets with same size randomly generated test sets, we observed that in most cases Interface Mutation based test sets are superior. The results suggest that Interface Mutation offers a viable test adequacy criteria for use at the integration level.

The establishment of a low-cost, effective testing and validation strategy has been pursued by many researchers at the program level as well as at the specification level. The application of mutation testing for validating specifications based on statecharts is proposed. A mutation operator set for statecharts, one of the crucial points for effectively applying mutation testing is defined; in this scope these operators can be taken as a fault model. We also provide strategies to abstract the statechart components according to different statechart features that may comprise the testing and validation activity aims, providing in this way mechanisms for the establishment of an incremental, hierarchical, mutation-based testing strategy. Implementation and functional aspects of PROTEUM/ST, a tool under development are also presented.

Although there are many algorithms to draw hierarchical structures such as directed graphs and trees none specifically treat the problem of visualizing program graphs. This paper presents an algorithm and the underlying tool — ViewGraph — designed to visualize program graphs. The algorithm is divided in two parts: (1) determine node positions, and (2) assign routes to branches. The first part has three steps: level assignment, scope and position calculation. A modified algorithm used to assign levels to nodes in a tree is used to find Y coordinates; a concept called scope is used to define X coordinates. Scope is a prediction of the space required by a node and its descendants. A search on the available positions left by the placement of nodes is performed to route branches. A set of aesthetic aspects meant to help the development of program graph visualization algorithms is also proposed. The algorithm runs in an acceptable time making it useful even for interactive applications. © 1997 John Wiley & Sons, Ltd.

A criterion for assessing the adequacy of test sets during integration testing is proposed. The criterion is based on a testing technique named Interface Mutation. The technique itself is designed to be scalable with the size of the software under test; the size being measured in the number of subsystems integrated. Using Interface Mutation it is possible to assess the adequacy of tests incrementally while integrating various subsystems. Also reported are results from a pilot experiment conducted to study the cost and error defection effectiveness of Interface Mutation.

Proposes the application of the mutation analysis criterion in the context of specifications based on finite state machines. The main concepts of finite state machines and mutation analysis are briefly introduced. An experiment is reported which manually applies mutation analysis to a finite state machine modeling a Class 0 ISO transport protocol specification, using two test-sequence generator criteria-the W method and the TT (transition tours) method. The results obtained are presented, and evidences are given that the use of mutation analysis is effective in this context. Finally, the lines of evolution of the work presented in this paper are briefly discussed.>