Software Process Improvement (SPI) consists of a set of changes in a software development company, introducing new and improved methods, techniques, and tools. We call SPI Diagnostic, the process to know the organization current status. Typically, these SPI Diagnostic processes are manually performed, thus demanding consultant practitioner and a high effort from the organization under analysis. Based on this we propose the following question: “What solutions have been proposed to SPI Diagnostic?” We looked for other Systematic Literature Reviews with the same goal and the found studies do not answer our question. Hence, we performed a Systematic Literature Review (SLR) to investigate solutions to the SPI Diagnostic process. We executed an SLR, based on snowballing technique. This SLR characterizes 14 solutions aiming at systematizing the SPI Diagnostic process through a method, model, or framework. As a result, we advocate that Artificial Intelligence (AI) based solutions should be more explored in research to deal with SPI Diagnostic complexity and, therefore helping the SPI Diagnostic.

Background: Systematic Reviews (SR) are crucial for collecting evidence related with a particular field or area. Hence, there are different protocols for conducting SR in software engineering. There is also several tools designed to provide support for these protocols. Aims: We understand that there is still a lack of a web-tool, fully integrating the features considered important for SR. Implementing such a tool would be a great contribution to the field. Method: We analyzed several studies discussing features for tools supporting SR. We defined design decisions based on these features. Results: We created Thoth, a web-based tool for SR. In this study we present a first version, containing a set of features. Before moving forward with the implementation, we conducted a quasi-experiment for evaluating and comparing Thoth with a similar tool. Conclusion: Despite being in the right direction for achieving our goal, the results of the quasi-experiment provided us with important evidence for improving Thoth.

Summary Software product lines (SPLs) are a well‐known solution to systematically create reusable software products. Among the approaches to create an SPL, the extractive approach is usually used when the organization already has a set of similar systems. These systems are analyzed to extract, categorize, and group their common and variant features throughout the SPL reengineering process. As there are different scenario variables, such as available artifacts and team experience, the activities and techniques used to perform these tasks may change. This may increase the effort and decrease the quality of retrieved features when users with low experience in SPL reengineering perform such tasks. However, there is a lack of a process supporting these tasks considering different scenarios. Therefore, we specify the Prepare, Assemble, and Execute Process for SPL Reengineering (PAxSPL), a process that provides support to prepare, assemble, and execute feature retrieval throughout the analysis of documentation and team experience. To initially evaluate PAxSPL, we conducted and reported an exploratory case study in a real development environment. The results indicated that our proposal helps in the assembly of a feature retrieval process according to user needs. Results were important to identify points for improvement in PAxSPL. We also could use the information gathered to improve the guidelines and provide this information to be used as basis of comparison for future users.

Despite all the efforts to reduce the cost of the testing phase in software development, it is still one of the most expensive phases. In order to continue to minimize those costs, in this paper, we propose a Domain-Specific Language (DSL), built on top of MetaEdit+ language workbench, to model performance testing for web applications. Our DSL, called Canopus, was developed in the context of a collaboration1between our university and a Technology Development Laboratory (TDL) from an Information Technology (IT) company. We present, in this paper, the Canopus metamodels, its domain analysis, a process that integrates Canopus to Model-Based Performance Testing, and applied it to an industrial case study.

Feature models are considered a reference in the majority of Software Product Line (SPL) methods.There are several different feature model notations to represent requirements of an SPL, at a higher or lower level of abstraction.Some notations present properties to distinguish functional, conceptual and non-functional features of an SPL.Non-functional requirements, especially those that involve the construction of user interfaces (UI), are usually not represented in the feature models, since the user interfaces are often created manually for each product.In this paper we present an experimental study performed in order to evaluate the effort required to, as well as the benefits and drawbacks of representing UI elements during the feature modeling of a Financial SPL.To this end, we applied the Odyssey-Fex and UI-Odyssey-Fex notations to design feature models from the perspective of Domain Engineers in the context of undergraduates, M.Sc.and Ph.D. students and software engineers with some expertise in SPL.Our results indicate that the effort to use the notations are similar, but the use of the UI-Odyssey-Fex notation provides a better representativeness of UI elements.

Performance testing is a highly specialized task, since it requires that a performance engineer knows the application to be tested, its usage profile, and the infrastructure where it will execute. Moreover, it requires that testing teams expend a considerable effort and time on its automation. In this paper, we present the PLeTsPerf, a model-based performance testing tool to support the automatic generation of scenarios and scripts from application models. PLetsPerf is a mature tool, developed in collaboration with an IT company, which has been used in several works, experimental studies and pilot studies. We present an example of use to demonstrate the process of generating test scripts and scenarios from UML models to test a Web application. We also present the lessons learned and discuss our conclusions about the use of the tool.

This work presents an experience report on the architectural decisions taken in the evolution of a Software Product Line (SPL) of Model-based Testing tools (PLeTs).This SPL was partially designed and developed with the intention of minimizing effort and time-to-market during the development of a family of performance testing tools.With the evolution of our research and the addition of new features to the SPL, we identified limitations in the initial architectural design of PLeTs' components, which led us to redesign its Software Product Line Architecture (SPLA).In this paper, we discuss the main issues that led to changes in our SPLA, as well as present the design decisions that facilitate its evolution in the context of an industrial environment.We will also report our experiences on architecture modifications in the evolution of our SPL with the intention of allowing easier maintenance in a volatile development environment.

This work presents an experience report on the architectural decisions taken in the evolution of a Software Product Line (SPL) of Model-based Testing tools (PLeTs). This SPL was partially designed and developed with the intention of minimizing effort and time-to-market during the development of a family of performance testing tools. With the evolution of our research and the addition of new features to the SPL, we identified limitations in the initial architectural design of PLeTs’ components, which led us to redesign its Software Product Line Architecture (SPLA). In this paper, we discuss the main issues that led to changes in our SPLA, as well as present the design decisions that facilitate its evolution in the context of an industrial environment. We will also report our experiences on architecture modifications in the evolution of our SPL with the intention of allowing easier maintenance in a volatile development environment.

[Context] A variety of testing tools have been developed to support and automate software performance testing activities. These tools may use different techniques, such as Model-Based Testing (MBT) or Capture and Replay (CR). [Goal] For software companies, it is important to evaluate such tools w.r.t. the effort required for creating test artifacts using them; despite its importance, there are few empirical studies comparing performance testing tools, specially tools developed with different approaches. [Method] We are conducting experimental studies to provide evidence about the required effort to use CR-based tools and MBT tools. In this paper, we present our first results, evaluating the effort (time spent) when using LoadRunner and Visual Studio CR-based tools, and the PLeTsPerf MBT tool to create performance test scripts and scenarios to test Web applications, in the context of a collaboration project between Software Engineering Research Center at PUCRS and a technological laboratory of a global IT company. [Results] Our results indicate that, for simple testing tasks, the effort of using a CR-based tool was lower than using an MBT tool, but as the testing complexity increases tasks, the advantage of using MBT grows significantly. [Conclusions] To conclude, we discuss the lessons we learned from the design, operation, and analysis of our empirical experiment.

Service-oriented architectures and Web services have been widely adopted by enterprises to pervade integration among software systems. As reliable services are essential to assure that these systems work correctly, formal and systematic testing should be performed. This paper reports the application of a model-based approach to test Web services in the context of real-world applications of a multinational computer technology corporation. The employed approach is called ESG4WSC, in which an event-driven model is provided to support modeling and test case generation, as well as an environment to support the concretization and test execution.