CONTEXT. Focus Group (FG) is a qualitative technique that collects data through moderated group discussion. Although widely used in fields where human behavior is central, their use in Software Engineering (SE) is comparatively less common. In SE, FG applications often need to accommodate technical domains, participants with specialized expertise, and research goals that differ from those typically addressed in traditional social science settings. OBJECTIVE. This study analyzes four FG experiences conducted to investigate distinct SE technologies and topics. The objective is to examine how design decisions, execution strategies, and contextual conditions support or hinder the identification of the benefits and drawbacks of using FGs in SE research. METHOD. We performed a reflective cross-case methodological analysis of four FGs conducted in both academic and industrial settings. The studies differed in goals, group configurations, participant expertise, and moderation strategies. Data sources included session transcripts, written artifacts, moderator observations, and participant feedback. RESULTS. The results indicate that data quality depended on participants’ prior knowledge, the moderator's impartiality, the execution context, and the use of multimodal data capture. Depth and generalizability were limited by time constraints, uneven engagement, and the diverse qualitative data that hindered comparison. CONCLUSIONS. FG is an effective complementary method in SE research, particularly for exploratory analysis and for understanding practitioner perspectives on SE technologies. However, its effectiveness depends on deliberate design choices, including participant selection and preparation, group configuration, and systematic documentation.

ABSTRACT The COVID-19 pandemic has permanently altered workplace structures, making remote work a widespread practice. While many employees advocate for flexibility, many employers reconsider their attitude toward remote work and opt for structured return-to-office mandates. Media headlines repeatedly emphasize that the corporate world returns to full-time office work. This study examines how companies in software-intensive industry regulate work location, whether corporate policies have evolved in the last five years, and, if so, how, and why. We collected data on remote work regulation from corporate HR and management representatives from 68 companies that vary in size, location, and preferred work modality. Our findings reveal that although many companies prioritize office-oriented work (50%), most companies in our sample permit hybrid work (84%) and only four companies are returning to full-time office work. Remote work regulation does not reveal any particular new “best practice” as policies differ greatly; however, the single most popular arrangement was the three in-office days per week. More than half of the companies (53%) encourage or mandate office attendance centrally, with additional 18% having decentralized mandates. Over a quarter (28%) have changed regulations gradually increasing the mandatory office presence or implementing differentiated conditions. Our key recommendation for office-oriented companies is to consider trust-based recommendations as an alternative to centralized office presence mandates, while for companies oriented toward remote working, we warn about the points of no (or hard) return. Finally, the current state of policies is clearly not final, as companies continue to experiment and adjust their work regulation

Abstract Software quality is undergoing a profound transformation, driven by state-of-the-art research on the application of emerging technologies in software development processes. Specifically, the use of generative Artificial Intelligence (AI) may represent an opportunity to advance the state of practice in this domain. This study aims to assess the industrial readiness and availability of Generative AI-based solutions for software quality, classifying them according to ISO/IEC 25010 attributes and SDLC phases. An empirical assessment of the state of practice was conducted, employing a Rapid Multivocal Literature Review (RMLR) protocol as a data collection instrument to screen evidence from academic databases (Scopus) and grey literature (Google, GitHub, PapersWithCode). We identified 24 potentially usable solutions. However, the analysis reveals a low technological maturity, with most solutions being academic prototypes hampered by fundamental technical limitations and adoption challenges. These include the “last mile problem” in translating research prototypes into reliable, production-ready tools; the “strategic adoption dilemma” forcing practitioners to trade off between proprietary lock-in and high open-source infrastructure costs; and the “scarcity of realistic public data,” which drives a generalization gap due to reliance on synthetic or leaked benchmarks. Generative AI in software quality remains an emerging but immature field, hampered by a critical reliability gap between academic prototypes and industrial needs. Advancing this domain requires moving beyond a narrow code-centric focus to address the quality of the AI systems themselves, expanding research across all SDLC phases and ISO 25010 attributes. We conclude with a roadmap advocating for contamination-free benchmarks, explainable architectures, and robust guidelines for real-world integration.

Background: As empirical software engineering evolves, more studies adopt data strategies-approaches that investigate digital artifacts such as models, source code, or system logs rather than relying on human subjects. Synthesizing results from such studies introduces new methodological challenges. Aims: This study assesses the effects of model quantization on correctness and resource efficiency in deep learning (DL) systems. Additionally, it explores the methodological implications of aggregating evidence from empirical studies that adopt data strategies. Method: We conducted a research synthesis of six primary studies that evaluate model quantization. We applied the Structured Synthesis Method (SSM) to aggregate the findings, which combines qualitative and quantitative evidence through diagrammatic modeling. A total of 19 evidence models were extracted and aggregated. Results: The aggregated evidence indicates that model quantization weakly negatively affects correctness metrics while consistently improving resource efficiency metrics, including storage size, inference latency, and GPU energy consumption-a manageable trade-off for many DL deployment contexts. Evidence across quantization techniques remains fragmented, underscoring the need for more focused empirical studies per technique. Conclusions: Model quantization offers substantial efficiency benefits with minor trade-offs in correctness, making it a suitable optimization strategy for resource-constrained environments. This study also demonstrates the feasibility of using SSM to synthesize findings from data strategy-based research.

As automotive software systems evolve toward high and full driving automation, evaluating their quality becomes increasingly challenging, especially concerning emerging behaviors. Context awareness is the capability to sense the environment and adapt behavior. Automotive software systems are context-aware software systems (CASS). Previous secondary studies in technical literature indicate a need for testing techniques for CASS. However, these studies should have investigated the information provided by the industry. Therefore, this article undertakes a gray literature study to uncover evidence of CASS testing using 20 reports from 16 automotive companies as primary sources. Our findings show that industry practices exhibit quality assurance best practices, but CASS abstraction adoption still needs to be completed. Industry reports emphasize testing challenges but lack technical resolutions, relying on amassing diverse datasets for testing. This article has the potential to impact the quality assurance of automotive software systems significantly and lead industry professionals to enhance their testing process.

Technical debt (TD) expresses the lack of internal quality directly affecting software evolution. Therefore, it has gained the attention of software researchers and practitioners recently. Software researchers have performed empirical studies to observe the perspective of TD in different software cultures and organizations. However, it is important to replicate such studies in more places and with more practitioners to strengthen the perception of TD. In this paper, we present the results of a set of new research questions from an evolved survey design of a survey replication in the Uruguayan software industry to characterize how the software industry professionals understand, perceive, and adopt TD management (TDM) activities. The results allow us to observe that different participant contexts (startups, government, job roles) show different levels of awareness and perception of TD. Details in the form of the adoption of each TDM activity were presented. We could observe some difficulties in conducting some TDM activities that the practitioners consider very important, especially in TDM and monitoring. Differences in specific organizational contexts like startups and government could indicate the need for research efforts in other software engineering communities that meet their specific TD challenges and needs.

IoT, Industry 4.0, and ubiquitous applications demand an intense exchange of information between actors (humans, animals, plants, or other objects) and things. Different interaction technologies, techniques, and methods can support it, including RFID, a well-known technology capable of identifying and locating actors. However, the variety of application domains and interaction possibilities jeopardize the use of RFID without a clear understanding of its usage issues. Therefore, we have performed a systematic mapping to characterize the adoption of RFID technology in this kind of interactive software system by intercepting human-computer interaction and software engineering issues. Our goal is to reveal (i) which application domains address RFID; (ii) why RFID is used instead of other technologies; (iii) who interacts through RFID and how the interaction is; (iv) what are the requirements and limitations associated with it; and (v) what are the common quality characteristics and measures which should be considered for their evaluation. The results showed that (i) RFID is specially used for implementing implicit interactions in healthcare, embedded systems, and manufacturing applications. (ii) Passive UHF RFID is usually adopted due to being contactless, low-cost, and well-performing indoors. (iii) It interacts with humans and things in the same proportion, most frequently to collect locations of sensor data constantly. (iv) RFID presents limitations, such as causing electromagnetic interference and not working well near metallic walls. However, tags work under high temperatures and can be well hidden in cement or clothes. (v) Interactive system designers usually create applications with invisible infrastructures, which tend to keep users' focus on the real world.?

Context: Context-awareness challenges the engineering of contemporary software systems and jeopardizes their testing. The variation of context represents a relevant behavior that deepens the limitations of available software testing practices and technologies. However, such software systems are mainstream. Therefore, researchers in non-academic settings also face challenges when developing and testing contemporary soft-ware systems. Objective: To understand how researchers deal with the variation of context when testing context-aware software systems developed in non-academic settings. Method: To undertake a secondary study (Rapid Review) to uncover the necessary evidence from primary sources describing the testing of context-aware software systems outside academia. Results: The current testing initiatives in non-academic settings aim to generate or improve test suites that can deal with the context variation and the sheer volume of test input possibilities. They mostly rely on modeling the systems' dynamic behavior and increasing computing resources to generate test inputs to achieve this. We found no evidence of test results aiming at managing context variation through the testing lifecycle process. Conclusions: So far, the identified testing initiatives and strategies are not ready for mainstream adoption. They are all domain-specific, and while the ideas and approaches can be reproduced in distinct settings, the technologies are to be re-engineered and tailored to the context-awareness of contemporary software systems in different problem domains. Further and joint investigations in academia and experiences in non-academic settings can evolve the body of knowledge regarding the testing of contemporary soft-ware systems in the field.

Context: The relevance of Requirements Engineering (RE) research to practitioners is vital for a long-term dissemination of research results to everyday practice. Some authors have speculated about a mismatch between research and practice in the RE discipline. However, there is not much evidence to support or refute this perception.Objective: This article presents the results of a study aimed at gathering evidence from practitioners about their perception of the relevance of RE research and at understanding the factors that influence that perception.Method: We conducted a questionnaire-based survey of industry practitioners with expertise in RE. The participants rated the perceived relevance of 435 scientific papers presented at five top RE-related conferences.Results: The 153 participants provided a total of 2,164 ratings. The practitioners rated RE research as essential or worthwhile in a majority of cases. However, the percentage of non-positive ratings is still higher than we would like. Among the factors that affect the perception of relevance are the research's links to industry, the research method used, and respondents’ roles. The reasons for positive perceptions were primarily related to the relevance of the problem and the soundness of the solution, while the causes for negative perceptions were more varied. The respondents also provided suggestions for future research, including topics researchers have studied for decades, like elicitation or requirement quality criteria.Conclusions: The study is valuable for both researchers and practitioners. Researchers can use the reasons respondents gave for positive and negative perceptions and the suggested research topics to help make their research more appealing to practitioners and thus more prone to industry adoption. Practitioners can benefit from the overall view of contemporary RE research by learning about research topics that they may not be familiar with, and compare their perception with those of their colleagues to self-assess their positioning towards more academic research.

JavaScript is one of the most used programming languages for front-end development of Web applications. The increase in complexity of front-end features brings concerns about performance, especially the load and execution time of JavaScript code. In this paper, we propose an evolutionary program improvement technique to reduce the size of JavaScript programs and, therefore, the time required to load and execute them in Web applications. To guide the development of this technique, we performed an experimental study to characterize the patches applied to JavaScript programs to reduce their size while keeping the functionality required to pass all test cases in their test suites. We applied this technique to 19 JavaScript programs varying from 92 to 15,602 LOC and observed reductions from 0.2 to 73.8 percent of the original code, as well as a relationship between the quality of a program's test suite and the ability to reduce the size of its source code.

Background: Technical debt (TD) has been an important focus of attention in recent years by the scientific community and the software industry. TD is a concept for expressing the lack of internal software quality that directly affects its capacity to evolve. Some studies have focused on the TD industry perspective. Aims: To characterize how the software industry professionals in Uruguay understand, perceive, and adopt technical debt management (TDM) activities. Method: To replicate a Brazilian survey with the Uruguayan software industry and compare their findings. Results: From 259 respondents, many indicated any awareness of the TD concept due to the faced difficult to realize how to associate such a concept with actual software issues. Therefore, it is possible to observe a considerable variability in the importance of TDM among the respondents. However, a small part of the respondents declares to carry out TDM activities in their organizations. A list of software technologies declared as used by practitioners was produced and can be useful to support TDM activities. Conclusions: The TD concept and its management are not common yet in Uruguay. There are indications of TD unawareness and difficulties in the conduction of some TDM activities considered as very important by the practitioners. There is a need for more effort aiming to disseminate the TD knowledge and to provide software technologies to support the adoption of TDM in Uruguay. It is likely other software engineering communities face similar issues. Therefore, further investigations in these communities can be of interest.

New technologies and approaches lead to the Internet of Things (IoT) paradigm. Such a paradigm enables the engineering of more autonomous and smarter software systems solutions. Due to its multidisciplinary nature, it involves several knowledge areas such as software and connectivity that should be combined coherently and uniformly. It should comprise different voices and expertise to deal with IoT in a multi-faceted way. In this context, what to consider while specifying, designing, and implementing IoT software systems? This work raises this discussion by defining a roadmap about what should be considered for engineering IoT software respecting all their facets. Such a roadmap is defined based on evidence acquired at the technical literature combined with a qualitative study.

Background: Software product quality assurance affects the acceptance of releases. The one dimensional observational perspective of current software product quality (SPQ) models constrains their use in continuous software engineering environments. Aims: To investigate multidimensional relationships between software product characteristics and build an evidence-based infrastructure to observe SPQ continuously. Method: To mine and manipulate datasets regarding software development and use. Next, to perform multidimensional analytical SPQ interpretations to observe quality. Results: There is empirical evidence on the multidimensionality linkage of quality characteristics throughout the software life cycle. Conclusions: The one-dimensional quality perspective is not enough to observe the SPQ in continuous environments. Alternative mathematical abstractions should be investigated.

The evidence-based software engineering approach advocates the use of scientific evidence by software engineers to support the adoption of software technologies in industrial software development and maintenance projects. Aside from the unavailability of scientific knowledge in industrial settings and the time required to acquire evidence in the software engineering (SE) field, additional challenges prevent practitioners, mainly those that are not experienced in research, from collecting knowledge from scientific sources to support the decisionmaking throughout their software projects.

Background: research synthesis is still challenge in Software Engineering due to the heterogeneity of primary studies in the area. Also, it generates a significant volume of information which is complex to manage. Aims: to provide support to this kind of studies in SE. Method: we present the Evidence Factory, a tool designed to support the Structured Synthesis Method (SSM). SSM is a research synthesis method that can be used to aggregate both quantitative and qualitative studies. It is a kind of integrative synthesis method, such as meta-analysis, but has several features from interpretative methods, such as meta-ethnography, particularly those concerned with conceptual development. Results: the tool is a web-based infrastructure, which supports the organization of synthesis studies. Researchers can compare findings from different studies by modeling their results according to the evidence meta-model. After deciding whether the evidence can be combined, the tool automatically computes the uncertainty associated with the aggregated results using the formalisms from the Mathematical Theory of Evidence. Conclusion: the tool was used in real synthesis studies and is freely available for the SE community.

The relevance of Requirements Engineering (RE) research to practitioners is a prerequisite for problem-driven research in the area and key for a long-term dissemination of research results to everyday practice. To understand better how industry practitioners perceive the practical relevance of RE research, we have initiated the RE-Pract project, an international collaboration conducting an empirical study. This project opts for a replication of previous work done in two different domains and relies on survey research. To this end, we have designed a survey to be sent to several hundred industry practitioners at various companies around the world and ask them to rate their perceived practical relevance of the research described in a sample of 418 RE papers published between 2010 and 2015 at the RE, ICSE, FSE, ESEC/FSE, ESEM and REFSQ conferences. In this paper, we summarize our research protocol and present the current status of our study and the planned future steps.

Context: The representativeness of samples in Software Engineering primary studies is still a great challenge, mainly when identifying available sources for establishing adequate sampling frames, characterizing subjects and stimulating their participation in (opinion) surveys. The lack of survey guidelines taking into account the specificities of Software Engineering increases the research challenge. Goal: To introduce a conceptual framework for supporting the identification of representative samples for surveys in Software Engineering. Method: Based on knowledge acquired in the technical literature and researchers' experience, to organize a set of guidelines to systematically support sampling in Software Engineering surveys. To perform in vitro empirical studies to observe and evolve the guidelines. Results: An empirically evaluated set of planning activities and tasks with recommendations to support the identification of representative samples for surveys in Software Engineering is available. Conclusion: Surveys have been supporting relevant investigations in Software Engineering in the last decades. This conceptual framework can contribute to strength representativeness of their results. However, some important issues regarding survey research are still open and deserves attention from the empirical Software Engineering community.

Context: Several empirical studies investigated the benefits and drawbacks of acquiring a Software Reference Architecture (SRA) to construct a family of software systems with similar architectural needs. However, these empirical results have not been synthesized by any study yet. Such synthesized evidence is essential to make informed decisions whether or not to adopt an SRA in an organization. Goal: To aggregate existing empirically- grounded evidence about the benefits and drawbacks of SRAs, aiming at supporting organizations' decision making on their adoption. Method: To identify primary studies in the technical literature through a systematic literature review, and then, use the Structured Synthesis Method (SSM) to aggregate qualitative and quantitative evidence through the use of diagrammatic models. Results: From the five identified primary studies, five SRA benefits have considerably increased their belief value after aggregation: interoperability of software systems, reduced development costs, improved communication among stakeholders, reduced risk, and reduced time- to-market. Also, one drawback of SRAs has increased its belief value: the required learning curve for developers. Conclusions: The aggregated results consolidate knowledge and confidence on some of the studied SRA effects. The commonly reported effects showed a clear increment of their belief and pointed out to broader generalization. The effects that did not show any belief increment are important to detect areas requiring further evidence to reach a higher degree of consolidation. Practitioners might benefit from these results to support the decision of adopting an SRA in practice.

Context: The results of large-scale studies in software engineering can be significantly impacted by samples' representativeness. Diverse population sources can be used to support sampling for such studies. Goal: To compare two samples, one from the crowdsourcing platform Mechanical Turk and another from the professional social network LinkedIn, in an online experiment for evaluating the relevance of Java code snippets to programming tasks. Method: To compare the samples (subjects' experience, programming habits) and experimental results concerned with three experimental trials. Results: LinkedIn's subjects present significantly higher levels of experience in Java programming and programming in general than Mechanical Turk's subjects. The experimental results revealed a significant difference between samples and suggested that LinkedIn's subjects were more pessimistic than Mechanical Turk's subjects despite a high level consistency in the experimental results. Conclusion: The combined use of sources of sampling can bring benefits to large scale studies in software engineering, especially when heterogeneity is desired in the population. Thus, it can be useful to investigate and characterize alternative sources of sampling for performing large-scale studies in software engineering.

Context: The low quality and small size of samples in empirical studies in software engineering hamper the interpretation and generalization of their results. Therefore, enlarging sample sizes and improving their quality represent an important research challenge. Goal: We aim to define a conceptual framework, including requirements for establishing adequate sources for sampling subjects in software engineering surveys. Method: We use previous experience on applying systematic sampling strategies combined with contemporary web technologies in previously executed surveys, to organize the conceptual framework. We analyze its application to different sources of sampling. Results: The framework was observed to be feasible after its application to nine different large-scale sources of sampling. Conclusions: The analyzed crowdsourcing tools do not support essential requirements to be considered sources of sampling, while free-lancing tools and professional social network do.

Context: Small and non-probabilistic samples represent relevant issues when discussing the external validity of empirical studies in Software Engineering. Goal: To investigate alternatives to improve the quality of samples (size, heterogeneity and level of confidence). Method: To replicate a survey on characteristics of agility in software processes by applying a systematic recruitment strategy over a professional social network. Results: It resulted in a sampling frame composed by 19 groups stratified according two perspectives: sharing of groups' members and main software engineering skills reported by the subjects. In total, 7,745 subjects were randomly recruited, resulting in 291 contributions. Conclusions: This sample was significantly larger, more heterogeneous and presents some strata with higher confidence levels than previous trials samples.

The technical literature on model-based testing (MBT) offers us several techniques with different characteristics and goals. Contemporary software projects usually need to make use of different software testing techniques. However, a lack of empirical information regarding their scalability and effectiveness is observed. It makes their application difficult in real projects, increasing the technical difficulties to combine two or more MBT techniques for the same software project. In addition, current software testing selection approaches offer limited support for the combined selection of techniques. Therefore, this paper describes the conception and evaluation of an approach aimed at supporting the combined selection of MBT techniques for software projects. It consists of an evidence-based body of knowledge with 219 MBT techniques and their corresponding characteristics and a selection process that provides indicators on the level of adequacy (impact indicator) amongst MBT techniques and software projects characteristics. Results from the data analysis indicate it contributes to improve the effectiveness and efficiency of the selection process when compared to another selection approach available in the technical literature. Aiming at facilitating its use, a computerized infrastructure, evaluated into an industrial context and evolved to implement all the facilities needed to support such selection approach, is presented.

Context: Two of the most common external threats to validity in quantitative studies in software engineering (SE) are concerned with defining the population by convenience and nonrandom sampling assignment. Although these limitations can be reduced by increasing the number of replications and aggregating their results, the acquired evidence rarely can be generalized to the field.

Quantitative studies in Software Engineering are frequently dependent on primary studies in which population is usually small and established by convenience. It brings several limitations for the analysis and strength of results due sampling issues. Therefore, when these studies are reapplied, different and non-clustered populations are established, making unfeasible evidence generalization and contributing for an imbalance between research and practice. Aiming at investigating ways to overcome the absence of large sampling frames in Software Engineering studies, this short paper presents the results of an initial experience concerned with the systematic recruitment of subjects for a survey regarding software requirements effort factors by using social networks compared with recruitment by convenience. We have observed in this particular case that using social networks technology does not guarantee sample enlargement by just posting invitations in specific forums. However, its usage can contribute to increase the subjects' heterogeneity and to increase the level of confidence of the sample, which consequently improve our capacity of observing the object under study, with the probable strengthen of results.

Usability is one of the most important software qualities regarding its acceptability by end users. It is even more important in the context of Web applications. One way of evaluating applications usability is through inspections. The WDP (Web Design Perspectives-based Usability Evaluation) is a usability inspection technique presenting evidence of industrial use feasibility; however the need for some computerized support was suggested by its users. Therefore, the WDP Tool was built aiming to provide automated support to the application of the WDP technique, reducing the effort involved in usability inspections with WDP. This paper presents two observational studies regarding the use of the WDP Tool, which aimed to analyze the cost-effectiveness of its application and its appropriateness to the industrial environment through the Technology Acceptance Model (TAM). The results provide feasibility indication of using the WDP Tool to support usability inspections in real software development projects in industry.

Background: Some scientific fields, such as automobile, drugs discovery or engineer have used simulation-based studies (SBS) to faster the observation of phenomena and evolve knowledge. All of them organize their working structure to perform computerized experiments based on explicit research protocols and evidence. The benefits have been many and great advancements are continuously obtained for the society. However, could the same approach be observed in Software Engineering (SE)? Are there research protocols and evidence based models available in SE for supporting SBS? Are the studies reports good enough to support their understanding and replication? AIM: To characterize SBS in SE and organize a set of reporting guidelines aiming at improving SBS' understandability, replicability, generalization and validity. METHOD: To undertake a secondary study to characterize SBS. Besides, to assess the quality of reports to understand the usually reported information regarding SBS. RESULTS: From 108 selected papers, it has been observed several relevant initiatives regarding SBS in software engineering. However, most of the reports lack information concerned with the research protocol, simulation model building and evaluation, used data, among others. SBS results are usually specific, making their generalization and comparison hard. No reporting standard has been observed. CONCLUSIONS: Advancements can be observed in SBS in Software Engineering. However, the lack of reporting consistency can reduce understandability, replicability, generalization and compromise their validity. Therefore, an initial set of guidelines is proposed aiming at improving SBS report quality. Further evaluation must be accomplished to assess the guidelines feasibility when used to report SBS in Software Engineering.

Software evolution is recognized as one of the most challenging areas in the field of Software Engineering. The observation of evolution is time-dependent, reducing opportunities for actual observations in short periods of time. Usually, maintenance cycles are proportional to the software life cycle. Therefore, the amount of research has not been enough to deal with all the issues related to the evolution of software. However, simulation through confident models represents an interesting strategy to support software decay observation in short period of time. Towards that, this paper describes a model aimed at supporting the software decay simulation through systems dynamics. The Laws of Software Evolution and ISO 9126 were used as initial knowledge to support the discovery of software characteristic (size, periodicity, complexity, effort, reliability, and maintainability) relationships. Next, evidence to strengthen the existence of such relationships was acquired through quasi-systematic literature reviews. In sequence, the model was applied to support the simulation of industrial software decay. The results suggested its feasibility and correctness, making it an interesting candidate to support future software decay studies.

Objective: to investigate studies about software processes looking for practices which can be used to obtain agility in software processes. Method: A systematic review including seven search engines was executed in Feb/2010. To apply the defined criteria to select papers and extract information regarding working practices bringing agility to software processes. Results: from 6696 retrieved papers, 441 were selected to support the identification of 236 occurrences of 51 distinct practices associated with the concept of agility. Their descriptions were deeply analyzed and consolidated. After discarding those which appeared in the technical literature in a small amount of papers, 17 agile practices were identified. Conclusion: although further studies are necessary to evaluate the efficacy of these 17 agile practices, 12 of them have been more commonly approached in the software projects and could be primarily considered: test driven development, continuous integration, pair programming, planning game, onsite customer, collective code ownership, small releases, metaphor, refactoring, sustainable pace, simple design and coding standards.

The evolution of Science has been supported by complex computerized infrastructures with growing interest in simulation based experiments. This trend can also be observed in Software Engineering. Our capacity of acquiring evidences to describe phenomena of interest in the field allowed the building of in silico models that can virtually replicate feasible software behaviors and improve our capacity of observation. In silico experiments demand additional concerns for its planning. One of them is regarding the scientific workflow conception. This task is not easy and to apply ad-hoc approaches can risky the experiment execution, validity and future replications. Regarding this topic, this paper presents some initial results from our research towards an approach to support the conception of scientific workflows for in silico experiments in Software Engineering are presented.

The selection of software technologies represents a risk factor to a software project. Therefore, using tailored software technologies to support this task can contribute to reduce the risk of inadequate choices made by software engineers. This paper presents the results of an experimental study conducted to evaluate if three dependent variables (selection time, % of correct choices, and used information) can be significantly affected by two different approaches to support the selection of model-based testing techniques. This study consists of an external replication of an experiment conducted to evaluate two approaches for the selection of testing techniques. Therefore, we present the adaptations performed in the planning and design of the experiment and the challenges observed in conducting a study replication. The results indicated that the time spent to select MBT techniques and the percentage of right selections can be affected by the approach used to select MBT techniques for different software project categories.

This paper presents a literature survey of action research (AR) studies published in nine major Software Engineering (SE) journals and three conference proceedings in the period 1993 to June 2009. A strict selection based on distinguishing SE from Information Systems research has identified 16 papers. Although they represent a very small fraction of the studies being conducted in SE, such papers concern with different SE contexts allowing to get information about the increasing tendency in the AR use in software engineering. However, as shown by the initial results, SE researchers should invest more on rigor when defining, applying and reporting AR studies inSE.

Experimentation in software engineering represents a means to understand the impact of adopting software technologies. In this paper we describe an on-going experimentation work aiming at to address a real industry need in Brazil: the understanding of how performance variables (such as cost, productivity, and quality) are affected by the adoption of a nationwide software development reference model. The main strategy consists in integrating the application of carefully planned surveys into the model implementation and assessment procedures. Besides the expected performance results, we hope it can be useful to reinforce to the Brazilian software industry the importance of experimentation to support software technologies evaluation.

In this paper, we describe the planning, execution, and initial results analysis of a survey accomplished with Model Based Testing researchers and practitioners. This survey has two purposes: observing (1) which attributes could be adequate to characterize Model-based Testing Approaches and (2) what is the relevance of them when selecting MBT approaches for software projects. An initial list of attributes with their relevance has been identified. These results compose a body of knowledge regarding Model-based Testing Approaches intended to support software engineers in using MBT approaches in software projects.

Experimental replications are very important to the advancement of empirical software engineering. Replications are one of the key mechanisms to confirm previous experimental findings. They are also used to transfer experimental knowledge, to train people, and to expand a base of experimental evidence. Unfortunately, experimental replications are difficult endeavors. It is not easy to transfer experimental know-how and experimental findings. Based on our experience, this paper discusses this problem and proposes a Framework for Improving the Replication of Experiments (FIRE). The FIRE addresses knowledge sharing issues both at the intra-group (internal replications) and inter-group (external replications) levels. It encourages coordination of replications in order to facilitate knowledge transfer for lower cost, higher quality replications and more generalizable results.

Experimental studies have been used as a mechanism to acquire knowledge through a scientific approach based on measurement of phenomena in different areas. However it is hard to run such studies when they require models (simulation), produce large amount of information, and explore science in large scale. In this case, a computerized infrastructure is necessary and constitutes a complex system to be built. In this paper we discuss an experimentation environment that has being built to support large scale experimentation and scientific knowledge management in Software Engineering.

Given the growth in the number and size of Web Applications worldwide, Web quality assurance, and more specifically Web usability have become key success factors. Therefore, this work proposes a usability evaluation technique based on the combination of Web design perspectives adapted from existing literature, and heuristics. This new technique is assessed using a controlled experiment aimed at measuring the efficiency and effectiveness of our technique, in comparison to Nielsen's heuristic evaluation. Results indicated that our technique was significantly more effective than and as efficient as Nielsen's heuristic evaluation.

The objective of this paper is to determine under what circumstances individual organizations would be able to rely on cross-company-based estimation models. We performed a systematic review of studies that compared predictions from cross-company models with predictions from within-company models based on analysis of project data. Ten papers compared cross-company and within-company estimation models; however, only seven presented independent results. Of those seven, three found that cross-company models were not significantly different from within-company models, and four found that cross-company models were significantly worse than within-company models. Experimental procedures used by the studies differed making it impossible to undertake formal meta-analysis of the results. The main trend distinguishing study results was that studies with small within-company data sets (i.e., $20 projects) that used leave-one-out cross validation all found that the within-company model was significantly different (better) from the cross-company model. The results of this review are inconclusive. It is clear that some organizations would be ill-served by cross-company models whereas others would benefit. Further studies are needed, but they must be independent (i.e., based on different data bases or at least different single company data sets) and should address specific hypotheses concerning the conditions that would favor cross-company or within-company models. In addition, experimenters need to standardize their experimental procedures to enable formal meta-analysis, and recommendations are made in Section 3.

OBJECTIVE – The objective of this paper is to determine under what circumstances individual organisations would be able to rely on cross-company based estimation models. METHOD – We performed a systematic review of studies that compared predictions from cross-company models with predictions from within-company models based on analysis of project data. RESULTS – Ten papers compared cross-company and within-company estimation models, however, only seven of the papers presented independent results. Of those seven, three found that cross-company models were as good as within-company models, four found cross-company models were significantly worse than within-company models. Experimental procedures used by the studies differed making it impossible to undertake formal meta-analysis of the results. The main trend distinguishing study results was that studies with small single company data sets (i.e. <20 projects) that used leave-one-out cross-validation all found that the within-company model was significantly more accurate than the cross-company model. CONCLUSIONS – The results of this review are inconclusive. It is clear that some organisations would be ill-served by cross-company models whereas others would benefit. Further studies are needed, but they must be independent (i.e. based on different data bases or at least different single company data sets). In addition, experimenters need to standardise their experimental procedures to enable formal meta-analysis.

There is a growing interest in empirical study in software engineering, both for validating mature technologies and for guiding improvements of less-mature technologies. This paper introduces an empirical methodology, based on experiences garnered over more than two decades of work by the Empirical Software Engineering Group at the University of Maryland and related organizations, for taking a newly proposed improvement to development processes from the conceptual phase through transfer to industry. The methodology presents a series of questions that should be addressed, as well as the types of studies that best address those questions. The methodology is illustrated by a specific research program on inspection processes for Object-Oriented designs. Specific examples of the studies that were performed and how the methodology impacted the development of the inspection process are also described.

Develops a domain engineering approach and the supporting tools to define software product lines. The approach is composed of a process and a notation. The notation is an extension of UML (Unified Modeling Language), while the process is influenced by the Synthesis and FAST domain engineering methods. Supporting tools have been built on top of a commercial CASE tool for UML. This paper focuses on the domain analysis part of the approach and specifically on the notation and tools to support it. The approach is being applied to sub-domains of the guidance, navigation and control domain, which are a set of functionalities used on-board satellites developed by the NASA Goddard Space Flight Center.

Inspections can be used to identify defects in software artifacts. In this way, inspection methods help to improve software quality, especially when used early in software development. Inspections of software design may be especially crucial since design defects (problems of correctness and completeness with respect to the requirements, internal consistency, or other quality attributes) can directly affect the quality of, and effort required for, the implementation.

We suggest that empirical studies of maintenance are difficult to understand unless the context of the study is fully defined. We developed a preliminary ontology to identify a number of factors that influence maintenance. The purpose of the ontology is to identify factors that would affect the results of empirical studies. We present the ontology in the form of a UML model. Using the maintenance factors included in the ontology, we define two common maintenance scenarios and consider the industrial issues associated with them. Copyright © 1999 John Wiley & Sons, Ltd.