Scenario-based testing is an important approach for the development and validation of autonomous driving systems, as it enables evaluation across different driving situations. Safety-critical scenarios are especially relevant, but they occur rarely in real-world data, which creates the need for generation methods. In this paper, we present a scalable AI-based approach based on a variational autoencoder that unifies the generation of different types of critical scenarios while introducing controllability through a structured latent space. The integration of unified generation and latent space control advances AI-based scenario generation towards practical use, thereby supporting the requirements of industrial validation pipelines.

ABSTRACT Context The microservices architectural style has revolutionized the way modern software systems are developed and operated. While the development of new microservices systems can leverage a wide range of resources and proven strategies, the migration of an existing monolithic system is not easily generalizable. Software architects look for guidance and predictable results in this highly individual process, in particular for generating a targeted, quality‐oriented, and semi‐automated decomposition. Objective To systematically guide software architects and developers in modernizing their software systems, we propose a holistic and quality‐oriented methodology to transform monolithic applications into microservices. Our work aims to provide industry‐relevant methods that address the gap between academia and practice by facilitating the transfer of knowledge. Methods In an overarching design science research process, we developed a framework that we implemented as a web‐based application. As a preliminary work, we conducted two initial interview studies with 25 software professionals to collect evidence on the intentions, strategies, and challenges in a migration process. An essential groundwork of our framework design constitute 110 scientific publications on approaches for architectural refactoring and migration to microservices, which we reviewed over four iterations. In a multifaceted evaluation with 26 participants, we examined our methodology's capability of providing actionable guidance for practitioners. This evaluation was complemented by two longitudinal case studies in an industrial context. Results We provide a framework for transforming monolithic applications to microservices, along with a dedicated quality assurance concept that supports a quality‐driven migration process. The evaluations among 19 software professionals showed an overall positive result in terms of effectiveness, usefulness, and usability. Two industrial case studies confirmed these promising results. Among practitioners, we discerned a need for flexibility, ease of use, and holistic guidance in a migration process. In this regard, we see potential to evolve our concept using artificial intelligence techniques for even more precise recommendations in a human‐like conversational dialog.

This study evaluates the integration of AI-powered robots in early childhood education, focusing on their impact on emotional selfregulation, engagement, and collaborative skills.A ten-week experimental design involving two groups of children assessed the robot's effectiveness through progress assessments, parental surveys, and teacher feedback.Results demonstrated that early exposure to the robot significantly enhanced emotional recognition, while sustained interaction further improved collaborative and social engagement.Parental and teacher feedback highlighted high acceptance levels, emphasizing the robot's ease of integration and positive influence on classroom dynamics.This research underscores the transformative potential of AI and robotics in education.The findings advocate for the broader adoption of AI-powered interventions, carefully examining equitable access, ethical considerations, and sustainable implementation.This work sets a foundation for exploring longterm impacts and expanding applications of AI in inclusive and impactful educational settings.

System-Level Test (SLT) is important in semiconductor testing as it can detect defects missed by traditional methods. Test engineers use off-the-shelf software to manually compose test suites, often written in high-level languages such as $\mathrm{C} / \mathrm{C}++$ or Rust. Several methods for automatically generating test programs have been investigated, using assembly language. However, one could argue that the resulting test programs are not capturing all possible interactions in actual software. Large Language Models (LLMs) can generate code in high-level languages closer to actual software. In this work, we examine the limitations of LLMs and high-level languages for generating SLT programs. We run an experiment using genetic programming (GP) to find an assembly snippet with the highest power consumption. Then, we utilize LLMs to generate $\mathbf{C}$ code and demonstrate that the compiler, the enabled optimization level, and the LLM have a significant influence on the resulting power consumption. Furthermore, we show via decompilation that the snippet from the GP run has no direct equivalent in C. Finally, we demonstrate that the initial values have a significant impact on power consumption for both the GP-generated and the decompiled snippet.

The world of software development has fundamentally changed because of the explosive growth of opensource repositories in recent years. Open-source repositories have become a valuable tool for software developers and researchers because they are free and usually easy to use. Likewise, learning Artificial Intelligence (AI) and Machine Learning (ML) skills are in high demand, especially among software engineering students, as they increasingly require AI skills to drive innovation, solve complex problems, and remain competitive. There are several AI/ML open-source projects that contain code explanations, e.g., comments and/or documentation, making them potential educational tools. However, it is currently unclear how well AI novices can benefit from these resources. Hence, we studied how computer science bachelor students perceive self-study with open-source repositories to build more complex AI/ML systems to gauge the usefulness of these repositories. After a learning period, we surveyed the perception and learning outcomes from the viewpoint of 112 students. By analyzing the responses, we found that 75 % of the students stated that they could now build complex AI/ML systems if provided with enough documentation and descriptions and are motivated to work on them. While this indicates that learning or improving AI/ML skills via open-source repositories is promising, more research beyond self-reporting is needed.

This study investigates teamwork dynamics in student software development projects through a mixed-method approach combining quantitative analysis of GitLab commit logs and qualitative survey data. We analyzed individual contributions across six project phases, comparing self-reported and actual contributions to measure discrepancies. Additionally, a survey captured insights on team leadership, conflict resolution, communication practices, and workload perceptions. Findings reveal that teams with minimal contribution discrepancies achieved higher project grades and exam pass rates. In contrast, teams with more significant discrepancies experienced lower performance, potentially due to role clarity and communication issues. These results underscore the value of shared leadership, structured conflict resolution, and regular feedback in fostering effective teamwork, offering educators strategies to enhance collaboration in software engineering education through self-reflection and balanced workload allocation.

Autonomous driving systems (ADS) shall safely navigate diverse scenarios encountered in an open world. Prior to deployment, it is crucial to evaluate their ability to handle these situations. A popular approach for ADS verification is scenario-based testing, which simulates relevant critical scenarios in a virtual environment. However, evaluating these behaviors can be challenging, as traditional scenario-based testing often relies on metrics that may overlook subtle risks without immediate critical impact that still pose long-term safety concerns. System-theoretic process analysis (STPA) is a hazard analysis technique suitable for the analysis of modern complex systems. In this work, we employ STPA to derive system-level evaluation criteria for scenario-based testing frameworks designed to be universally applicable in any sim-ulation run. Their utilization provides a data-driven approach to safety assurance, helping to uncover unknown safety issues that, if unaddressed, could compromise the long-term reliability of ADS. Furthermore, the approach is demonstrated on a construction zone navigation extension for ADS, illustrating its potential to provide continuous, data-driven feedback from testing to safety assurance. The method shall contribute to the early detection of arising safety issues and ultimately lead to the development of safer systems.

Abstract Explainable artificial intelligence (XAI) is a field of study that focuses on the development process of AI-based systems while making their decision-making processes understandable and transparent for users. Research already identified explainability as an emerging requirement for AI-based systems that use machine learning (ML) techniques. However, there is a notable absence of studies investigating how ML practitioners perceive the concept of explainability, the challenges they encounter, and the potential trade-offs with other quality attributes. In this study, we want to discover how practitioners define explainability for AI-based systems and what challenges they encounter in making them explainable. Furthermore, we explore how explainability interacts with other quality attributes. To this end, we conducted semi-structured interviews with 14 ML practitioners from 11 companies. Our study reveals diverse viewpoints on explainability and applied practices. Results suggest that the importance of explainability lies in enhancing transparency, refining models, and mitigating bias. Methods like SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanation (LIME) are frequently used by ML practitioners to understand how models work, while tailored approaches are typically adopted to meet the specific requirements of stakeholders. Moreover, we have discerned emerging challenges in eight categories. Issues such as effective communication with non-technical stakeholders and the absence of standardized approaches are frequently stated as recurring hurdles. We contextualize these findings in terms of requirements engineering and conclude that industry currently lacks a standardized framework to address arising explainability needs.

Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. • International survey gathering insights from 188 practitioners across 25 countries. • Overview of current practices and challenges in engineering ML-enabled systems. • Inferential quantitative analysis reporting the status quo with confidence intervals. • Qualitative analysis mapping ML life cycle challenges and causes of project failure.

The rising complexity of integrated devices has led to new defect types and failure modes at the system level that are not detected by structural tests. System-Level Test (SLT) is another test step to combat this challenge. SLT is in charge of exercising system-level interactions between hardware components and software. Non-functional properties, e.g., temperature, play a major role in SLT.This work focuses on the automatic generation of assembly test programs for SLT that aim to indirectly maximize a particular non-functional property, for example, the temperature. It is based on two-step generation with genetic algorithms. First, a fast architectural simulation is used with the genetic algorithm to provide a structure for the test programs. Afterward, an additional generation is done on the hardware to optimize the initial register contents of the program.The case study for gathering experimental results is a super-scalar out-of-order RISC-V processor, the Berkeley Out-of-Order Machine (BOOM). Experimental results show that the two-step generation is more effective in converging to a better power-hungry test program than only using the power consumption as a fitness function for the genetic algorithm.

System-Level Test (SLT) has been an integral part of integrated circuit test flows for over a decade and continues to be significant. Nevertheless, there is a lack of systematic approaches for generating test programs, specifically focusing on the non-functional aspects of the Device under Test (DUT). Currently, test engineers manually create test suites using commercially available software to simulate the end-user environment of the DUT. This process is challenging and laborious and does not assure adequate control over non-functional properties. This paper proposes to use Large Language Models (LLMs) for SLT program generation. We use a pre-trained LLM and fine-tune it to generate test programs that optimize non-functional properties of the DUT, e.g., instructions per cycle. Therefore, we use Gem5, a microarchitectural simulator, in conjunction with Reinforcement Learning-based training. Finally, we write a prompt to generate C code snippets that maximize the instructions per cycle of the given architecture. In addition, we apply hyperparameter optimization to achieve the best possible results in inference.

Background: The "Technical Debt Dataset" (TDD) is a comprehensive dataset on technical debt (TD) in the main branches of more than 30 Java projects. However, some TD items produced by Sonar-Qube are not included for many commits, for instance because the commits failed to compile. This has limited previous studies using the dataset. Aims and Method: In this paper, we provide an addition to the dataset that includes an analysis of 278,320 commits of all branches in a superset of 37 projects using Teamscale. We then demonstrate the utility of the dataset by exploring the relationship between developer personality by replicating a prior study. Results: The new dataset allows us to use a larger sample than prior work could, and we analyze the personality of 111 developers and 5,497 of their commits. The relationships we find between developer personality and the introduction and removal of TD differ from those found in prior work. Conclusions: We offer a dataset that may enable future studies into the topic of TD and we provide additional insights on how developer personality relates to TD.

Abstract A Software Product Line (SPL) is a software development paradigm in which a family of software products shares a set of core assets. Testing has a vital role in both single-system development and SPL development in identifying potential faults by examining the behavior of a product or products, but it is especially challenging in SPL. There have been many research contributions in the SPL testing field; therefore, assessing the current state of research and practice is necessary to understand the progress in testing practices and to identify the gap between required techniques and existing approaches. This paper aims to survey existing research on SPL testing to provide researchers and practitioners with up-to-date evidence and issues that enable further development of the field. To this end, we conducted a Systematic Literature Review (SLR) with seven research questions in which we identified and analyzed 118 studies dating from 2003 to 2022. The results indicate that the literature proposes many techniques for specific aspects (e.g., controlling cost/effort in SPL testing); however, other elements (e.g., regression testing and non-functional testing) still need to be covered by existing research. Furthermore, most approaches are evaluated by only one empirical method, most of which are academic evaluations. This may jeopardize the adoption of approaches in industry. The results of this study can help identify gaps in SPL testing since specific points of SPL Engineering still need to be addressed entirely.

Abstract Artificial intelligence (AI) permeates all fields of life, which resulted in new challenges in requirements engineering for artificial intelligence (RE4AI), e.g., the difficulty in specifying and validating requirements for AI or considering new quality requirements due to emerging ethical implications. It is currently unclear if existing RE methods are sufficient or if new ones are needed to address these challenges. Therefore, our goal is to provide a comprehensive overview of RE4AI to researchers and practitioners. What has been achieved so far, i.e., what practices are available, and what research gaps and challenges still need to be addressed? To achieve this, we conducted a systematic mapping study combining query string search and extensive snowballing. The extracted data was aggregated, and results were synthesized using thematic analysis. Our selection process led to the inclusion of 126 primary studies. Existing RE4AI research focuses mainly on requirements analysis and elicitation, with most practices applied in these areas. Furthermore, we identified requirements specification, explainability, and the gap between machine learning engineers and end-users as the most prevalent challenges, along with a few others. Additionally, we proposed seven potential research directions to address these challenges. Practitioners can use our results to identify and select suitable RE methods for working on their AI-based systems, while researchers can build on the identified gaps and research directions to push the field forward.

Studies in empirical software engineering are often most useful if they make causal claims because this allows practitioners to identify how they can purposefully influence (rather than only predict) outcomes of interest. Unfortunately, many non-experimental studies suffer from potential endogeneity, for example, through omitted confounding variables, which precludes claims of causality. In this conceptual tutorial, we aim to transfer the proven solution of instrumental variables and two-stage models as a means to account for endogeneity from econometrics to the field of empirical software engineering. To this end, we discuss causality and causal inference, provide a definition of endogeneity, explain its causes, and lay out the conceptual idea behind instrumental variable approaches and two-stage models. We also provide an extensive illustration with simulated data and a brief illustration with real data to demonstrate the approach, offering Stata and R code to allow researchers to replicate our analyses and apply the techniques to their own research projects. We close with concrete recommendations and a guide for researchers on how to deal with endogeneity.

With the continued success of the open RISC-V architecture, practical deployment of RISC-V processors necessitates an in-depth consideration of their testability, safety and security aspects. This survey provides an overview of recent developments in this quickly-evolving field. We start with discussing the application of state-of-the-art functional and system-level test solutions to RISC-V processors. Then, we discuss the use of RISC-V processors for safety-related applications; to this end, we outline the essential techniques necessary to obtain safety both in the functional and in the timing domain and review recent processor designs with safety features. Finally, we survey the different aspects of security with respect to RISC-V implementations and discuss the relationship between cryptographic protocols and primitives on the one hand and the RISC-V processor architecture and hardware implementation on the other. We also comment on the role of a RISC-V processor for system security and its resilience against side-channel attacks.

System-Level Test (SLT) emerged as an additional test step to detect manufacturing defects not caught by traditional testing. For SLT, the Device Under Test (DUT) is embedded into an environment that emulates the end-user application as closely as possible and runs workloads composed of existing off-the-shelf software. We present an automatic greybox SLT program generation method to find code snippets that control the DUT’s extra-functional properties, to achieve better characterization, or to improve the coverage of emerging defect types. In contrast to ATPG or formal methods, our method does not require structural information and relies solely on simulation results or hardware measurements to guide the generation. We show that our method outperforms hand-crafted snippets on a RISC-V super-scalar processor and look into possible reasons why the snippets perform the way they do.

Searching for clues, gathering evidence, and reviewing case files are all techniques used by criminal investigators to draw sound conclusions and avoid wrongful convictions. Medicine, too, has a long tradition of evidence-based practice, in which administering a treatment without evidence of its efficacy is considered malpractice. Similarly, in software engineering (SE) research, we can develop sound methodologies and mitigate threats to validity by basing study design decisions on evidence. Echoing a recent call for the empirical evaluation of design decisions in program comprehension experiments, we conducted a 2-phases study consisting of systematic literature searches, snowballing, and thematic synthesis. We found out (1) which validity threat categories are most often discussed in primary studies of code comprehension, and we collected evidence to build (2) the evidence profiles for the three most commonly reported threats to validity. We discovered that few mentions of validity threats in primary studies (31 of 409) included a reference to supporting evidence. For the three most commonly mentioned threats, namely the influence of programming experience, program length, and the selected comprehension measures, almost all cited studies (17 of 18) did not meet our criteria for evidence. We show that for many threats to validity that are currently assumed to be influential across all studies, their actual impact may depend on the design and context of each specific study. Researchers should discuss threats to validity within the context of their particular study and support their discussions with evidence. The present paper can be one resource for evidence, and we call for more meta-studies of this type to be conducted, which will then inform design decisions in primary studies. Further, although we have applied our methodology in the context of program comprehension, our approach can also be used in other SE research areas to enable evidence-based experiment design decisions and meaningful discussions of threats to validity.

Background: Technical debt (TD) has been widely discussed in software engineering research, and there is an emerging literature linking it to developer characteristics. However, developer personality has not yet been studied in this context. Aims and Method: We explore the relationship between various personality traits (Five Factor Model, regulatory focus, and narcissism) of developers and the introduction and removal of TD. To this end, we complement an existing TD dataset with novel self-report personality data gathered by surveying developers, and analyze 2,145 commits from 19 developers. Results: We find that conscientiousness, emotional stability, openness to experience, and prevention focus are negatively associated with TD. There were no significant results for extraversion, agreeableness, promotion focus, or narcissism. Conclusions: We take our results as first evidence that developer personality has a systematic influence on the introduction and removal of TD. This has implications not only for future research, which could, for example, study the effects of personality on downstream consequences of TD like defects, but also for software engineering practitioners who may, for example, consider developer personality in staffing decisions.

Abstract The domain of cyber‐physical systems (CPS) has recently seen strong growth, for example, due to the rise of the Internet of Things (IoT) in industrial domains, commonly referred to as “Industry 4.0.” However, CPS challenges like the strong hardware focus can impact modern software development practices, especially in the context of modernizing legacy systems. While microservices and DevOps have been widely studied for enterprise applications, there is insufficient coverage for the CPS domain. Our goal is therefore to analyze the peculiarities of such systems regarding challenges and practices for using and migrating towards microservices and DevOps. We conducted a rapid review based on 146 scientific papers, and subsequently validated our findings in an interview‐based case study with nine CPS professionals in different business units at Siemens AG. The combined results picture the specifics of microservices and DevOps in the CPS domain. While several differences were revealed that may require adapted methods, many challenges and practices are shared with typical enterprise applications. Our study supports CPS researchers and practitioners with a summary of challenges, practices to address them, and research opportunities.

Test methods that can keep up with the ongoing increase in complexity of semiconductor products and their underlying technologies are an essential prerequisite for maintaining quality and safety of our daily lives and for continued success of our economies and societies. There is a huge potential how test methods can benefit from recent breakthroughs in domains such as artificial intelligence, data analytics, virtual/augmented reality, and security. The Graduate School on “Intelligent Methods for Semiconductor Test and Reliability” (GS-IMTR) at the University of Stuttgart is a large-scale, radically interdisciplinary effort to address the scientific-technological challenges in this domain. It is funded by Advantest, one of the world leaders in automatic test equipment. In this paper, we describe the overall philosophy of the Graduate School and the specific scientific questions targeted by its ten projects.

Communication is essential in software development, and even more in distributed settings. Communication activities need to be organized and coordinated to defend against the threat of productivity losses, increases in cognitive load, and stress among team members. With a plethora of communication channels that were identified by previous research in open-source projects, there is a need to explore organizational issues in how these communication channels are introduced, explained, and motivated for use among all project members. In this study, we wanted to understand which communication channels are used in GitHub projects and how they are presented to the GitHub project audience. We employed thematic analysis to analyze 151 artifacts in 90 GitHub projects. Our results revealed 32 unique communications channels that can be divided into nine different types. Projects mostly provide channels of different types, but for some types (e.g., chat) it is common to provide several channels. Maintainers are aware that channels have different properties and help the developers to decide which channel should be used in which case. However, this is not true for all projects, and often we have not found any explicit reasons why maintainers chose to provide one channel over another. Different channels can be used for different purposes and have different affordances, so maintainers have to decide wisely which channels they want to provide and make clear which channel should be used in which case. Otherwise, developers might feel overwhelmed of too many channels and information can get fragmented over multiple channels [31].

Context: Citations are a key measure of scientific performance in most fields, including software engineering. However, there is limited research that studies which characteristics of articles’ metadata (title, abstract, keywords, and author list) are driving citations in this field. Objective: In this study, we propose a simple theoretical model for how citations come to be with respect to article metadata, we hypothesize theoretical linkages between metadata characteristics and citations of articles, and we empirically test these hypotheses. Method: We use multiple regression analyses to examine a data set comprising the titles, abstracts, keywords, and authors of 16,131 software engineering articles published between 1990 and 2020 in 20 highly influential software engineering venues. Results: We find that number of authors, number of keywords, number of question marks and dividers in the title, number of acronyms, abstract length, abstract propositional idea density, and corresponding authors in the core Anglosphere are significantly related to citations. Conclusion: Various characteristics of articles’ metadata are linked to the frequency with which the corresponding articles are cited. These results partially confirm and partially go counter to prior findings in software engineering and other disciplines.

Abstract In regulated domains like automotive, release planning is a complex process. This complex process consists of an agreement between product development processes for hardware as well as mechanic systems and approaches for software development. Particularly in automotive, the creation and synchronization of release plans for hardware as well as software is a challenge. Within the whole complex system development, it is challenging to consider the relevant stakeholders in the initial creation of a release plan. Depending on the context that a release plan shall be created for, there are different stakeholders that have to be considered from the beginning. There are numerous publications in the area of release planning, but there is no detailed research that shows which stakeholders have to be addressed in the automotive context. The aim of this work is to identify stakeholders of a release plan as an appropriate approach to create transparency in release planning in the automotive domain. Action research to elaborate relevant stakeholders for release planning was conducted at Dr. Ing. h. c. F. Porsche AG. We present a detailed overview of identified stakeholders as well as their required content and added value regarding two pilot projects. With this contribution, identified stakeholders of release planning from the hardware and software points of view are introduced. We discuss, based on the results, why there are common stakeholders for the two projects and why there are individual stakeholders for each project. With this work, we present a more complete stakeholder identification and a more detailed understanding of their needs.

Abstract Mutation testing is a technique that changes code instructions to assess the quality of automated software tests. Industry has not broadly adopted the technique because execution and analysis times are too long and not considered worth the effort. To change this, a variation called “extreme mutation testing” emerged, which mutates whole methods instead of instructions. The extreme variant trades accuracy for speed gains and also provides pre‐analyzed results. In this study, we aim to analyze both techniques on their granularity levels, look for benefits when combining them, and find motivations when a developer considers killing mutants. For that, we conducted a case study in a company from the semiconductor industry. We mutated a large Java software project which is tested by more than 11,000 unit tests, analyzed the results, manually inspected more than 1000 mutants, and conducted a focus group with five developers of the software. Among other results, we provide the distribution of traditional across extreme mutants as well as qualitative coding results of our mutant inspection and focus group transcript. We conclude that the traditional approach can be similarly strategically applied as the extreme one and that motivations of developers to target mutants are mostly not code related.

Abstract Observing and controlling the dependability of service provision of complex IoT systems is challenging. In practice, many organizations struggle to derive consumer needs related to quality and to observe and quantify the service provision in the context of the dynamic behavior of a complex distributed system. In this paper, we present an approach to define and evaluate the dependability of complex IoT systems. Our approach is an adaptation of the ISO/IEC 25040, an international standard for the evaluation process for system and software quality, which is part of the systems and software quality requirements and evaluation (SQuaRE) series. Our approach was designed and evaluated with action research in an industrial study at Robert Bosch GmbH. Based on the framework of the SQuaRE series, we integrated different elements of site reliability engineering (SRE) and combined them with distributed tracing as a promising measurement method. Our approach introduces the IoT transaction concept to reduce modeling and observation efforts while increasing operationalization to measure performance against dependability targets. Our adaption was effectively applied, consumer-centricity along different system stakeholders were enhanced, and negative consequences of organizational silos were reduced. This has improved the dependability evaluation of service provision to enable fast feedback cycles for service performance control and improvement.

Abstract In component‐based software development, integration testing is a crucial step in verifying the composite behaviour of a system. However, very few formally or empirically validated approaches are available for systematically testing if components have been successfully integrated. In practice, integration testing of component‐based systems is usually performed in a time‐ and resource‐limited context, which further increases the demand for effective test selection strategies. In this work, we therefore analyse the relationship between different component and interface coupling measures found in literature and the distribution of failures found during integration testing of an automotive system. By investigating the correlation for each measure at two architectural levels, we discuss its usefulness to guide integration testing at the software component level as well as for the hardware component level where coupling is measured among multiple electronic control units (ECUs) of a vehicle. Our results indicate that there is a positive correlation between coupling measures and failure‐proneness at both architectural level for all tested measures. However, at the hardware component level, all measures achieved a significantly higher correlation when compared to the software‐level correlation. Consequently, we conclude that prioritizing testing of highly coupled components and interfaces is a valid approach for systematic integration testing, as coupling proved to be a valid indicator for failure‐proneness.

A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments need evaluation. In this article, we provide an introduction to psychometric theory for the evaluation of measurement instruments for SE researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature framed by the Standards for Educational and Psychological Testing. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We provide an openly available example of a psychometric evaluation based on our guideline. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology. To improve the quality of behavioral research in SE, studies focusing on introducing, validating, and then using psychometric instruments need to be more common.

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.

Program comprehension is a cognitive psychological process. Accordingly, literature and intuition suggest that a developer's intelligence and personality have an impact on their performance in comprehending source code. Some researchers have made this suggestion in the past when discussing threats to validity of their study results. However, the lack of studies investigating the relationship of intelligence and personality to performance in code comprehension makes scientifically sound reasoning about their influence difficult. We conduct the first large-scale empirical evaluation, a correlational study with undergraduates, to investigate the correlation of intelligence and personality with performance in code comprehension, that is, in this context, with correctness in answering comprehension questions on code snippets. We found that personality traits are unlikely to impact code comprehension performance, at least not when their influence is considered in isolation. Conscientiousness, in combination with other factors, however, explains some of the variance in code comprehension performance. For intelligence, significant small to moderate positive effects on code comprehension performance were found for three of four factors measured, i.e., fluid intelligence, visual perception, and cognitive speed. Crystallized intelligence has a positive but statistically insignificant effect on code comprehension performance. According to our results, several intelligence facets as well as the personality trait conscientiousness are potential confounders that should not be neglected in code comprehension studies of individual performance and should be controlled for via an appropriate study design. We call for the conduct of further studies on the relationship between intelligence and personality with code comprehension, in part because code comprehension involves more facets than we can measure in a single study and because our regression model explains only a small portion of the variance in code comprehension performance.

Mutation testing is used to evaluate the effectiveness of test suites. In recent years, a promising variation called extreme mutation testing emerged that is computationally less expensive. It identifies methods where their functionality can be entirely removed, and the test suite would not notice it, despite having coverage. These methods are called pseudo-tested. In this paper, we compare the execution and analysis times for traditional and extreme mutation testing and discuss what they mean in practice. We look at how extreme mutation testing impacts current software development practices and discuss open challenges that need to be addressed to foster industry adoption. For that, we conducted an industrial case study consisting of running traditional and extreme mutation testing in a large software project from the semiconductor industry that is covered by a test suite of more than 11,000 unit tests. In addition to that, we did a qualitative analysis of 25 pseudo-tested methods and interviewed two experienced developers to see how they write unit tests and gathered opinions on how useful the findings of extreme mutation testing are. Our results include execution times, scores, numbers of executed tests and mutators, reasons why methods are pseudo-tested, and an interview summary. We conclude that the shorter execution and analysis times are well noticeable in practice and show that extreme mutation testing supplements writing unit tests in conjunction with code coverage tools. We propose that pseudo-tested code should be highlighted in code coverage reports and that extreme mutation testing should be performed when writing unit tests rather than in a decoupled session. Future research should investigate how to perform extreme mutation testing while writing unit tests such that the results are available fast enough but still meaningful.

RISC-V has emerged as a viable solution on academia and industry. However, to use open source hardware for safety-critical applications, we need a deep understanding of the way in which well established mechanisms for testing and reliability could be integrated and deployed on the RISC-V ecosystem, and we need a clear knowledge on how such an ecosystem can be leveraged to improve security. This paper includes four contributions presenting the potential of RISC-V in security research, the way in which RISC-V can be hardened against power analysis attacks, how to implement, using RISC-V, software and hardware/software solutions for dual core lock step, and how to perform system-level testing in the RISC-V ecosystem.

Static code analysis tools and integrated development environments present developers with quality-related software metrics, some of which describe the understandability of source code. Software metrics influence overarching strategic decisions that impact the future of companies and the prioritization of everyday software development tasks. Several software metrics, however, lack in validation: we just choose to trust that they reflect what they are supposed to measure. Some of them were even shown to not measure the quality aspects they intend to measure. Yet, they influence us through biases in our cognitive-driven actions. In particular, they might anchor us in our decisions. Whether the anchoring effect exists with software metrics has not been studied yet. We conducted a randomized and double-blind experiment to investigate the extent to which a displayed metric value for source code comprehensibility anchors developers in their subjective rating of source code comprehensibility, whether performance is affected by the anchoring effect when working on comprehension tasks, and which individual characteristics might play a role in the anchoring effect. We found that the displayed value of a comprehensibility metric has a significant and large anchoring effect on a developer's code comprehensibility rating. The effect does not seem to affect the time or correctness when working on comprehension questions related to the code snippets under study. Since the anchoring effect is one of the most robust cognitive biases, and we have limited understanding of the consequences of the demonstrated manipulation of developers by non-validated metrics, we call for an increased awareness of the responsibility in code quality reporting and for corresponding tools to be based on scientific evidence.

In the age of autonomously driving vehicles, functionality and complexity of embedded systems are increasing tremendously. Safety aspects become more important and require such systems to operate with the highest possible level of fault tolerance. Simulation and systematic testing techniques often reach their limits in this regard. Here, formal verification as a long established technique can be an appropriate complement. However, the necessary preparatory work like adequately modeling a system and specifying properties in temporal logic are anything but trivial. In this paper, we report on our experiences applying model checking to verify the arbitration logic of a Vehicle Control System. We balance pros and cons of different model checking techniques and tools, and reason about our choice of the symbolic model checker NuSMV. We describe the process of modeling the architecture, resulting in ~1500 LOC, 69 state variables and 38 LTL constraints. To handle this large-scale model, we automate and optimize the model checking procedure for use on multi-core CPUs and employ Bounded Model Checking to avoid the state explosion problem. We share our lessons learned and provide valuable insights for architects, developers, and test engineers involved in this highly present topic.

Abstract Context Microservices as a lightweight and decentralized architectural style with fine-grained services promise several beneficial characteristics for sustainable long-term software evolution. Success stories from early adopters like Netflix, Amazon, or Spotify have demonstrated that it is possible to achieve a high degree of flexibility and evolvability with these systems. However, the described advantageous characteristics offer no concrete guidance and little is known about evolvability assurance processes for microservices in industry as well as challenges in this area. Insights into the current state of practice are a very important prerequisite for relevant research in this field. Objective We therefore wanted to explore how practitioners structure the evolvability assurance processes for microservices, what tools, metrics, and patterns they use, and what challenges they perceive for the evolvability of their systems. Method We first conducted 17 semi-structured interviews and discussed 14 different microservice-based systems and their assurance processes with software professionals from 10 companies. Afterwards, we performed a systematic grey literature review (GLR) and used the created interview coding system to analyze 295 practitioner online resources. Results The combined analysis revealed the importance of finding a sensible balance between decentralization and standardization. Guidelines like architectural principles were seen as valuable to ensure a base consistency for evolvability and specialized test automation was a prevalent theme. Source code quality was the primary target for the usage of tools and metrics for our interview participants, while testing tools and productivity metrics were the focus of our GLR resources. In both studies, practitioners did not mention architectural or service-oriented tools and metrics, even though the most crucial challenges like Service Cutting or Microservices Integration were of an architectural nature. Conclusions Practitioners relied on guidelines, standardization, or patterns like Event-Driven Messaging to partially address some reported evolvability challenges. However, specialized techniques, tools, and metrics are needed to support industry with the continuous evaluation of service granularity and dependencies. Future microservices research in the areas of maintenance, evolution, and technical debt should take our findings and the reported industry sentiments into account.

Abstract An increasing number of companies aim to enable their development teams to work in an agile manner. When introducing agile teams, companies face several challenges. This paper explores the kind of leadership needed to support teams to work in an agile way. One theoretical agile leadership concept describes a Scrum Master who is supposed to empower the team to lead itself. Empirical findings on such a leadership role are controversial. We still have not understood how leadership unfolds in a team that is by definition self-organizing. Further exploration is needed to better understand leadership in agile teams. Our goal is to explore how leadership changes while the team matures using the example of the Scrum Master. Through a grounded theory study containing 75 practitioners from 11 divisions at the Robert Bosch GmbH we identified a set of nine leadership roles that are transferred from the Scrum Master to the Development Team while it matures. We uncovered that a leadership gap and a supportive internal team climate are enablers of the role transfer process, whereas role conflicts may diminish the role transfer. To make the Scrum Master change in a mature team, team members need to receive trust and freedom to take on a leadership role which was previously filled by the Scrum Master. We conclude with practical implications for managers, Product Owners, Development Teams and Scrum Masters which they can apply in real settings.

System-level test, or SLT, is an increasingly important process step in today's integrated circuit testing flows. Broadly speaking, SLT aims at executing functional workloads in operational modes. In this paper, we consolidate available knowledge about what SLT is precisely and why it is used despite its considerable costs and complexities. We discuss the types or failures covered by SLT, and outline approaches to quality assessment, test generation and root-cause diagnosis in the context of SLT. Observing that the theoretical understanding for all these questions has not yet reached the level of maturity of the more conventional structural and functional test methods, we outline new and promising directions for methodical developments leveraging on recent findings from software engineering.

Scenario-based analysis is a comprehensive technique to evaluate software quality and can provide more detailed insights than e.g. maintainability metrics. Since such methods typically require significant manual effort, we designed a lightweight scenario-based evolvability evaluation method. To increase efficiency and to limit assumptions, the method exclusively targets service- and microservice-based systems. Additionally, we implemented web-based tool support for each step. Method and tool were also evaluated with a survey (N=40) that focused on change effort estimation techniques and hands-on interviews (N=7) that focused on usability. Based on the evaluation results, we improved method and tool support further. To increase reuse and transparency, the web-based application as well as all survey and interview artifacts are publicly available on GitHub. In its current state, the tool-supported method is ready for first industry case studies.

Background: Developers spend a lot of their time on understanding source code. Static code analysis tools can draw attention to code that is difficult for developers to understand. However, most of the findings are based on non-validated metrics, which can lead to confusion and code that is hard to understand not being identified.

The use of autonomous bots for automatic support in software development tasks is increasing. In the past, however, they were not always perceived positively and sometimes experienced a negative bias compared to their human counterparts. We conducted a qualitative study in which we deployed an autonomous refactoring bot for 41 days in a student software development project. In between and at the end, we conducted semi-structured interviews to find out how developers perceive the bot and whether they are more or less critical when reviewing the contributions of a bot compared to human contributions. Our findings show that the bot was perceived as a useful and unobtrusive contributor, and developers were no more critical of it than they were about their human colleagues, but only a few team members felt responsible for the bot.

In practice there are deficiencies in precise interteam communications about system anomalies to perform troubleshooting and postmortem analysis along different teams operating complex IoT systems. We evaluate the quality in use of an adaptation of IEEE Std. 1044-2009 with the objective to differentiate the handling of fault detection and fault reaction from handling of defect and its options for defect correction. We extended the scope of IEEE Std. 1044-2009 from anomalies related to software only to anomalies related to complex IoT systems. To evaluate the quality in use of our classification a study was conducted at Robert Bosch GmbH. We applied our adaptation to a postmortem analysis of an IoT solution and evaluated the quality in use by conducting interviews with three stakeholders. Our adaptation was effectively applied and interteam communications as well as iterative and inductive learning for product improvement were enhanced.

A number of qualitative studies find that team leadership is one essential success factor for evolving into a mature agile team. One such qualitative study suggests the 9-Factor Theory of Scrum Master roles, which claims that the Scrum Master performs a set of 9 leadership roles which are transferred to the team over time [ 14 ]. We aimed at conducting a quantitative exploration that examines the presence and change of the 9-Factor Theory in relation to team maturity. We conducted an online survey with 67 individuals at the conglomerate Robert Bosch GmbH. Descriptive statistics reveal that the Scrum Master and the agile team score differently on the 9 factors and that the Scrum Master role is most often distributed in teams that had been working between 3 and 5 months in an agile manner. Yet, we also find that the leadership roles predominantly remain with one dedicated Scrum Master. Based on our results we suggest to group the 9-Factor Theory into three clusters: the Scrum Master is rather linked to psychological team factors (1), while the team tends to be linked to rather product-related factors (2). Organizational factors (3) are less often present. Our practical implications suggest an extension of the Scrum Master description. Furthermore, our study lays groundwork for future quantitative testing of leadership in agile teams.

Group student software projects are important in computer science education. Students are encouraged to self-organize and learn technical skills, preparing them for real life software development. However, the projects contribute to multiple learning objectives, making coaching students a time consuming task. Thus, it is important to have a suitable best practice development process. For providing better insights for the students, the resulting software has to be of value and meet quality requirements, including maintainability, as in real life software development. Using source code quality metrics and by data mining repository data like commit history, we analyze six student projects, measuring their quality and identifying contributing factors to success or failure of a student project. Based on the findings, we formulate recommendations to improve future projects for students and researchers alike.

Mutation testing is a means to assess the effectiveness of a test suite and its outcome is considered more meaningful than code coverage metrics. However, despite several optimizations, mutation testing requires a significant computational effort and has not been widely adopted in industry. Therefore, we study in this paper whether test effectiveness can be approximated using a more light-weight approach. We hypothesize that a test case is more likely to detect faults in methods that are close to the test case on the call stack than in methods that the test case accesses indirectly through many other methods. Based on this hypothesis, we propose the minimal stack distance between test case and method as a new test measure, which expresses how close any test case comes to a given method, and study its correlation with test effectiveness. We conducted an empirical study with 21 open-source projects, which comprise in total 1.8 million LOC, and show that a correlation exists between stack distance and test effectiveness. The correlation reaches a strength up to 0.58. We further show that a classifier using the minimal stack distance along with additional easily computable measures can predict the mutation testing result of a method with 92.9% precision and 93.4% recall. Hence, such a classifier can be taken into consideration as a light-weight alternative to mutation testing or as a preceding, less costly step to that.

Context: The use of agile software development is increasing, even in regulated domains like the automotive domain.At the same time, traditional sequential processes are still in use.Collaboration between agile and hybrid projects within these complex traditional product development processes is difficult.Especially the creation and synchronization of a qualification phase plan is challenging.Objective: The aim of this study is to provide insights into the state of the practice to understand challenges related to the combined use of agile and traditional paradigms in release planning in the automotive domain.Method: Based on semi-structured interviews, an online survey with 39 respondents was conducted at Dr. Ing. h.c.F. Porsche AG.Results: We present the challenges identified in release planning, such as lack of transparency regarding the status quo of related projects.Furthermore, we motivate how agile development methods could improve collaboration between projects in release planning.Conclusions: There are many challenges in the context of co-existing agile and traditional projects.We discuss how agile practices like daily standup or continuous integration could address the identified challenges.

Advances in artificial intelligence have renewed interest in conversational agents. So-called chatbots have reached maturity for industrial applications. German insurance companies are interested in improving their customer service and digitizing their business processes. In this work we investigate the potential use of conversational agents in insurance companies by determining which classes of agents are of interest to insurance companies, finding relevant use cases and requirements, and developing a prototype for an exemplary insurance scenario. Based on this approach, we derive key findings for conversational agent implementation in insurance companies.

While Microservices promise several beneficial characteristics for sustainable long-term software evolution, little empirical research covers what concrete activities industry applies for the evolvability assurance of Microservices and how technical debt is handled in such systems. Since insights into the current state of practice are very important for researchers, we performed a qualitative interview study to explore applied evolvability assurance processes, the usage of tools, metrics, and patterns, as well as participants' reflections on the topic. In 17 semi-structured interviews, we discussed 14 different Microservice-based systems with software professionals from 10 companies and how the sustainable evolution of these systems was ensured. Interview transcripts were analyzed with a detailed coding system and the constant comparison method. We found that especially systems for external customers relied on central governance for the assurance. Participants saw guidelines like architectural principles as important to ensure a base consistency for evolvability. Interviewees also valued manual activities like code review, even though automation and tool support was described as very important. Source code quality was the primary target for the usage of tools and metrics. Despite most reported issues being related to Architectural Technical Debt (ATD), our participants did not apply any architectural or service-oriented tools and metrics. While participants generally saw their Microservices as evolvable, service cutting and finding an appropriate service granularity with low coupling and high cohesion were reported as challenging. Future Microservices research in the areas of evolution and technical debt should take these findings and industry sentiments into account.

To remain competitive in a fast changing environment, many companies started to migrate their legacy applications towards a Microservices architecture. Such extensive migration processes require careful planning and consideration of implications and challenges likewise. In this regard, hands-on experiences from industry practice are still rare. To fill this gap in scientific literature, we contribute a qualitative study on intentions, strategies, and challenges in the context of migrations to Microservices. We investigated the migration process of 14 systems across different domains and sizes by conducting 16 in-depth interviews with software professionals from 10 companies. Along with a summary of the most important findings, we present a separate discussion of each case. As primary migration drivers, maintainability and scalability were identified. Due to the high complexity of their legacy systems, most companies preferred a rewrite using current technologies over splitting up existing code bases. This was often caused by the absence of a suitable decomposition approach. As such, finding the right service cut was a major technical challenge, next to building the necessary expertise with new technologies. Organizational challenges were especially related to large, traditional companies that simultaneously established agile processes. Initiating a mindset change and ensuring smooth collaboration between teams were crucial for them. Future research on the evolution of software systems can in particular profit from the individual cases presented.

Modern automotive E/E systems are implemented as distributed real-time software systems. The constantly growing complexity of safety-relevant software functions leads to an increased importance of testing during system integration of such systems. Systematic metrics are required to guide the testing process during system integration by providing coverage measures and stopping criteria but few studied approaches exist. For this purpose, we introduce a data-flow based observation scheme which captures the interplay behavior of involved ECUs during test execution and failure occurrences. In addition, we introduce a data flow-based coverage criterion designed for black box integration. By applying the observation scheme to test cases and associated faults found during execution, we first analyze similarities in data flow coverage. By further analyzing the data flow of failures, that slipped through the phase of system integration testing, we evaluate the usefulness of test gaps identified by using the suggested coverage criterion. We found major differences in the usage of data flow between undetected failures and existing test cases. In addition, we found that for the studied system under test the occurrence of failures is not necessarily a direct consequence of the test execution due to functional dependencies and side effects. Overall, these findings highlight the potential and limitations of data flow-based measures to be formalized as coverage or stopping criteria for the integration testing of distributed software systems.

Abstract Motivation: How immature teams can become agile is a question that puzzles practitioners and researchers alike. Scrum is one method that supports agile working. Empirical research on the Scrum Master role remains scarce and reveals contradicting results. While the Scrum Master role is often centred on one person in rather immature teams, the role is expected to be shared among multiple members in mature teams. Objective: Therefore, we aim to understand how the Scrum Master role changes while the team matures. Method: We applied Grounded Theory and conducted qualitative interviews with 53 practitioners of 29 software and non-software project teams from Robert Bosch GmbH. Results: We discovered that Scrum Masters initially play nine leadership roles: Method Champion, Disciplinizer on Equal Terms, Coach, Change Agent, Helicopter, Moderator, Networker, Knowledge Enabler and Protector. They transfer some of those roles to the team while it matures. The Scrum Master provides a leadership gap, which allows team members to take on a leadership role. Conclusion: The Scrum Master role changes while the team matures. Trust and freedom to take over a leadership role in teams are essential enablers. Our results support practitioners in implementing agile teams in established companies.

Requirements Engineering (RE) has established itself as a software engineering discipline over the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that was first piloted in Germany and, after making substantial modifications, has now been replicated in 10 countries worldwide. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from the replication studies conducted in 10 countries with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirical theory of RE practice. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents are common, that requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly internally driven. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.

Maintainability assurance techniques are used to control this quality attribute and limit the accumulation of potentially unknown technical debt. Since the industry state of practice and especially the handling of Service- and Microservice-Based Systems in this regard are not well covered in scientific literature, we created a survey to gather evidence for a) used processes, tools, and metrics in the industry, b) maintainability-related treatment of systems based on service-orientation, and c) influences on developer satisfaction w.r.t. maintainability. 60 software professionals responded to our online questionnaire. The results indicate that using explicit and systematic techniques has benefits for maintainability. The more sophisticated the applied methods the more satisfied participants were with the maintainability of their software while no link to a hindrance in productivity could be established. Other important findings were the absence of architecture-level evolvability control mechanisms as well as a significant neglect of service-oriented particularities for quality assurance. The results suggest that industry has to improve its quality control in these regards to avoid problems with long-living service-based software systems.

Context: Cryptographic APIs are said to be not usable and researchers suggest to add example code to the documentation. Aim: We wanted to create a free platform for cryptographic code examples that improves the usability and security of created applications by non security experts. Method: We created the open-source web platform CryptoExamples and conducted a controlled experiment where 58 students added symmetric encryption to a Java program. We then measured the usability and security. Results: The participants who used the platform were not only significantly more effective (+73 %) but also their code contained significantly less possible security vulnerabilities (-66 %). Conclusions: With CryptoExamples the gap between hard to change API documentation and the need for complete and secure code examples can be closed. Still, the platform needs more code examples.

Context: Poor usability of cryptographic APIs is a severe source of vulnerabilities. Aim: We wanted to find out what kind of cryptographic libraries are present in Rust and how usable they are. Method: We explored Rust's cryptographic libraries through a systematic search, conducted an exploratory study on the major libraries and a controlled experiment on two of these libraries with 28 student participants. Results: Only half of the major libraries explicitly focus on usability and misuse resistance, which is reflected in their current APIs. We found that participants were more successful using rust-crypto which we considered less usable than ring before the experiment. Conclusion: We discuss API design insights and make recommendations for the design of crypto libraries in Rust regarding the detail and structure of the documentation, higher-level APIs as wrappers for the existing low-level libraries, and selected, good-quality example code to improve the emerging cryptographic libraries of Rust.

We developed an approach that protects users from using faulty spreadsheets in collaborative settings. Results from an empirical evaluation with 71 spreadsheet users indicate that the approach is both helpful and easy to learn and apply.

Abstract Spreadsheets are powerful tools that play a business‐critical role in many organizations. However, many bad decisions taken due to faulty spreadsheets show that these tools need serious quality assurance. Furthermore, while collaboration on spreadsheets for maintenance tasks is common, there has been almost no support for ensuring that the spreadsheets remain correct during this process. We have developed an approach named Spreadsheet Guardian, which separates the specification of spreadsheet test rules from their execution. By automatically executing user‐defined test rules, our approach is able to detect semantic faults. It also protects all collaborating spreadsheet users from introducing faults during maintenance, even if only few end‐users specify test rules. To evaluate Spreadsheet Guardian, we implemented a representative testing technique as an add‐in for Microsoft Excel. We evaluated the testing technique in 2 empirical evaluations with 29 end‐users and 42 computer science students. The results indicate that the technique is easy to learn and to apply. Furthermore, after finishing maintenance, participants with spreadsheets “protected” by the technique are more realistic about the correctness of their spreadsheets than participants who use only “classic,” noninteractive test rules on the basis of static analysis techniques. Hence, we believe that Spreadsheet Guardian can be of use for business‐critical spreadsheets.

In a time of digital transformation, the ability to quickly and efficiently adapt software systems to changed business requirements becomes more important than ever. Measuring the maintainability of software is therefore crucial for the long-term management of such products. With Service-based Systems (SBSs) being a very important form of enterprise software, we present a holistic overview of such metrics specifically designed for this type of system, since traditional metrics - e.g. object-oriented ones - are not fully applicable in this case. The selected metric candidates from the literature review were mapped to 4 dominant design properties: size, complexity, coupling, and cohesion. Microservice-based Systems (μSBSs) emerge as an agile and fine-grained variant of SBSs. While the majority of identified metrics are also applicable to this specialization (with some limitations), the large number of services in combination with technological heterogeneity and decentralization of control significantly impacts automatic metric collection in such a system. Our research therefore suggest that specialized tool support is required to guarantee the practical applicability of the presented metrics to μSBSs.

Software repositories contain a lot of information that can be transformed into suggestions other files they need to modify during maintenance tasks (so-called "coupled changes"). Existing studies however ignore developer feedback and their influence on the developer strategy for getting help during maintenance tasks. We used the Grounded Theory approach to investigate screen capture videos from an experiment to find which information sources developers use to find help and what is their relevance. We compared the frequency as well as the sequence patterns of used information sources both for the developers using coupled change suggestions and those not using them. We found a set of information sources where the developers seek for help and identified two categories of relevance. Also, we discovered that for the tasks using coupled change suggestions, the developers used mostly the internal IDE elements as an information source whereby the developers not using coupled change suggestions often used external sources like the documentation or the web. Coupled change suggestions influence the strategy how the developers seek for help by reducing the search for information on external locations which makes the process of solving maintenance tasks more compact.

Software repositories include information which can be made available for bug fixing or maintenance using repository mining. The identification of coupled changes have been proposed several times. Yet, existing studies focus on the found couplings and ignore feedback from developers. We investigate three development projects and their repositories to find files that frequently change together to support the software developers. We complement the coupled files information with details from the issue tracking system and the project documentation. We contrast our findings with feedback from the developers about how interesting our findings are for them. We found that the small size of the repositories made an insightful analysis difficult. The response to coupled changes both from experienced and inexperienced developers was mostly neutral. They accepted most of the additional attributes we presented. Furthermore, developers also suggested other additional issues to be relevant, e.g. the context of the coupled changes and the way they are presented, which we did not cover in this study. Therefore, coupled change analysis research will need to take the presentation and context information into account.

[Context] Problems in Requirements Engineering (RE) can lead to serious consequences during the software development lifecycle. [Goal] The goal of this paper is to propose empirically-based guidelines that can be used by different types of organisations according to their size (small, medium or large) and process model (agile or plan-driven) to help them in preventing such problems. [Method] We analysed data from a survey on RE problems answered by 228 organisations in 10 different countries. [Results] We identified the most critical RE problems, their causes and mitigation actions, organizing this information by clusters of size and process model. Finally, we analysed the causes and mitigation actions of the critical problems of each cluster to get further insights into how to prevent them. [Conclusions] Based on our results, we suggest preliminary guidelines for preventing critical RE problems in response to context characteristics of the companies.

Software repositories contain valuable information about the history of software changes. Using data mining, researchers have identified file changes that happened together frequently to present hints for necessary changes to developers. However, not all file change sets are related. This can affect the recommendations about coupled file changes negatively by delivering irrelevant couplings to the developers. The commit time and branching characteristics of Git have not been investigated together in previous heuristics for grouping related change sets. We exploit the mappings between commit messages and issue ids for judging the relatedness of change sets. We propose a heuristic for Git and investigate the influence of two factors, the time between the commits and their branching on the relatedness of change sets using the repositories of five open-source systems using logistic regression. According to our findings, the combination of these two factors influences the relatedness of change sets. Individually measured, only the time significantly influences the relatedness, the branching itself does not. Our results support previous heuristic that also in Git repositories the commit time is important for grouping related change sets.

Background: Code cloning - copying and reusing pieces of source code -- is a common phenomenon in software development in practice. There have been several empirical studies on the effects of cloning, but there are contradictory results regarding the connection of cloning and faults. Objective: Our aim is to clarify the relationship between code clones and faults. In particular, we focus on inconsistent (or type-3) clones in this work. Method: We conducted a case study with TWT GmbH where we detected the code clones in three Java systems, set them into relation to information from issue tracking and version control and interviewed three key developers. Results: Of the type-3 clones, 17 % contain faults. Developers modified most of the type-3 clones simultaneously and thereby fixed half of the faults in type-3 clones consistently. Type-2 clones with faults all evolved to fixed type-3 clones. Clone length is only weakly correlated with faultiness. Conclusion: There are indications that the developers in two cases have been aware of clones. It might be a reason for the weak relationship between type-3 clones and faults. Hence, it seems important to keep developers aware of clones, potentially with new tool support. Future studies need to investigate if the rate of faults in type-3 clones justifies using them as cues in defect detection.

The results of an automatic static analysis run can be overwhelming, especially for beginners. The overflow of information and the resulting need for many decisions is mentally tiring and can cause stress symptoms. There are several models in health care which are designed to fight stress. One of these is the salutogenesis model created by Aaron Antonovsky. In this paper, we will present an idea on how to transfer this model into a triage and recommendation model for static analysis tools and give an example of how this can be implemented in FindBugs, a static analysis tool for Java.

Sketching is an important means of communication in software engineering practice. Yet, there is little research investigating the use of sketches. We want to contribute a better understanding of sketching, in particular its use during program comprehension. We propose a controlled experiment to investigate the effectiveness and efficiency of program comprehension with the support of sketches as well as what sketches are used in what way.

There are repetitive patterns in strategies of manipulating source code. For example, modifying source code before acquiring knowledge of how a code works is a depth-first style and reading and understanding before modifying source code is a breadth-first style. To the extent we know there is no study on the influence of personality on them. The objective of this study is to understand the influence of personality on programming styles. We did a correlational study with 65 programmers at the University of Stuttgart. Academic achievement, programming experience, attitude towards programming and five personality factors were measured via self-assessed survey. The programming styles were asked in the survey or mined from the software repositories. Performance in programming was composed of bug-proneness of programmers which was mined from software repositories, the grades they got in a software project course and their estimate of their own programming ability. We did statistical analysis and found that Openness to Experience has a positive association with breadth-first style and Conscientiousness has a positive association with depth-first style. We also found that in addition to having more programming experience and better academic achievement, the styles of working depth-first and saving coarse-grained revisions improve performance in programming.

Context: Today's safety critical systems are increasingly reliant on software. Software becomes responsible for most of the critical functions of systems. Many different safety analysis techniques have been developed to identify hazards of systems. FTA and FMEA are most commonly used by safety analysts. Recently, STPA has been proposed with the goal to better cope with complex systems including software. Objective: This research aimed at comparing quantitatively these three safety analysis techniques with regard to their effectiveness, applicability, understandability, ease of use and efficiency in identifying software safety requirements at the system level. Method: We conducted a controlled experiment with 21 master and bachelor students applying these three techniques to three safety-critical systems: train door control, anti-lock braking and traffic collision and avoidance. Results: The results showed that there is no statistically significant difference between these techniques in terms of applicability, understandability and ease of use, but a significant difference in terms of effectiveness and efficiency is obtained. Conclusion: We conclude that STPA seems to be an effective method to identify software safety requirements at the system level. In particular, STPA addresses more different software safety requirements than the traditional techniques FTA and FMEA, but STPA needs more time to carry out by safety analysts with little or no prior experience.

Object-oriented programming (OOP) is widely used in the software industry and university introductory courses today. Following the structure of most textbooks, such courses frequently are organised starting with the concepts of imperative and structured programming and only later introducing OOP. An alternative approach is to begin directly with OOP following the Outside-In teaching method as proposed by Meyer. Empirical results for the effects of Outside-In teaching on students and lecturers are sparse, however. We describe the conceptual design and empirical evaluation of two OOP introductory courses from different universities based on Outside-In teaching. The evaluation results are compared to those from a third course serving as the control group, which was taught OOP the "traditional" way. We evaluate the initial motivation and knowledge of the participants and the learning outcomes. In addition, we analyse results of the end term exams and qualitatively analyse the results of interviews with the lecturers and tutors. Regarding the learning outcomes, the results show no significant differences between the Outside-In and the "traditional" teaching method. In general, students found it harder to solve and implement algorithmic problems than to understand object oriented (OO) concepts. Students taught OOP by the Outside-In method, however, were less afraid that they would not pass the exam at the end of term and understood the OO paradigm more quickly. Therefore, the Outside-In method is no silver bullet for teaching OOP regarding the learning outcomes but has positive effects on motivation and interest.

Background/Context: Gathering empirical knowledge is a time consuming task and the results from empirical studies often are soon outdated by new technological solutions. As a result, the impact of empirical results on software engineering practice is often not guaranteed.Objective/Aim: In this paper, we summarise the ongoing discussion on "Empirical Software Engineering 2.0" as a way to improve the impact of empirical results on industrial practices. We propose a way to combine data mining and analysis with domain knowledge to enable fast feedback cycles in empirical software engineering research.Method: We identify the key concepts on gathering fast feedback in empirical software engineering by following an experience-based line of reasoning by argument. Based on the identified key concepts, we design and execute a small proof of concept with a company to demonstrate potential benefits of the approach.Results: In our example, we observed that a simple double feedback mechanism notably increased the precision of the data analysis and improved the quality of the knowledge gathered.Conclusion: Our results serve as a basis to foster discussion and collaboration within the research community for a development of the idea.

Context] Many software projects fail due to problems in requirements engineering (RE).[Objective] The goal of this paper is to gather information on relevant RE problems and to represent knowledge on their most common causes.[Method] We replicated a global family of RE surveys in Brazil and used the data to identify critical RE problems and to build probabilistic causeeffect diagrams to represent knowledge on their common causes.[Results] The survey was answered by 74 different organizations, including small, medium and very large sized companies, conducting both, plan-driven and agile development.The most critical RE problems, according to those organizations, are related to communication and to incomplete or underspecified requirements.We provide the full probabilistic cause-effect diagrams with knowledge on common causes of the most critical identified RE problems online.[Conclusion] We believe that the knowledge presented in the diagrams can be helpful to support organizations in conducting causal analysis sessions by providing an initial understanding on what usually causes critical RE problems.

Context: Requirements engineering process improvement (REPI) approaches have gained much attention in research and practice. Goal: So far, there is no comprehensive view on the research in REPI in terms of solutions and current state of reported evidence. We aims to provide an overview on the existing solutions, their underlying principles and their research type facets, i.e. their state of empirical evidence. Method: To this end, we conducted a systematic mapping study of the REPI publication space. Results: This paper reports on the first findings regarding research type facets of the contributions as well as selected methodological principles. We found a strong focus in the existing research on solution proposals for REPI approaches that concentrate on normative assessments and benchmarks of the RE activities rather than on holistic RE improvements according to individual goals of companies. Conclusions: We conclude, so far, that there is a need to broaden the work and to investigate more problem-driven REPI which also targets the improvement of the quality of the underlying RE artefacts, which currently seem out of scope.

In our work with industry partners as well as with students in seminars we noticed that many people often stick to "aged" metrics when they want to evaluate the maintainability of software. They consider these metrics for this purpose without second thoughts, because the metrics are so present that almost every developer has at least heard the name and has some kind of knowledge about it. This smattering leads to an unreflected usage of theses metrics today without fully understanding them. To find suitable metrics for maintainability evaluation, we did a preliminary study, which indicated that still McCabe, Halstead and some kind of lines of code measurement dominate maintainability evaluation. We will discuss examples that question the ability of those metrics to reliably and automatically evaluate maintainability of software. Instead we will present metrics we consider more suitable to make solid statements about software maintainability. We conclude that our proposed metrics are better evaluators for maintainability because their impact on maintenance is clearer to interpret. Future work on metrics should carefully consider the use of "aged" metrics.

Context: For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, those studies either concentrate on validating specific methods or on single companies or countries. Therefore, they allow only for limited empirical generalisations. Objective: To lay an empirical and generalisable foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. Method: We designed a globally distributed family of surveys in joint collaborations with different researchers from different countries. The instrument is based on an initial theory inferred from available studies. As a long-term goal, the survey will be regularly replicated to manifest a clear understanding on the status quo and practical needs in RE. In this paper, we present the design of the family of surveys and first results of its start in Germany. Results: Our first results contain responses from 30 German companies. The results are not yet generalisable, but already indicate several trends and problems. For instance, a commonly stated problem respondents see in their company standards are artefacts being underrepresented, and important problems they experience in their projects are incomplete and inconsistent requirements. Conclusion: The results suggest that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.

Context:Static analysis of source code is a promising opportunity to detect faults and badly designed areas, which will have a negative effect on the overall product quality. In addition, it is inexpensive to integrate it into the development process, because it is easy to automate. Objective:Despite these benefits, static analysis is not as commonly accepted as other quality assurance techniques. To be able to realise its full potential, we need to better understand the problems which prevent software developers from successfully using static analysis tools. Method: To gain new insights we propose to combine the observation methodologies eye tracking, think aloud and questionnaires for a comprehensive investigation on the usage barriers. Results: Our first experimental results with students show new issues: Automated static analysis tools could benefit from a more direct feedback to changes and a more clear guidance of the users. Conclusion:Therefore, we consider the approach as promising and we will conduct further experiments to understand the barriers in static analysis usage.

PROMISE conference is an annual forum for researchers and practitioners to present, discuss and exchange ideas, results, expertise and experiences in construction and/or application of prediction models in software engineering. Such models could be targeted at: planning, design, implementation, testing, maintenance, quality assurance, evaluation, process improvement, management, decision making, and risk assessment in software and systems development. PROMISE is distinguished from similar forums with its public data repository and focus on methodological details, providing a unique interdisciplinary venue for software engineering and machine learning communities, and seeking for verifiable and repeatable prediction models that are useful in practice.

Empirical research has become an important means in any science and continues to grow in importance also in software engineering research. The ISERN and ESEM community together with the Empirical Software Engineering Journal have advanced the state of the art in how empirical research is conducted today in software engineering. We now have a plethora of empirical research techniques at our disposal, and it sometimes becomes challenging to choose the appropriate one. Also the application of the techniques, such as experiments, case studies, surveys or systematic literature reviews, is complicated and needs detailed understanding. This doctoral symposium aimed at supporting PhD students who are already conducting or plan to conduct empirical research as part of their research project. Experienced empirical researchers form the board of advisors reviewed the research plans from PhD students and gave them detailed feedback and guidance. The most promising research plans were invited for presentation at the symposium where the students could get more direct feedback in the discussion. Each of the presenting PhD students was assigned a mentor from the board of advisors who supported them in improving the research plan.

Quality requirements are an often neglected part of requirements engineering. If specified at all, they tend to be either too abstract or very technical and without a rationale. In this paper, we evaluate a quality requirements approach, which makes use of activity-based quality models. To this end, we conduct a comparative case study at Siemens in which we compare the requirements resulting from applying our quality model with the requirements previously used in the same environment. The results indicate an improvement of the requirements regarding, e.g., structured ness and trace ability, but also that the productivity perceived by the industry participants could not be increased. The study thus gives first insights into strengths and limitations of using a quality model in an industrial requirements engineering process.

Published software quality models either provide abstract quality attributes or concrete quality assessments. There are no models that seamlessly integrate both aspects. In the project Quamoco, we built a comprehensive approach with the aim to close this gap. For this, we developed in several iterations a meta quality model specifying general concepts, a quality base model covering the most important quality factors and a quality assessment approach. The meta model introduces the new concept of a product factor, which bridges the gap between concrete measurements and abstract quality aspects. Product factors have measures and instruments to operationalise quality by measurements from manual inspection and tool analysis. The base model uses the ISO 25010 quality attributes, which we refine by 200 factors and 600 measures for Java and C# systems. We found in several empirical validations that the assessment results fit to the expectations of experts for the corresponding systems. The empirical analyses also showed that several of the correlations are statistically significant and that the maintainability part of the base model has the highest correlation, which fits to the fact that this part is the most comprehensive. Although we still see room for extending and improving the base model, it shows a high correspondence with expert opinions and hence is able to form the basis for repeatable and understandable quality assessments in practice.

[Background:] Nowadays, industries are facing the problem that the Requirements Engineering (RE) process is highly volatile, since it depends on project influences from the customer's domain or from process models used. Artefact-based approaches promise to provide guidance in the creation of consistent artefacts in volatile project environments, because these approaches concentrate on the artefacts and their dependencies, instead of prescribing processes. Yet missing, however, is empirical evidence on the advantages of applying artefact-based RE approaches in real projects. [Aim:] We developed a customisable artefact-based RE approach for the domain of business information systems. Our goal is to investigate the advantages and limitations of applying this customisable approach in an industrial context. [Method:] We conduct a case study with our artefact-based RE approach and its customisation procedure. For this, we apply it at a software development project at Siemens following the steps of the customisation procedure. We assess our approach in direct comparison with the previously used RE approach considering possible improvements in the process and in the quality of the produced artefacts. [Results:] We show that our approach is flexible enough to respond to the individual needs in the analysed project environment. Although the approach is not rated to be more productive, we find an improvement in the syntactic and the semantic quality of the created artefacts. [Conclusions:] We close a gap in the RE literature by giving empirical evidence on the advantages of artefact orientation in RE in an industrial setting.

Continuous quality assessment is crucial for the long-term success of evolving software. On the one hand, code analysis tools automatically supply quality indicators, but do not provide a complete overview of software quality. On the other hand, quality models define abstract characteristics that influence quality, but are not operationalized. Currently, no tool chain exists that integrates code analysis tools with quality models. To alleviate this, the Quamoco project provides a tool chain to both define and assess software quality. The tool chain consists of a quality model editor and an integration with the quality assessment toolkit ConQAT. Using the editor, we can define quality models ranging from abstract characteristics down to operationalized measures. From the quality model, a ConQAT configuration can be generated that can be used to automatically assess the quality of a software system.

Software becomes ever more feature-rich and thereby harder to distinguish based on its functionality. Instead, quality is starting to differentiate between similar software products. Specifying, constructing, and assuring quality has been under research for several decades and continues to be a long- term research area because of its many facets and its com- plexity. Current national and international initiatives show that there is an active research community in academia and industry. This workshop builds on the rich experiences of a series of previous workshops and aims to bring this community together to discuss current issues and future developments.

Due to their pivotal role in software engineering, considerable effort is spent on the quality assurance of software requirements specifications. As they are mainly described in natural language, relatively few means of automated quality assessment exist. However, we found that clone detection, a technique widely applied to source code, is promising to assess one important quality aspect in an automated way, namely redundancy that stems from copy&paste operations. This paper describes a large-scale case study that applied clone detection to 28 requirements specifications with a total of 8,667 pages. We report on the amount of redundancy found in real-world specifications, discuss its nature as well as its consequences and evaluate in how far existing code clone detection approaches can be applied to assess the quality of requirements specifications in practice.

Driven by market needs and laws, automotive manufacturers develop ever more feature-rich and complex vehicles. This new functionality plays even an active role in driving, what poses many new challenges on assuring the safety of the vehicle. Safety cases constitute a proven technique to systematically use existing information about a system, its environment, and development context to show its safety. We construct the safety case for a cruise control system describe in a case study in the automotive domain with a special consideration of existing domain-specific models. In the case study, we identify generic safety case modules and several reoccurring patterns, which will simplify the development of future automotive safety cases.

This paper presents the findings of a survey on quality models in practice conducted among four software companies in Germany. In the first phase of the study, 25 quality managers and users of software quality models were interviewed regarding the use of quality models, quality assurance techniques, and problems arising from the current situation in their companies. We present qualitative and quantitative findings as well as our plans for the second study phase including an international online questionnaire.

Code cloning is not only assumed to inflate maintenance costs but also considered defect-prone as inconsistent changes to code duplicates can lead to unexpected behavior. Consequently, the identification of duplicated code, clone detection, has been a very active area of research in recent years. Up to now, however, no substantial investigation of the consequences of code cloning on program correctness has been carried out. To remedy this shortcoming, this paper presents the results of a large-scale case study that was undertaken to find out if inconsistent changes to cloned code can indicate faults. For the analyzed commercial and open source systems we not only found that inconsistent changes to clones are very frequent but also identified a significant number of faults induced by such changes. The clone detection tool used in the case study implements a novel algorithm for the detection of inconsistent clones. It is available as open source to enable other researchers to use it as basis for further investigations.

Software productivity has been analyzed traditionally in terms of size measures such as LOC or FP. These measures have failed to provide a comprehensive basis for productivity analysis. In the research department of the Siemens AG the new approach ProdFLOW™ for the analysis and management of a research & development organization's productivity is being created based on a revised understanding of the term productivity. Former studies often start with fixed, typical indicators and quantitatively analyze the relation between productivity and the indicator by regression models. ProdFLOW™ departs from the fixed model approach, which might not fit to the conditions of the organization. Instead an organization-specific model based on the substantial levers of the productivity, which are both influenceable and measurable, are compiled together with the experts of the organization. The paper explains the new approach as well as gives an example to illustrate our approach based on the results of three performed case studies.

Model-based development is becoming an increasingly common development methodology. In important domains like embedded systems already major parts of the code are generated from models specified with domain-specific modelling languages. Hence, such models are nowadays an integral part of the software development and maintenance process and therefore have a major economic and strategic value for the software-developing organisations. Nevertheless almost no work has been done on a quality defect that is known to seriously hamper maintenance productivity in classic code-based development: Cloning. This paper presents an approach for the automatic detection of clones in large models as they are used in model-based development of control systems. The approach is based on graph theory and hence can be applied to most graphical data-flow languages. An industrial case study demonstrates the applicability of our approach for the detection of clones in Matlab/Simulink models that are widely used in model-based development of embedded systems in the automotive domain.

Automated static analysis is a promising technique to detect defects in software. However, although considerable effort has been spent for developing sophisticated detection possibilities, the effectiveness and efficiency has not been treated in equal detail. This paper presents the results of two industrial case studies in which two tools based on bug patterns for Java are applied and evaluated. First, the economic implications of the tools are analysed. It is estimated that only 3-4 potential field defects need to be detected for the tools to be cost-efficient. Second, the capabilities of detecting field defects are investigated. No field defects have been found that could have been detected by the tools. Third, the identification of fault-prone classes based on the results of such tools is investigated and found to be possible. Finally, methodological consequences are derived from the results and experiences in order to improve the use of bug pattern tools in practice.

There are various models in software engineering that are used to predict quality-related aspects of the process or artefacts. The use of these models involves elaborate data collection in order to estimate the input parameters. Hence, an interesting question is which of these input factors are most important. More specifically, which factors need to be estimated best and which might be removed from the model? This paper describes an approach based on global sensitivity analysis to answer these questions and shows its applicability in a case study on the COCOMO application at NASA.

Maintainability is a key quality attribute of successful software systems. However, its management in practice is still problematic. Currently, there is no comprehensive basis for assessing and improving the maintainability of software systems. Quality models have been proposed to solve this problem. Nevertheless, existing approaches do not explicitly take into account the maintenance activities, that largely determine the software maintenance effort. This paper proposes a 2-dimensional model of maintainability that explicitly associates system properties with the activities carried out during maintenance. The separation of activities and properties facilitates the identification of sound quality criteria and allows to reason about their interdependencies. This transforms the quality model into a structured and comprehensive quality knowledge base that is usable in industrial project environments. For example, review guidelines can be generated from it. The model is based on an explicit quality metamodel that supports its systematic construction and fosters preciseness as well as completeness. An industrial case study demonstrates the applicability of the model for the evaluation of the maintainability of Matlab Simulink models that are frequently used in model-based development of embedded systems.

One of the main cost factors in software development is the detection and removal of defects. However, the relationships and influencing factors of the costs and revenues of defect-detection techniques are still not well understood. This paper proposes an analytical, stochastic model of the economics of defect detection and removal to improve this understanding. The model is able to incorporate dynamic as well as static techniques in contrast to most other models of that kind. We especially analyse the model with state-ofthe-art sensitivity analysis methods to (1) identify the most relevant factors for model simplification and (2) prioritise the factors to guide further research and measurements.

Model-based testing relies on behavior models for the generation of model traces: input and expected output---test cases---for an implementation. We use the case study of an automotive network controller to assess different test suites in terms of error detection, model coverage, and implementation coverage. Some of these suites were generated automatically with and without models, purely at random, and with dedicated functional test selection criteria. Other suites were derived manually, with and without the model at hand. Both automatically and manually derived model-based test suites detected significantly more requirements errors than hand-crafted test suites that were directly derived from the requirements. The number of detected programming errors did not depend on the use of models. Automatically generated model-based test suites detected as many errors as hand-crafted model-based suites with the same number of tests. A sixfold increase in the number of model-based tests led to an 11% increase in detected errors.

There are various ways to evaluate defect-detection techniques. However, for a comprehensive evaluation the only possibility is to reduce all influencing factors to costs. There are already some models and metrics for the cost of quality that can be used in that context. The existing metrics for the effectiveness and efficiency of defect-detection techniques and experiences with them are combined with cost metrics to allow a more fine-grained estimation of costs and a comprehensive evaluation of defect-detection techniques. The current model is most suitable for directly comparing concrete applications of different techniques