High-quality data is a prerequisite for most types of analysis provided by software systems. However, since data quality does not come for free, it has to be assessed and managed continuously. The increasing quantity, diversity, and velocity that characterize big data today make these tasks even more challenging. We identified challenges that are specific for big data quality assessments with particular emphasis on their usage in smart ecosystems and make a proposal for a scalable cross-organizational approach that addresses these challenges. We developed an initial prototype to investigate scalability in a multi-node test environment using big data technologies. Based on the observed horizontal scalability behavior, there is an indication that the proposed approach also allows dealing with increasing volumes of heterogeneous data.

Besides model-based development, model-based quality assurance and the tighter integration of static and dynamic quality assurance activities are becoming increasingly relevant in the development of software-intensive systems. Thus, this paper reports on an empirical study aimed at investigating the promises regarding quality improvements and cost savings. The evaluation comprises data from 13 industry case studies conducted during a three-year large-scale research project in the transportation domain (automotive, avionics, rail system). During the evaluation, we identified major goals and strategies associated with (integrated) model-based analysis and testing and evaluated the improvements achieved. The aggregated results indicate an average cost reduction of between 29% and 34% for verification and validation and of between 22% and 32% for defect removal. Compared with these cost savings, improvements regarding test coverage (~8%), number of remaining defects (~13%), and time to market (~8%) appear less noticeable.

Published software quality models either provide abstract quality attributes or concrete quality assessments. There are no models that seamlessly integrate both aspects. In the project Quamoco, we built a comprehensive approach with the aim to close this gap. For this, we developed in several iterations a meta quality model specifying general concepts, a quality base model covering the most important quality factors and a quality assessment approach. The meta model introduces the new concept of a product factor, which bridges the gap between concrete measurements and abstract quality aspects. Product factors have measures and instruments to operationalise quality by measurements from manual inspection and tool analysis. The base model uses the ISO 25010 quality attributes, which we refine by 200 factors and 600 measures for Java and C# systems. We found in several empirical validations that the assessment results fit to the expectations of experts for the corresponding systems. The empirical analyses also showed that several of the correlations are statistically significant and that the maintainability part of the base model has the highest correlation, which fits to the fact that this part is the most comprehensive. Although we still see room for extending and improving the base model, it shows a high correspondence with expert opinions and hence is able to form the basis for repeatable and understandable quality assessments in practice.

Existing software quality models typically focus on common quality characteristics such as the ISO 25010 software quality characteristics. However, most of them provide insufficient operationalization for quality assessments of source code. Moreover, they usually focus on software in general or on information systems and do not sufficiently cover the particularities of embedded systems. We have developed a quality model that covers quality requirements for source code that are specific for embedded systems software. It provides comprehensive operationalization (with 336 measures) for C and C++ systems, which allows for largely automated quality assessments. The empirical evaluations performed acknowledge moderate completeness of the requirements and the associated measures. Therefore, we still see room for improvements to allow covering even more aspects of embedded systems software quality. Nevertheless, the empirical validation (based on three industrial products) shows good concordance between the results gained by the automatic model-based assessment and independent expert judgment on code quality.

Reliable predictions are essential for managing software projects with respect to cost and quality. Several studies have shown that hybrid prediction models combining causal models with Monte Carlo simulation are especially successful in addressing the needs and constraints of today's software industry: They deal with limited measurement data and, additionally, make use of expert knowledge. Moreover, instead of providing merely point estimates, they support the handling of estimation uncertainty, e.g., estimating the probability of falling below or exceeding a specific threshold. Although existing methods do well in terms of handling uncertainty of information, we can show that they leave uncertainty coming from imperfect modeling largely unaddressed. One of the consequences is that they probably provide over-confident uncertainty estimates. This paper presents a possible solution by integrating bootstrapping into the existing methods. In order to evaluate whether this solution does not only theoretically improve the estimates but also has a practical impact on the quality of the results, we evaluated the solution in an empirical study using data from more than sixty projects and six estimation models from different domains and application areas. The results indicate that the uncertainty estimates of currently used models are not realistic and can be significantly improved by the proposed solution.

Defining strategies on how to perform quality assurance (QA) and how to control such activities is a challenging task for organizations developing or maintaining software and software-intensive systems. Planning and adjusting QA activities could benefit from accurate estimations of the expected defect content of relevant artifacts and the effectiveness of important quality assurance activities. Combining expert opinion with commonly available measurement data in a hybrid way promises to overcome the weaknesses of purely data-driven or purely expert-based estimation methods. This article presents a case study of the hybrid estimation method HyDEEP for estimating defect content and QA effectiveness in the telecommunication domain. The specific focus of this case study is the use of the method for gaining quantitative predictions. This aspect has not been empirically analyzed in previous work. Among other things, the results show that for defect content estimation, the method performs significantly better statistically than purely data-based methods, with a relative error of 0.3 on average (MMRE).

This paper presents the findings of a survey on quality models in practice conducted among four software companies in Germany. In the first phase of the study, 25 quality managers and users of software quality models were interviewed regarding the use of quality models, quality assurance techniques, and problems arising from the current situation in their companies. We present qualitative and quantitative findings as well as our plans for the second study phase including an international online questionnaire.

Quality assurance (QA) plays a crucial role in today's software development. However, methods and models proposed in literature to support QA management suffer from several drawbacks. Many are specialized to certain activities like system test or inspections. They commonly support only one application purpose, e.g., planning or controlling, and are often applicable only after measurement data has been collected for several historical applications. To overcome these drawbacks, we developed a method that can be applied to QA activities during any phase, and which supports comprehensive quality management related tasks: improvement, planning, and controlling. To be applicable in practice, the method combines the available measurement data with expert judgment to build context-specific models. In addition, the method provides early benefits, while motivating the collection of measurement data by presenting possible improvement directions. The paper presents the general concepts behind the method and research questions to be answered in upcoming empirical studies.

Planning quality assurance (QA) activities in a systematic way and controlling their execution are challenging tasks for companies that develop software or software-intensive systems. Both require estimation capabilities regarding the effectiveness of the applied QA techniques and the defect content of the checked artifacts. Existing approaches for these purposes need extensive measurement data from his-torical projects. Due to the fact that many companies do not collect enough data for applying these approaches (es-pecially for the early project lifecycle), they typically base their QA planning and controlling solely on expert opinion. This article presents a hybrid method that combines commonly available measurement data and context-specific expert knowledge. To evaluate the method’s applicability and usefulness, we conducted a case study in the context of independent verification and validation activities for critical software in the space domain. A hybrid defect content and effectiveness model was developed for the software requirements analysis phase and evaluated with available legacy data. One major result is that the hybrid model pro-vides improved estimation accuracy when compared to applicable models based solely on data. The mean magni-tude of relative error (MMRE) determined by cross-validation is 29.6% compared to 76.5% obtained by the most accurate data-based model.