Special issues on

Abstract Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary qualities of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to re-interpret existing qualities for ML systems or add new ones (such as trustworthiness). We have to be very precise about which quality property is relevant for which entity of interest (such as completeness of training data or correctness of trained model), and how to objectively evaluate adherence to quality requirements. In this article, we present how to systematically construct quality models for ML systems based on an industrial use case. This quality model enables practitioners to specify and assess qualities for ML systems objectively. In addition to the overall construction process described, the main outcomes include a meta-model for specifying quality models for ML systems, reference elements regarding relevant views, entities, quality properties, and measures for ML systems based on existing research, an example instantiation of a quality model for a concrete industrial use case, and lessons learned from applying the construction process. We found that it is crucial to follow a systematic process in order to come up with measurable quality properties that can be evaluated in practice. In the future, we want to learn how the term quality differs between different types of ML systems and come up with reference quality models for evaluating qualities of ML systems.

As the applications of machine learning algorithms in various fields are widely demanded, the development of machine learning software systems (MLS) is rapidly increasing. The quality of MLS is different from that of conventional software systems, in the sense that it depends on the amount and distribution of training data in a model learning and input data during operation. This is a major challenge in quality assurance of MLS development for the enterprise. In this paper, we propose a requirements-driven method to determine the quality characteristics of the MLS. Major contributions of this paper include: (1) Extending the quality characteristics of ISO 25010, which defines the conventional software quality, to those unique to MLS; this paper also defines its measuring method. (2) A method to identify requirements, i.e., issues to be determined in the requirements definition, in order to derive the quality characteristics and measurement methods for MLS, since the quality characteristics and the measurement method depend on the goals of the system under development. In order to evaluate the proposed method, we carried out an empirical study of the quality characteristics and measurement methods related to functional correctness and the maturity of the MLS for the enterprise. Based on the study, we compare the quality characteristics and measurement methods derived by the proposed method with those suggested by developers, and demonstrate the effectiveness of the proposed method.

Software productivity and effort estimationEvery software business has to be able to budget and plan its software development projects realistically.This is the prerequisite for professional project management, for competitive offers, and for a more objective assessment of the offers made by potential contractors.In order to achieve these objectives, a software business must be capable of estimating their own productivity in developing software at an appropriate level of quality and produce accurate estimates for the effort required to develop software.Since 1999, the International Conference on Product-Focused Software Process Improvement (PROFES) has established itself as one of the most recognized international conferences on professional software process improvement (SPI) motivated by product, process, and service quality needs.PROFES addresses both quality engineering and management topics, including processes, methods, techniques, tools, organizations, and enabling SPI.Both solutions found in practice and relevant research results from academia are presented.The 14 th instance of the conference -PROFES 2013was held in Paphos, Cyprus, from 12 June to 14 June 2013.The technical program was selected by a committee of leading experts in SPI, software process modeling, and empirical software engineering research.Overall, 41 papers were submitted, with each paper getting reviewed by at least three reviewers.After thorough evaluation, the Program Committee finally selected 22 technical full papers.The topics addressed in these papers indicate that SPI is a vibrant research discipline and is also of high interest for industry.Many papers report on case studies or SPI-related experience gained in industry.The technical program consisted of the following tracks: decision support in software engineering, empirical software engineering, managing software processes, safety-critical software engineering, software measurement, SPI, and software maintenance.We would like to thank all authors, presenters, and session chairs for their time and effort in making PROFES 2013 a success.Our special thanks go to our keynote speakers for highlighting the most recent findings and novel results in the area of process improvement.We were proud to have three keynote speakers: Stefan Wagner (University of Stuttgart, Germany) on 'Making Software Quality Visible', Alexis Ocampo (ECOPETROL S.A., Colombia) on 'ECO-MAPS: Information Quality Driven Enterprise Modeling', and Christos Xenis (LogiSoft, Cyprus) on the 'Implementation of an Online Multi-Level and Device-Independent Time & Attendance System'.Furthermore, we also would like to thank the organizers of the co-located tutorial sessions: Hermann Kaindl (Vienna University of Technology, Austria) on 'Model-based Transition from Requirements to High-level Software Design' and Jens Heidrich (Fraunhofer IESE, Germany) on 'Software Effort Estimation and Risk Management'.In addition, we sincerely thank all Program Committee members and reviewers who provided excellent support in reviewing the papers, Andreas Jedlitschka (Fraunhofer IESE, Germany) for his work as General Chair, and George Angelos Papadopoulos (

The concept of "software quality" is often hard to capture for an organization. Quality models aim at making the concept more operational by refining the "quality" of software development products and processes into sub-concepts down to the level of concrete metrics and indicators. In practice, it is difficult for an organization to come up with a reliable quality model because quality depends on numerous organizational context factors, and the model as well as the metrics and indicators need to be tailored to the specifics of the organization. This paper presents experiences in developing custom-tailored quality models for an organization, exemplified by Ecopetrol, a Colombian oil and gas company. The general approach taken is illustrated and excerpts from the initial model are presented.

Companies increasingly recognize that software and IT play a significant role for their current and future business strategies. Therefore, it is important to align IT/software-related strategies with the business goals across the organization. Currently, little experience exists regarding how to effectively create this missing business-IT link. For this purpose, the GQM+Strategies®approach was developed to support companies in aligning IT/software-related strategies with business goals through measurement. This paper focuses on facilitating the approach for aligning IT/software projects with an organization's higher-level goals. Lessons learned from applying the approach in the context of the Japanese Information-technology Promotion Agency (IPA), specifically its Software Engineering Center (SEC), are presented. The transparent documentation of goals and strategies, and the collection of key performance indicators were helpful for effectively aligning the projects with overall organizational goals and strategies as well as for evaluating the degree of alignment and the risk of misalignment.

Abstract Software cockpits, also known as Software Project Control Centers, support the management and controlling of software and system development projects and provide means for quantitative, measurement‐based project control. Currently, many companies are developing simple control dashboards that are mainly based on Spreadsheet applications. Alternatively, they use solutions providing a fixed set of project control functionalities that cannot be sufficiently customized to their specific needs and goals. Specula is a systematic approach for defining reusable, customizable control components and instantiating them according to different organizational goals and characteristics based on the Quality Improvement Paradigm (QIP) and the Goal Question Metric (GQM) approach. This article gives an overview of the Specula approach, including the basic conceptual model, goal‐oriented composition of control centers based on explicitly stated measurement goals and a conceptual architecture supporting the approach. Related approaches are discussed, a practical usage example is given, and evaluation results from using Specula as part of industrial case studies are presented. Copyright © 2010 John Wiley & Sons, Ltd.

One means for institutionalizing project control, systematic quality assurance, and management support on the basis of measurement and explicit models is the establishment of so-called Software Project Control Centers. Nowadays many companies develop their own dashboards for project control or use off-the-shelf tools that provide a predefined functionality. It is not clear how to tailor an existing tool to the specific needs and goals. An engineering-like approach providing the methodological foundation is needed for systematically defining and applying project control mechanisms. "Soft-Pit" is a research project focusing on an improvement-oriented approach for setting up and applying project control mechanisms in a goal-oriented way and evaluating the practical benefits of such a control center. This article describes the results of industrial case studies conducted in the context of the project. Moreover, lessons learned are discussed, related work is described, and future work is presented.

GQM+Strategies is a measurement approach that builds on the well-tested GQM approach to planning and implementing software measurement. Although GQM has proven itself useful in a variety of industrial settings, one recognized weakness is the difficulty for GQM users to link software measurement goals to higher-level goals of the organization in which the software is being developed. This linkage is important, as it helps to justify software measurement efforts and allows measurement data to contribute to higher-level decisions. GQM+strategies provides mechanisms for explicitly linking software measurement goals, to higher-level goals for the software organization, and further to goals and strategies at the level of the entire business.

Many software development organizations still lack support for detecting and reacting to critical project states in order to achieve planned goals. One means to institutionalize project control, systematic quality assurance, and management support on the basis of measurement and explicit models is the establishment of so-called software project control centers. However, there is only little experience reported in the literature with respect to setting up and applying such control centers in industrial environments. One possible reason is the lack of appropriate evaluation instruments (such as validated questionnaires and appropriate analysis procedures). Therefore, we developed an initial measurement instrument to systematically collect experience with respect to the deployment and use of control centers. Our main research goal was to develop and evaluate the measurement instrument. The instrument is based on the technology acceptance model (TAM) and customized to project controlling. This article illustrates the application and evaluation of this measurement instrument in the context of industrial case studies and provides lessons learned for further improvement. In addition, related work and conclusions for future work are given.

SOFTPIT 2007 is the first workshop on Measurement-based Cockpits for Distributed Software and Systems Engineering Projects. Its goal is to discuss technical and societal challenges for software cockpits in global software development projects.

Cost estimation is a very crucial field for software developing companies. The acceptance of an estimation technique is highly dependent on estimation accuracy. Often, this accuracy is only determined after an initial application. Possible further steps for improving the underlying estimation model typically do not influence the decision on whether to discard the technique or deploy it. In addition, most estimation techniques do not explicitly support the evolution of the underlying estimation model in an iterative manner. This increases the risk of overlooking some important cost drivers or data inconsistencies. This paper presents an enhanced process for developing a CoBRA® cost estimation model by systematically including iterative analysis and feedback cycles, and its evaluation in a software development unit of Oki Electric Industry Co., Ltd., Japan. During the model improvement cycles, estimation accuracy was improved from an initial 120% down to 14%. In addition, lessons learned with the iterative development approach are described.