Background. Effort-aware metrics (EAMs) are widely used to evaluate the effectiveness of software defect prediction models, while accounting for the effort needed to analyze the software modules that are estimated defective. The usual underlying assumption is that this effort is proportional to the modules’ size measured in LOC. However, the research on module analysis (including code understanding, inspection, testing, etc.) suggests that module analysis effort may be better correlated to code attributes other than size. Aim. We investigate whether assuming that module analysis effort is proportional to other code metrics than LOC leads to different evaluations. Method. We show mathematically that the choice of the code measure used as the module effort driver crucially influences the resulting evaluations. To illustrate the practical consequences of this, we carried out a demonstrative empirical study, in which the same model was evaluated via EAMs, assuming that effort is proportional to either McCabe’s complexity or LOC. Results. The empirical study showed that EAMs depend on the underlying effort model, and can give quite different indications when effort is modeled differently. It is also apparent that the extent of these differences varies widely. Conclusions. Researchers and practitioners should be aware that the reliability of the indications provided by EAMs depend on the nature of the underlying effort model. The EAMs used until now appear to be actually size-aware, rather than effort-aware: when analysis effort does not depend on size, these EAMs can be misleading.

Context: ROC (Receiver Operating Characteristic) curves are widely used to represent how well fault-proneness models (e.g., probability models) classify software modules as faulty or non-faulty. AUC , the Area Under the ROC Curve, is usually used to quantify the overall discriminating power of a fault-proneness model. Alternative indicators proposed, e.g., RRA (Ratio of Relevant Areas), consider the area under a portion of a ROC curve. Each point of a ROC curve represents a binary classifier, obtained by setting a specified threshold on the fault-proneness model. Several performance metrics (Precision, Recall, the F-score, etc.) are used to assess a binary classifier. Objectives: We investigate the relationships linking “under the ROC curve area” indicators such as AUC and RRA to performance metrics. Methods: We study these relationships analytically. We introduce iso-PM ROC curves, whose points have the same value P M ¯ for a given performance metric PM. When evaluating a ROC curve, we identify the iso-PM curve with the same value of AUC or RRA . Its P M ¯ can be seen as a property of the ROC curve and fault-proneness model under evaluation. Results: There is an S-shaped relationship between P M ¯ and AUC for performance metrics that do not depend on the proportion ρ of faulty modules, i.e., dataset balancedness. ϕ (Matthews Correlation Coefficient) depends on ρ : with very imbalanced datasets, AUC appears over-optimistic and ϕ over-pessimistic. RRA defines the region of interest in terms of ρ , so all performance metrics depend on ρ . RRA is related to performance metrics via S-shaped curves. Conclusion: Our proposal helps gain a better quantitative understanding of the goodness of a ROC curve, especially in practically relevant regions of interest. Also, showing a ROC curve and iso-PM curves provides an intuitive perception of the goodness of a fault-proneness model.

Binary classifiers are commonly used in software engineering research to estimate several software qualities, e.g., defectiveness or vulnerability. Thus, it is important to adequately evaluate how well binary classifiers perform, before they are used in practice. The Area Under the Curve (AUC) of Receiver Operating Characteristic curves has often been used to this end. However, AUC has been the target of some criticisms, so it is necessary to evaluate under what conditions and to what extent AUC can be a reliable performance metric.

Abstract Context Insufficient code understandability makes software difficult to inspect and maintain and is a primary cause of software development cost. Several source code measures may be used to identify difficult-to-understand code, including well-known ones such as Lines of Code and McCabe’s Cyclomatic Complexity, and novel ones, such as Cognitive Complexity. Objective We investigate whether and to what extent source code measures, individually or together, are correlated with code understandability. Method We carried out an empirical study with students who were asked to carry out realistic maintenance tasks on methods from real-life Open Source Software projects. We collected several data items, including the time needed to correctly complete the maintenance tasks, which we used to quantify method understandability. We investigated the presence of correlations between the collected code measures and code understandability by using several Machine Learning techniques. Results We obtained models of code understandability using one or two code measures. However, the obtained models are not very accurate, the average prediction error being around 30%. Conclusions Based on our empirical study, it does not appear possible to build an understandability model based on structural code measures alone. Specifically, even the newly introduced Cognitive Complexity measure does not seem able to fulfill the promise of providing substantial improvements over existing measures, at least as far as code understandability prediction is concerned. It seems that, to obtain models of code understandability of acceptable accuracy, process measures should be used, possibly together with new source code measures that are better related to code understandability.

Measuring software functional size via standard Function Points Analysis (FPA) requires the availability of fully specified requirements and specific competencies. Most of the time, the need to measure software functional size occurs well in advance with respect to these ideal conditions, under the lack of complete information or skilled experts. To work around the constraints of the official measurement process, several estimation methods for FPA have been proposed and are commonly used. Among these, the International Function Points User Group (IFPUG) has adopted the “High-level FPA” method (also known as the NESMA method). This method avoids weighting each data and transaction function by using fixed weights instead. Applying High-level FPA, or similar estimation methods, is faster and easier than carrying out the official measurement process but inevitably yields an approximation in the measures. In this article, we contribute to the problem of estimating software functional size measures by using machine learning. To the best of our knowledge, machine learning methods were never applied to the early estimation of software functional size. Our goal is to understand whether machine learning techniques yield estimates of FPA measures that are more accurate than those obtained with High-level FPA or similar methods. An empirical study on a large dataset of functional size predictors was carried out to train and test three of the most popular and robust machine learning methods, namely Random Forests, Support Vector Regression , and Neural Networks. A systematic experimental phase, with cycles of dataset filtering and splitting, parameter tuning, and model training and validation, is presented. The estimation accuracy of the obtained models was then evaluated and compared to that of fixed-weight models (e.g., High-level FPA) and linear regression models, also using a second dataset as the test set. We found that Support Vector Regression yields quite accurate estimation models. However, the obtained level of accuracy does not appear significantly better with respect to High-level FPA or to models built via ordinary least squares regression. Noticeably, fairly good accuracy levels were obtained by models that do not even require discerning among different types of transactions and data.

Abstract Context The F-measure has been widely used as a performance metric when selecting binary classifiers for prediction, but it has also been widely criticized, especially given the availability of alternatives such as ϕ (also known as Matthews Correlation Coefficient). Objectives Our goals are to (1) investigate possible issues related to the F-measure in depth and show how ϕ can address them, and (2) explore the relationships between the F-measure and ϕ . Method Based on the definitions of ϕ and the F-measure, we derive a few mathematical properties of these two performance metrics and of the relationships between them. To demonstrate the practical effects of these mathematical properties, we illustrate the outcomes of an empirical study involving 70 Empirical Software Engineering datasets and 837 classifiers. Results We show that ϕ can be defined as a function of Precision and Recall , which are the only two performance metrics used to define the F-measure, and the rate of actually positive software modules in a dataset. Also, ϕ can be expressed as a function of the F-measure and the rates of actual and estimated positive software modules. We derive the minimum and maximum value of ϕ for any given value of the F-measure, and the conditions under which both the F-measure and ϕ rank two classifiers in the same order. Conclusions Our results show that ϕ is a sensible and useful metric for assessing the performance of binary classifiers. We also recommend that the F-measure should not be used by itself to assess the performance of a classifier, but that the rate of positives should always be specified as well, at least to assess if and to what extent a classifier performs better than random classification. The mathematical relationships described here can also be used to re-interpret the conclusions of previously published papers that relied mainly on the F-measure as a performance metric.

Background: Functional size measures of software are widely used for effort estimation, because they provide a fairly objective quantification of software size, which is one of the main factors affecting software development effort. Unfortunately, in some conditions, performing the standard Function Point Analysis process may be too long and expensive. Moreover, functional measures could be needed before functional requirements have been elicited completely and at the required detail level. Aim: Basing effort estimation on measures that are simpler than Function Points---hence, faster and cheaper to obtain---could be beneficial, if good estimation accuracy is possible. In this paper, the level of accuracy that can be obtained using simple measures instead of Function Points---as defined by IFPUG, the International Function Point User Group---is evaluated, based on an empirical study. Method: An analysis of the data provided in the ISBSG dataset was performed. Effort models based on standard IFPUG Function Points and on the number of transactions were derived. The accuracy of the effort estimates delivered by the two models were then compared. Results: Effort models based on the number of transactions appear marginally less accurate than models based on standard IFPUG Function Points for new development projects, and marginally more accurate for projects extending previously developed software. Conclusions: Based on the results of the empirical study, it seems that using the number of transactions instead of the standard IFPUG Function Points measures does not cause estimate accuracy to change to a practically appreciable extent. This conclusion applies to effort models using size measure as the only independent variable. Additional research is necessary to evaluate the performance of effort models based on the number of transactions in combination with other factors.

Background. Open Source Software (OSS) is experiencing an increasing popularity both in industry and in academia. Aim. We investigated models for the selection, evaluation, and adoption of OSS, focusing on factors that affect most the evaluation of OSS. Method. We conducted a Systematic Literature Review of 262 studies published until the end of 2019, to understand whether OSS selection is still an interesting topic for researchers, and which factors are considered by stakeholders and are assessed by the available models. Result. We selected 60 primary studies: 20 surveys and 5 lessons learned studies elicited the motivations for OSS adoption; 35 papers proposed several OSS evaluation models focusing on different technical aspects. This Systematic Literature Review provides an overview of the available OSS evaluation methods, highlighting their limits and strengths, based on the wide range of technicalities and aspects explored by the selected primary studies. Conclusion. OSS producers can benefit from our results by checking if they are providing all the information commonly required by potential adopters. Users can learn how models work and which models cover the relevant characteristics of OSS they are most interested in.

Background. Practical use of software code measures for faultiness estimation often requires setting thresholds on measures, to separate software modules that are likely to be faulty from modules that are likely to be non-faulty. Several threshold proposals exist in the literature. However, different proposals may recommend different threshold values for the same measure, so practitioners may be unsure which threshold value they should use.Objective. Our goal is to investigate whether it is possible to define a single threshold for a code measure, or at least a small range within which to select a threshold.Method. We carried out an empirical study based on two collections of datasets available in the SEACRAFT repository. For each dataset, we built all statistically significant univariate Binary Logistic Regression fault-proneness models, using a set of the most commonly used code measures as independent variables. We then derived thresholds for single software measures by setting an acceptability threshold to fault-proneness. We then checked whether the distribution of the thresholds obtained for the same measure is concentrated enough that it is possible to take a value of the code measure as a “universal” threshold for it. We repeated the same method with bivariate Binary Logistic Regression fault-proneness models.Results. The threshold distributions we obtained are quite dispersed, so it does not seem to be possible to define “universal” thresholds for the code measures we considered.Conclusions. According to the collected evidence, it appears hardly possible to effectively control software faultiness by recommending that the most commonly used code measures satisfy fixed thresholds.

Abstract Software defect prediction models are classifiers often built by setting a threshold t on a defect proneness model, i.e., a scoring function. For instance, they classify a software module non-faulty if its defect proneness is below t and positive otherwise. Different values of t may lead to different defect prediction models, possibly with very different performance levels. Receiver Operating Characteristic (ROC) curves provide an overall assessment of a defect proneness model, by taking into account all possible values of t and thus all defect prediction models that can be built based on it. However, using a defect proneness model with a value of t is sensible only if the resulting defect prediction model has a performance that is at least as good as some minimal performance level that depends on practitioners’ and researchers’ goals and needs. We introduce a new approach and a new performance metric (the Ratio of Relevant Areas) for assessing a defect proneness model by taking into account only the parts of a ROC curve corresponding to values of t for which defect proneness models have higher performance than some reference value. We provide the practical motivations and theoretical underpinnings for our approach, by: 1) showing how it addresses the shortcomings of existing performance metrics like the Area Under the Curve and Gini’s coefficient; 2) deriving reference values based on random defect prediction policies, in addition to deterministic ones; 3) showing how the approach works with several performance metrics (e.g., Precision and Recall) and their combinations; 4) studying misclassification costs and providing a general upper bound for the cost related to the use of any defect proneness model; 5) showing the relationships between misclassification costs and performance metrics. We also carried out a comprehensive empirical study on real-life data from the SEACRAFT repository, to show the differences between our metric and the existing ones and how more reliable and less misleading our metric can be.

Abstract Functional size measures are used in several software development processes, because they can be applied as soon as the functional user requirements of a software application have been specified. Quite often, functional user requirements are used as the main input of effort and cost estimation. Since effort and cost estimates are fundamental for decision making, project managers would like to get reasonably accurate estimates even before user requirements have been specified completely and at the proper level of detail. To this end, early size estimation methods have been proposed. In this paper, we consider a specific situation; namely, we consider processes in which the requirements elicitation and modeling phases are carried out according to the ICONIX process. The ICONIX process involves a few stages, each one tackling an aspect of requirements and delivering a specific artifact, in the form of a UML model. We explore the possibility of deriving from ICONIX artifacts the information needed to estimate the final size of the application. To this end, we analyzed the requirements of 21 software applications and derived statistical models that correlate the information extracted from ICONIX artifacts to the size, expressed in function points, of the application. The analysis of the achieved estimates shows that the approach is viable, in that it provides early estimates with good accuracy.

Background Binary Logistic Regression is widely used in Empirical Software Engineering to build estimation models, e.g., fault-proneness models, which estimate the probability that a given module is faulty, based on some measures of the module. Fault-proneness models are then used to build faultiness model, i.e., models that estimate whether a given module is faulty or non-faulty.

Background. Early identification of software modules that are likely to be faulty helps practitioners take timely actions to improve these modules' quality and reduce development costs in the remainder of the development process. To this end, module faultiness estimation models can be built at any point during development by using measures collected up to that time. Models available in later phases are expected to be more accurate than those available in earlier phases. However, waiting until late in the development process may reduce the impact of the effectiveness and efficacy of any software quality improvement actions and increase their cost.

Background: Technical debt is currently receiving increasing attention from practitioners and researchers. Several metaphors, concepts, and indications concerning technical debt have been introduced, but no agreement exists about a solid definition of technical debt.

Background. Using accurate effort estimation models can help software companies plan, monitor, and control their development process and development costs. It is therefore important to define sound accuracy indicators that allow practitioners and researchers to assess and rank different effort estimation models so that practitioners can select the most accurate, and therefore useful one. Several accuracy indicators exist, with different advantages and disadvantages.

Background. Practical use of a measure X for an internal attribute (e.g., size, structural complexity, cohesion, coupling) of a software module often requires setting a threshold on X, to make decisions as to which software modules may be estimated to be potentially faulty. To keep quality under control, practitioners may want to set a threshold on X to identify "early symptoms" of possible faultiness of a module, which should be closely monitored and possibly modified.

Background. When estimating whether a software module is faulty based on the value of a measure X for a software internal attribute (e.g., size, structural complexity, cohesion, coupling), it is sensible to set a threshold on fault-proneness first and then induce a threshold on X by using a fault-proneness model where X plays the role of independent variable. However, some modules cannot be estimated as either faulty or non-faulty with confidence: they belong to a "grey zone" and estimating them as either would be quite aleatory and may result in several erroneous decisions.

Background. Function Point Analysis is the most used technique for sizing software functional specifications. Function Point measures are widely used to estimate the effort needed to develop software, hence the cost of software. However, Function Point Analysis adopts the point of view of the end user, and-consistently-considers a software application as a whole. This approach does not allow for assessing the role of reusable components in software development. In fact, reusing available components decreases the cost of software development, but standard Function Point measures are not able to account for the savings deriving from component reuse. Objective. We aim at modifying the definition of Function Point Analysis so that the role of components can be taken into account. More specifically, we redefine the measurement so that when no components are used the resulting measure is the same yielded by the standard measurement process, but in presence of components, our modified measure is less than the standard measure (the bigger the role of components, the smaller the measure). Method. Components partly support the realization of elementary processes. Therefore, we split elementary processes into sub-processes, such that each sub-process is either totally supported by a component or it is not supported at all by any component, the size of the elementary process is defined to be inversely proportional to the size of sub-processes supported by components. Results. The proposed approach was applied to a Web application, which was developed in two versions: one from scratch and one using available components. As expected, the 'component-aware' measures obtained are smaller than the standard measures. We also compared the reduction in size with the reduction in development effort. Conclusions. The proposed method proved effective in taking into account the usage of components in the development of the considered application. However, the observed decrease in size is smaller than the decrease of development effort. The latter result suggests that this initial proposal needs further experimentation to support accurate effort estimation.

Background. Under specific circumstances -especially in the early phases of software development projects-a thorough application of the COSMIC method may require more time and effort than available. Thus, early approximate estimation methods have been proposed for estimating the functional size of a given application, instead of properly measuring it. Objective. This paper aims at empirically evaluating the accuracy of two COSMIC early size estimation methods. The goal is to provide practitioners with some empirical evidence on the accuracy that can be expected from these methods. Method. We evaluated the Average Functional Process and the Equal Size Bands methods by applying them to a set of applications that were previously measured according to the standard COSMIC method, and for which the data necessary to perform estimations were readily available. The application conditions and performance of the Equal Size Bands method were also evaluated from a theoretical point of view. Results. Our analyses show that in a few cases the Average Functional Process method features estimation errors that are too large to be acceptable, while on average it provides reasonable estimates. On the contrary, the Equal Size Bands method can provide quite accurate estimates, but only if the human measurer is sufficiently good at classifying each functional process in the correct band. From a theoretical point of view, it is shown that-perhaps counterintuitively- the Average Functional Process and the Equal Size Bands methods provide essentially equivalent estimates when the distribution of Functional Processes across the bands is the same in the historical datasets and in the new software to be estimated. When such distributions are quite different, experimental results show that the Equal Size Bands method performs much better than the Average Functional Process method. Conclusions. Our results show that in a few cases the Average Functional Process method fails to provide acceptably small estimation errors. On the contrary, the Equal Size Bands method is sufficiently accurate to provide good size estimates. However, organizations that plan to use it need to properly train measurers that are able to identify the correct size band in which every functional process belongs.

Simple Function Point is a functional size measurement method that can be used in place of IFPUG Function Point, but requires a much simpler - hence less time and effort consuming - measurement process. Simple Function Point was designed to be equivalent to IFPUG Function Point in terms of numerical results. This paper reports an empirical study aiming at verifying the effectiveness of Simple Function Point as a functional size measurement method, especially suitable to support estimation of software development effort. The data from a large popular public dataset were analyzed to verify the correlation of Simple Function Point with IFPUG Function Point, and the correlation of both size measures to development effort. The results obtained confirm, at a reasonable level of confidence, the hypothesis that Simple Function Point can be effectively used in place of IFPUG Function Point.

Abstract It is well known that the analysis of requirements involves several stakeholders and perspectives. Very often several points of view at different abstraction levels have to be taken into account: all these features make requirements analysis a complex task. Such intrinsic complexity makes it difficult to understand several of the basic concepts that underlie requirements engineering. Actually, there is some confusion – especially in industry – about what really a user requirement is, what are the differences between user requirements and user needs, and what are their relationships with business processes. The paper aims at clarifying the aforementioned issues, by providing a systematic and clear method for establishing requirements hierarchies. The problem of describing requirements hierarchies is tackled using the problem frames concepts and notation. A case study is used throughout the paper to illustrate the proposed approach. The description of requirements at different levels of abstractions and requirements hierarchies are illustrated. The resulting models are coherent with the reference model for requirements specifications and the problem frames. An analysis process that is aware of the differences between user needs and requirements is also provided, to illustrate the process of refining high‐level goals into requirements that can be satisfied by a hardware/software machine. The proposed method appears promising to model, study, and evaluate the relationships between business processes and the strategies for achieving business goals based on the usage of information technology.

The measurement of Function Points is based on Base Functional Components. The process of identifying and weighting Base Functional Components is hardly automatable, due to the informality of both the Function Point method and the requirements documents being measured. So, Function Point measurement generally requires a lengthy and costly process. We investigate whether it is possible to take into account only subsets of Base Functional Components so as to obtain functional size measures that simplify Function Points with the same effort estimation accuracy as the original Function Points measure. Simplifying the definition of Function Points would imply a reduction of measurement costs and may help spread the adoption of this type of measurement practices. Specifically, we empirically investigate the following issues: whether available data provide evidence that simplified software functionality measures can be defined in a way that is consistent with Function Point Analysis; whether simplified functional size measures by themselves can be used without any appreciable loss in software development effort prediction accuracy; whether simplified functional size measures can be used as software development effort predictors in models that also use other software requirements measures. We analyze the relationships between Function Points and their Base Functional Components. We also analyze the relationships between Base Functional Components and development effort. Finally, we built effort prediction models that contain both the simplified functional measures and additional requirements measures. Significant statistical models correlate Function Points with Base Functional Components. Basic Functional Components can be used to build models of effort that are equivalent, in terms of accuracy, to those based on Function Points. Finally, simplified Function Points measures can be used as software development effort predictors in models that also use other requirements measures. The definition and measurement processes of Function Points can be dramatically simplified by taking into account a subset of the Base Functional Components used in the original definition of the measure, thus allowing for substantial savings in measurement effort, without sacrificing the accuracy of software development effort estimates.

Background. Outliers and corrupted data points may unduly bias software development effort estimation models. However, given the usually limited size of software engineering data sets, removing too many data points may seriously reduce the power of the statistical tests used and the likelihood of statistically significant result. Also, statistical techniques are typically based on assumptions that are either believed to be true a priori or, at best, checked via statistical tests, without ever achieving 100% certainty on their truthfulness. Estimation models based on less strict assumptions have broader applicability and lower risks of drawing unwarranted conclusions. Aim. We investigate the usefulness of Robust Regression when building effort estimation models, by varying the degree of robustness and, thus, the number of data points that are excluded from the data analysis as outliers. Method. We have used Least Quantile of Squares (LQS) Robust Regression, a generalization of the Least Median of Squares (LMS). LMS builds a regression line by minimizing the median squared residual. LQS minimizes the order statistic of square residuals corresponding to any specified quantile, and not just the median, which is the order statistic corresponding to the 50% quantile. We have extended a statistical significance test for univariate LQS regression models. We have also built a weighted model, obtained from statistically significant LQS models, where each LQS model contributes proportionally to the quantile used. Results. We have applied LQS Linear Regression to estimate development effort on four projects from the PROMISE data set and obtained valid and significant univariate models. Conclusions. LQS may provide a valid alternative to LMS and Ordinary Least Square regressions to build estimation models when (1) balancing the need for excluding outliers and keeping enough data points to build statistically significant models and (2) using less strict assumptions underlying the regression technique.

The quantification of several software attributes (e.g., size, complexity, cohesion, coupling) is usually carried out in a static fashion, and several hundreds of measures have been defined to this end. However, static measurement may only be an approximation for the measurement of these attributes during software use. The paper proposes a theoretical framework based on Axiomatic Approaches for the definition of sensible dynamic software measures that can dynamically capture these attributes. Dynamic measures based on this framework are defined for dynamically quantifying size and coupling. In this paper, we also compare dynamic measures of size and coupling against well-known static measures by correlating them with fault-pronenesses of four case studies.

Background: Several Functional Size Measurement methods have been introduced after the success of Function Point Analysis, thus originating the problem of converting Function Point measures into other units. To this end, several methods -- ranging from statistical analysis to "manual" conversion -- have been used. However, none of the proposed conversion methods guarantees the necessary accuracy. Objectives: The objective of the work reported here is to define a seamless and cheap procedure that allows measurers to derive functional size measures expressed in COSMIC Function Points from size measures expressed in Function Points. Method: To get accurate conversions, we propose to exploit all the available information provided by the measurement process, that is, not only the size in Function Points, but also the details of basic functional components. To make the procedure efficient, a mapping of Function Point Analysis concepts onto COSMIC concepts is proposed. Results: A conversion procedure based on the aforementioned mapping is proposed. Such procedure is supported by a software tool that eases the conversion process. The usage of both the procedure and tool is illustrated via an example of realistic complexity. Conclusion: The proposed procedure and tool can be effectively used to perform the required measurement with very good accuracy. The tool can also be used to perform a sort of "double" measurement, i.e., both Function Points and COSMIC Function Points are measured at the same time.

Historical datasets, or Historical Data Repositories (HDR) are important in software engineering, because they support a variety of useful analyses for benchmarking as well as estimation process purposes. Besides private repositories, some public domain data repositories - such as the PROMISE and the ISBSG datasets - have been available for over 15 years. Regardless of their importance, such repositories evolved more in amount of data points, than quality or conformity to commonly agreed practices for data collection and verification. This paper provides some considerations about current issues and possible improvements of (public) HDR's.

Background. Open Source Software (OSS) provides increasingly serious and viable alternatives to traditional closed source software. The number of OSS users is continuously growing, as is the number of potential users that are interested in evaluating the quality of OSS. The latter would greatly benefit from simple methods for evaluating the trustworthiness of OSS.

Functional Size Measures are often used for effort estimation. However, these measures do not take into account the amount and complexity of elaboration required, concentrating instead on the amount of data accessed or moved. Methods for measuring the functional complexity have been proposed, but, being based on the textual description of requirements, are not very easy to apply. In this paper we show that measurement-oriented UML modeling can support the measurement of both functional size and functional complexity. We show, by means of a case study, that it is reasonably easy to derive different types of functional size measures, as well as complexity measures, from UML models. We show also that it is possible to build models for effort estimation that use the functional size and complexity measures as independent variables.

Problem frames are an approach to requirements modeling that is gaining increasing attention and popularity. A few meta-models have already been proposed to precisely define the notation and -in some cases- to support the construction of tools. However, the meta-models proposed till now concentrate on modeling the single problem frame, without addressing the whole problem. This is particularly limiting, since one of the strengths of the problem frames approach is in tackling the complexity of problems by guiding their decomposition into sub-problems, and in identifying recurring patterns of elementary problems. In order to support this fundamental activity, a meta-model has to support the notion of sub-problems and sub-domains, moreover it must provide suitable means to represent problems in intermediate situations during the (de)composition phases. This paper presents a meta-model that solves the aforementioned problems and provides a suitable base for the construction of a tool based on the EMF/GEF technology.

The developers of real-time and embedded software face - just like the developers of other types of software - the problem of estimating the cost of development. To this end, the most widely used methods and tools require that the functional size of the program to be developed is measured. However, the functional size measurement methods available - namely, Function Point Analysis and its evolutions - are traditionally considered not well suited for representing the functionality of real-time and embedded software. Actually, the problem is that the definition of Function Points and their counting rules make reference almost exclusively to traditional "business" software. In this paper, the problem of applying FPA to embedded and real-time software is tackled. A set of hints and examples - derived from industrial experience - are given, supporting the application of standard function point counting to real-time and embedded software. It is then shown that the obtained measures successfully supported the estimation of a set of programs in the avionics domain.

The approach to requirements modelling based on problem frames is gaining increasing attention and popularity, especially when the decomposition of complex problems into simpler recurring problems (the ldquoframesrdquo) is concerned. However, the usage of problem frames in industrial software development processes requires that management practices like cost estimation are supported. The COSMIC method for measuring the functional size of software is gaining consensus as a means for sizing the software at the stage of requirements specification. Data are being collected to support the construction of effort estimation models that take the COSMIC size as input. This paper explores the possibility of applying the COSMIC measurement method to user requirements represented using problem frames. To this end, problems are modelled using problem frames, both in their native notation and using UML; the COSMIC concepts are mapped onto the elements of problem frames; a procedure is defined to apply COSMIC compliant measurement rules to requirements models built according to the Problem Frames method.

Problem frames are an approach to requirements modeling that is gaining increasing attention and popularity. The approach provides useful concepts and methodological guidelines. However, problem frames are not equipped with an expressive and complete notation and they lack tools support. These limitations can be addressed by introducing a suitable meta-model to formally define the notation. In this way it is also possible to identify the aspects that are not covered by the problem frames notation and to provide hooks for user-defined extensions. The meta-model is also expected to support the underlying analysis methodology. Furthermore, it can provide the basis for building a tool supporting both the editing of problem frames and the other activities associated with the approach (frame concern, composition, etc.). This paper presents a meta-model that addresses the former issues and was used for building a tool with the EMF/GMF technology.

This article summarizes our effort, since 2004 up to the present time, for improving the current industrial Systems-on-Chip and Embedded Systems design by joining the capabilities of the unified modeling language (UML) and SystemC/C programming languages to operate at system-level. The proposed approach exploits the OMG model-driven architecture—a framework for Model-driven Engineering—capabilities of reducing abstract, coarse-grained and platform-independent system models to fine-grained and platform-specific models. We first defined a design methodology and a development flow for the hardware, based on a SystemC UML profile and encompassing different levels of abstraction. We then included a multithread C UML profile for modelling software applications. Both SystemC/C profiles are consistent sets of modelling constructs designed to lift the programming features (both structural and behavioral) of the two coding languages to the UML modeling level. The new codesign flow is supported by an environment, which allows system modeling at higher abstraction levels (from a functional executable level to a register transfer level) and supports automatic code-generation/back-annotation from/to UML models.

Function Point Analysis (FPA) is the most widely used method for measuring the size of software requirements, usually for the purpose of cost estimation. Unfortunately, FPA is affected by several drawbacks: it must be performed by specifically skilled personnel, it is expensive, and the resulting measures are subject to high variability. In order to solve –at least partially– these problems, researchers have proposed to base FP counting on UML models. However, models built without having FPA in mind hardly provide the required information at the proper detail level, so that the measures of the models tend to vary accordingly. On the contrary, building models that are suitable for FPA generally requires additional notations, skills and effort, thus partly spoiling the advantages of the approach. This paper illustrates a technique for building FPA-oriented UML models that do not need to include more information than usually required by the development process, and are easy to measure. As a result, FPA can be performed in a seamless way, while yielding reliable results. The proposed technique was validated by means of a controlled experiment and a set of pilot applications, which are also briefly described in the paper.

Abstract: Problem frames are an approach to requirements modelling that is gaining increasing attention and popularity, as it addresses the essential activity of understanding and describing the problem to be solved before undertaking the actual development phases. This paper reports a study concerning the possibility of enhancing the problem frames methodology with concepts derived from requirements modelling techniques based on scenarios and histories. The goal of this research is to make problem frames even more appealing to software developers, who are generally familiar with the ideas underlying scenario‐based modelling. The results presented here are encouraging: a well‐known problem, often used to illustrate problem frames (the sluice gate control), was studied with the help of scenarios, modelled by means of histories. Scenarios were also employed to support the merging of subproblems, for addressing the frame concern issues and for supporting design models. In all these activities the proposed approach proved helpful.

Recently, SysML has been adopted by the Object Management Group as a modelling language for Systems Engineering. SysML is a UML profile that represents a subset of UML 2 with extensions. A wide adoption of the language could be hindered by the lack of a methodology that drives the modelling activities. Problem Frames (PFs) are a rigorous approach to requirements modelling that has the potential to improve the software development process. Unfortunately, PFs are not supported by an intuitive notation and easy to use tools. As a consequence, their adoption in industry is limited. This paper explores the possibility of exploiting the PFs ideas in the context of SysML models. The goal is to provide model-based development processes using SysML with a set of concepts and guidelines that are sound and have already been used and validated.

Potential users of Open Source Software (OSS) face the problem of evaluating OSS, in order to assess the convenience of adopting OSS instead of commercial software, or to choose among different OSS proposals. Different metrics were defined, addressing different OSS properties: the Total Cost of Ownership (TCO) addresses the cost of acquiring, adapting and operating OSS; the Total Account Ownership (TAO) represents the degree of freedom of the user with respect to the technology provider; indexes like the Open Business Quality Rating (Open BQR) assess the quality of the software with respect to the user's needs. However, none of the proposed methods and models addresses all the aspects of OSS in a balanced and complete way. For this purpose, the paper explores the possibility of adapting the Balanced Scorecard (BSC) technique to OSS. A preliminary definition of the BSC for OSS is given and discussed.

Some of the problems with process measurement are generally due to the fact that the definition of measurement plans does not rely on a reference model of the development process that can drive and explain the measuring activities. One of the most popular methodologies addressing the definition of process measurement plans is the GQM (Goal/Question/Metrics). This paper discusses how to support the creation of GQM plans by means of an explicit model of the process being measured. Such a model guides the GQM process, and makes it possible to define precisely –if not formally– the metrics involved. A tool supporting the proposed method is also illustrated.

Professional operators who deal with the cultural goods of Italy constantly face the problem of retrieving the needed information from heterogeneous sources. Such information has also to be combined and elaborated; this operation is made difficult by the heterogeneity of the data models and vocabularies employed at the different information sources. This paper illustrates the architecture of a Web-based system that provides users with a unified view of the metadata describing the information available from the various sources. In this way the user can navigate through and choose among homogeneous virtual descriptions that finally bring him/her to the sought real data. The peculiarities of the illustrated system are: it is built exclusively employing open source software, it allows the data sources to maintain their "independence", i.e., they remain the unique managers of their own data, it supports different virtual communities of users, each with its own vocabulary and ontology.

UML is being increasingly used to model real-time software. On one hand this is reasonable, since UML is very popular and relatively easy to use. On the other hand, the semantics of UML is not well defined, thus UML does not support formal analysis, which is needed to prove properties like safety, utility, liveness, etc. This article describes a way to make UML models formally verifiable. The presented approach is made possible by extending UML in order to represent time-dependent information and time constraints, and by formalizing the resulting language. The formalization is achieved by mapping UML state diagrams to Timed Statecharts. UML state models are translated into timed automata, so that the model checking tool Kronos can be employed to verify time-dependent properties. A central issue of the work presented is that developers can take advantage of the formal methods being employed while skipping the complex and expensive formal modeling phase.

In the recent past, a relevant effort has been devoted to the definition of process modeling languages (PMLs). The resulting languages and environments –although technically successful– did not receive much attention from industry. On the contrary, researchers and practitioners have recently started experimenting with the usage of UML as a PML. Being so popular and widely used, UML has an important competitive advantage compared to any specialized PML. However, it has also a main limitation. While most PMLs are executable by some process engine, UML was conceived as a non-executable, semi-formal language. The work described here aims at assessing the possibility of employing a subset of UML as an executable PML. The article proposes a formalization of the semantics of the UML subset and presents the translation of UML process models into code, which can be enacted in the OPSS process-centered environment. The paper also presents a case study to validate the approach. We expect that process modeling by means of UML is easier and available to a larger community of software process managers. Moreover, process enactment makes the process more efficient, reliable, predictable and controllable, as widely shown by previous research.

This article explores a dual approach to real-time software development. Models are written in UML, as this is expected to be relatively easy and economic. Then models are automatically translated into a formal notation that supports the verification of properties such as safety, utility, liveness, etc. In this way, developers can exploit the advantages of formal notations while skipping the complex and expensive formal modelling phase. The proposed appraoch is applied to the Generalised Railroad Crossing (GRC) problem, one of the best known benchmarks porposed in the literature. A UML model of the GRC is built, and then translated into TRIO (a first order temporal logic). The resulting specification properties are tested by a history checking tool which exploits the formality of TRIO. The work described here highlights the shortcomings of UML as a real-time modelling language, proposes enhancements and workarounds to overcome UML limitations, and demonstrates the viability of using UML as a front-end for a formal real-time notation. By translating the GRC model into TRIO, we also give formal semantics to some of the UML constructs.

Software developers are increasingly exploiting the Web as a document management system. However, the Web has some limitations, since it is not aware of the structure and semantics associated to pieces of information (e.g., the fact that a document is a requirement specification) and of the semantics of relationships between pieces of information (e.g., the fact that a requirement specification document may be associated to some design specification document). In the Labyrinth project we enhance the capabilities of the Web as a document management system by means of a semantic model (called schema, in analogy with database schemas), which is associated to Web documents. This model is itself a Web document and can be accessed and navigated through a simple Web browser.

Goal/Question/Metric GQM) is a paradigm for the systematic definition, establishment, and exploitation of measurement programs supporting the quantitative evaluation of softare processes and products. Although GQM is a quite well-known method, detailed guidelines for establishing a GQM program in an industrial environment are still limited. Also, there are few reported experiences on the application of GQM to industrial cases. Finally, the technological support for GQM is still inadequate. This article describes the experience we have gained in applying GQM at Digital Laboratries in Italy. The procedures, experiences, and technology that have been employed in this study are largely reusable by other industrial organizations willing to introduce a GQM-based measurement program in their development environments.

Configuration Management (CM) is widely recognized as one of the key practices in software development, specially as far as the development of big and complex software systems is concerned. Therefore, tools supporting CM have been constantly evolving, and are now sophisticated software products, providing many features, including the possibility to model-to some extent-the work-flow concerning the configuration items. Process-Centered Software Engineering Environments (PSEEs) aim at providing development environments where production activities are carried out according to a given model of the software process. Since CM has a fundamental role in any software process, it is interesting to understand to what extent a PSEE can be used to support a process model which includes CM issues, and to compare the PSEE-based environments with state-of-the-art commercial CM tools. The work described here is a first step towards this goal, concerning the development of a specific CM environment (featuring a subset of the functionalities provided by CCC/Harvest) based on the SPADE PSEE. The results reported can however be generalized to several CM tools and PSEEs.

This paper presents an experience in software process assessment that has been conducted in a mid-size Italian company.The assessment has been carried out using the CMM(Capability Maturity Model) and taking into account also the indications offered by QIP (Quality Improvement Paradigm).The paper discusses the resr.dtso ftheassessmen tandthe lessons we have learned from running it.It particular, it argues that it is necessa~to broaden the scope of assessment methods, and to evaluate the applicability to software processes of the experiences, methods, and techniques developed in other business domains.

The paper discusses the problems that a software development organization must address in order to assess and improve its software processes. In particular, the authors are involved in a project aiming at assessing and improving the current practice and the quality manual of the Business Unit Telecommunications for Defense (BUTD) of a large telecommunications company. The paper reports on the usage of formal process modeling languages to detect inconsistencies, ambiguities, incompleteness, and opportunities for improvement of both the software process and its documentation.>

A method for designing and prototyping program construction systems using relational databases is presented. Relations are the only data structures used inside the systems and for interfaces; programs extensively use relational languages, in particular relational algebra. Two large projects are described. The Ada Relational Translator (ART) is an experimental compiler-interpreter for Ada in which all subsystems, including the parser, semantic analyzer, interpreter, kernel, and debugger, use relations as their only data structure; the relational approach has been pushed to the utmost to achieve fast prototyping in a student environment. Multi-Micro Line (MML) is a tool set for constructing programs for multimicroprocessors' targets, in which relations are used for allocation and configuration control. Both experiences validate the approach for managing teamwork in evolving projects, identify areas where this approach is appropriate, and raise critical issues.>