ABSTRACT Methodology In this research, we conducted a case study to establish a foundation for size measurement and effort estimation in DWH projects. We first applied a productivity‐based estimation approach using linear regression with the ISBSG repository to assist organizations without historical data. We then evaluated various machine learning algorithms to improve estimation accuracy. Finally, we tested a combined model that integrates both approaches for estimating effort in external projects. Results Using the ISBSG dataset, linear regression models based on productivity achieved a Mean Magnitude of Relative Error (MMRE) of 0.285. Machine learning algorithms improved accuracy by 22.81%, reducing the MMRE to 0.220. The final model, applied to external projects, yielded MRE values between 0.010 and 0.245. Conclusion The ISBSG repository is a valuable resource for effort estimation in DWH projects. Combining productivity‐based estimation with machine learning enhances accuracy and predictive performance, making it a more reliable approach than traditional models.

Software Size Measurement (SSM) plays an essential role in software project management as it enables the acquisition of software size, which is the primary input for development effort and schedule estimation. However, many small and medium-sized companies cannot perform objective SSM and Software Effort Estimation (SEE) due to the lack of resources and an expert workforce. This results in inadequate estimates and projects exceeding the planned time and budget. Therefore, organizations need to perform objective SSM and SEE using minimal resources without an expert workforce. In this research, we conducted an exploratory case study to predict the functional size of software project requirements using state-of-the-art large language models (LLMs). For this aim, we fine-tuned BERT and BERT_SE with a set of user stories and their respective functional size in COSMIC Function Points (CFP). We gathered the user stories included in different project requirement documents. In total size prediction, we achieved 72.8% accuracy with BERT and 74.4% accuracy with BERT_SE. In data movement-based size prediction, we achieved 87.5% average accuracy with BERT and 88.1% average accuracy with BERT_SE. Although we use relatively small datasets in model training, these results are promising and hold significant value as they demonstrate the practical utility of language models in SSM.

Effort estimation is a fundamental task during the planning of software projects. Prediction models usually rely on two essential factors: software size and effort data. Measuring the size of the software can be done at various stages of the project with desired accuracy. Nevertheless, the industry faces challenges when it comes to collecting reliable actual effort data. Consequently, organizations encounter difficulties in establishing effort prediction models. Benchmarking datasets are available, but, in most cases, they have huge variances that make them less useful for effort prediction. In this study, we aimed to answer whether creating a software benchmarking dataset is possible by gathering the data from the literature. To the best of our knowledge, a comprehensive dataset that gathers the functional size and effort data of the studies from the literature is unavailable. For this purpose, we performed a systematic literature review to find studies that include projects measured with the COSMIC Functional Size Measurement (FSM) method and the related effort. As a result, we formed a dataset including 337 records from 18 studies that shared the corresponding size and effort data. Although we performed a limited search, we created a larger dataset than many datasets in the literature. In light of our review, we obtained that most studies did not share their dataset, and many lacked case details such as implementation environment and the scope of software development life cycle activities included in the effort data. We also compared the dataset with the ISBSG repository and found that our dataset has less variation in productivity. Our review showed the applicability of creating a software benchmarking dataset is possible by gathering the data from the literature. In conclusion, this study addresses gaps in the literature through a cost-free and easily extendable dataset.

Abstract Industry 4.0 changes traditional manufacturing relationships from isolated optimized cells to fully integrated data and product flows across borders with its technological pillars. However, the transition to Industry 4.0 is not a straightforward journey in which organizations need assistance. A well‐known approach that can be utilized during the early phases of the transition is to assess the capability of the organization. Maturity models are frequently used to improve capability. In this systematic literature review (SLR), we analyzed 22 maturity and readiness models based on 10 criteria: year, type, focus, structure, research methodology followed during the design of models, base frameworks, tool support, community support, objectivity, and extent of usage in practice. Our SLR provides a well‐defined comparison for organizations to choose and apply available models. This SLR showed that (1) there is no widely accepted maturity/readiness model for Industry 4.0, as well as no international standard; (2) only a few models have received positive feedback from the industry, whereas most do not provide any practical usage information; and (3) the objectivity of the assessment method is controversial in most of the models. We have also identified a number of issues as open research areas for assessing readiness and maturity models for Industry 4.0.

The rapid evolution of cloud technology has led to the development of highly responsive and flexible applications. Microservice-based Architecture (MSbA) is a popular approach for creating cloud-compatible applications. However, MSbA presents challenges in system verification and analysis. This paper explores using Petri nets for modelling microservice-based systems. The proposed method utilises abstraction and composition techniques to model complex systems using 1-safe Petri nets. A case study on Banking as a Service (BaaS) demonstrates the suitability of Petri nets for modelling, analysis and verification of microservice-based systems.

Software project management plays an important role in producing high-quality software, and effort estimation can be considered as a backbone for successful project management. Size is a very significant attribute of software by being the only input to perform early effort estimation. Even though functional size measurement methods showed successful results in effort estimation of traditional data-centric architectures such as monoliths, they were not designed for today’s architectures which are more service-based and decentralized such as microservices. In these new systems, the event concept is highly used specifically for communication among different services. By being motivated by this fact, in this study, we looked for more microservice-compatible ways of sizing microservices using events and developed a method accordingly. Then, we conducted an exploratory case study in an organization using agile methods and measured the size of 17 Product Backlog Items (PBIs) to assess how this proposed method can be useful in effort estimation in microservices. The implication from the case study is that despite performing a more accurate effort estimation using the proposed size measurement than COSMIC, we were unable to significantly outperform using the total number of events. However, our suggested approach demonstrated to us a different way to use software size in terms of events, namely, to determine the coupling complexity of the project. This finding can be beneficial specifically when evaluating the change requests.

Functional size measurement (FSM) methods, by being systematic and repeatable, are beneficial in the early phases of the software life cycle for core project management activities such as effort, cost, and schedule estimation. However, in agile projects, requirements are kept minimal in the early phases and are detailed over time as the project progresses. This situation makes it challenging to identify measurement components of FSM methods from requirements in the early phases, hence complicates applying FSM in agile projects. In addition, the existing FSM methods are not fully compatible with today’s architectural styles, which are evolving into event-driven decentralized structures. In this study, we present the results of a case study to compare the effectiveness of different size measures: functional -COSMIC Function Points (CFP)-, event-based - Event Points-, and code length-based - Line of Code (LOC)-on projects that were developed with agile methods and utilized a microservice-based architecture. For this purpose, we measured the size of the project and created effort estimation models based on three methods. It is found that the event-based method estimated effort with better accuracy than the CFP and LOC-based methods.

Organizations may create a sustainable competitive advantage against competitors by using data warehouse systems with which they can assess the current status of their operations at any moment. They can analyze trends and connections using up-to-date data. However, data warehouse projects tend to fail more often than other projects as it can be tough to estimate the effort required to build a data warehouse system. Functional size measurement is one of the methods used as an input for estimating the amount of work in a software project. In this study, we formed a measurement basis for DWH projects in an organization based on the COSMIC Functional Size Measurement Method. We mapped COSMIC rules on two different architectures used for DWH projects in the organization and measured the size of the projects. We calculated the productivity of the projects and compared them with the organization’s previous projects and DWH projects in the ISBSG repository. We could not create an organization-wide effort estimation model as we had a limited number of projects. As an alternative, we evaluated the success of effort estimation using DWH projects in the ISBSG repository. We also reported the challenges we faced during the size measurement process.

The rapidly developing internet infrastructure together with the advances in software technology has enabled the development of cloud-based modern web applications that are much more responsive, flexible, and reliable compared to traditional monolithic applications. Such modern applications require new software design paradigms and architectures. Microservice-based architecture (MSbA), which aims to create small, isolated, loosely-coupled applications that work in cohesion, becoming widespread as one of these approaches. MSbA allows the developed applications to be deployed and maintained separately, as well as scaled on demand. However, there is no de facto method for the analysis and design of systems for these architectures. In this paper, we compared the usefulness of the object-oriented (OO) and event-oriented (EO) approaches for analyzing and designing MS-based systems. More specifically, we performed an exploratory case study to analyze, design, and implement a software application dealing with the ‘application and evaluation process of graduate students at IzTech’. This paper discusses the results of this case study. We observe that the EO approaches have significant advantages with respect to the OO approaches.

Functional size measurement (FSM) has been used in software engineering for decades as a main driver for estimation and significant input for other various project management activities throughout the project life span. To apply FSM accurately at the early stages of software development process, especially for estimation purposes, functional user requirements need to be available in detail as required by the adopted FSM method. However, in agile software development, requirement specifications, in general, are kept minimal. For this reason, the adjustment of the requirements to the necessary granularity level has been articulated as one of the barriers preventing the diffusion of FSM practices among agile teams. In this paper, we take a closer look at this problem in order to investigate the usability of FSM and to reveal FSM related challenges empirically through case studies on real agile projects from different software organizations. This study also provides a snapshot of agile organizations in terms of requirement specification and estimation related practices.

Software project scope defines functional and non-functional requirements. These requirements may change to satisfy the customers' needs. However, the control of scope creep represents one of the success keys in software project management. Changes in non-functional requirements affect the ISO/IEC 25010 quality characteristics such as security, portability, etc. Furthermore, some of these quality characteristics may evolve throughout the software life cycle into functional requirements. In this paper, we explore the use of COSMIC method - ISO/IEC 19761 to quantify and evaluate security change requests. Measuring the functional size of security change requests allows stakeholders to make appropriate decisions about whether to accept, defer, or deny the change.

Software engineering researchers have, over the years, proposed different critical success factors (CSFs) which are believed to be critically correlated with the success of software projects. To conduct an empirical investigation into the correlation of CSFs with success of software projects, we adapt and extend in this work an existing contingency fit model of CSFs. To archive the above objective, we designed an online survey and gathered CSF-related data for 101 software projects in the Turkish software industry. Among our findings is that the top three CSFs having the most significant associations with project success were: (1) team experience with the software development methodologies, (2) team’s expertise with the task, and (3) project monitoring and controlling. A comprehensive correlation analysis between the CSFs and project success indicates positive associations between the majority of the factors and variables, however, in most of the cases at non-significant levels. By adding to the body of evidence in this field, the results of the study will be useful for a wide audience. Software managers can use the results to prioritize the improvement opportunities in their organizations w.r.t. the discussed CSFs. Software engineers might use the results to improve their skills in different dimensions, and researchers might use the results to prioritize and conduct follow-up in-depth studies on those factors.

The analysis, design, implementation and testing of software for embedded systems are not trivial. Software modeling is a commonly used approach in the embedded software industry to manage complexity of these phases. The modeling approaches vary since the characteristics of modeling such as its purpose, the medium type used, the lifecycle phase used, differ among systems and industrial sectors. Our previous research identified and defined the modeling approach patterns in embedded software development projects based on quantitative data. In this paper, to validate and improve the pre-investigated pattern set, we present a series of semi-structured interviews over eight months with 53 embedded software professionals across a variety of target industrial sectors and roles. With the help of these interviews, the different modeling approach patterns in embedded software development were better understood and the hidden patterns not evident in the previous study were identified along with a documentation of personalized modeling experiences.

Enterprise Resource Planning (ERP) projects have special characteristics that differentiate them from other kinds of software projects. Main difference that affects effort estimation results is high reuse rates of ERP projects. This paper presents our exploratory work to establish an approach to calculate reuse reflective size of ERP projects, which could be a primary input for effort estimation. We explored the usability of COSMIC function points as the base unit and its convertibility to reuse reflective size using reuse levels. We have performed a case study on an SAP Implementation project. We were able to calculate COSMIC size and reuse levels using the documents available for the project. We discuss the results as well as challenges and opportunities in the light of the case study.

Agile software development approaches have emerged to overcome the inherent challenges of the traditional software development methodologies. With that in mind, they offer a set of simple yet effective practices based on the values and supporting principles of the Agile Manifesto. Today, more and more organizations are practicing Agile but transitioning to Agile is not as straightforward as it might seem. The benefits afforded by the agile values, principles and practices are not fully obtained and adapters struggle to become more mature; i.e. increase their Agility. For, adapters to address this problem, agility assessments have been designed to be aware of the situation of their transition and make sure that they are heading in the right direction. However, these assessments still require significant amount of time and effort to apply. To overcome this problem, we have developed a web-based agility assessment tool which is a part of an integrated agility assessment approach. The approach consists of a structured and comprehensive agility assessment model (AgilityMod), an exemplar agility assessment process and a web based agility assessment tool (AssessAgility). This research focuses on the final component of the approach: AssessAgility tool and presents a multiple case study that was conducted to evaluate tool's capabilities in terms of providing guidance, automation and reducing the effort spent on the assessments.

To cope with growing complexity of embedded software, modeling has become popular. The usage of models in embedded software industry and the relevant practices usually vary since the purposes of diagram development and usage differ. Since a large variety of software modeling practices used in embedded software industry, it is important to understand its state-of-the-practice and its usage degree while investigating the relations between its attributes (e.g., modeling rigor, purpose, code correspondence, stakeholder, medium used while modeling, etc). To achieve this, we have designed and conducted a survey in our earlier work. In this paper, we present a conceptual model of development and usage for software modeling that is based on the findings of this survey and incorporates expert opinions. The conceptual model, which characterizes the attributes of a diagram development and usage, will help to express the meaning of terms used by domain experts to discuss the problems and find the relationships between these attributes.

Process ontologies contain organization's process elements which are related to the business activities, in a structured format. Once organizations establish process ontologies, searching process information as well as discovery, modeling and automation become more efficient. Process ontologies are frequently developed from organizational resources such as guidelines, regulations and procedures by interpreting the document, discovering the process information and building the ontology. The studies depict that the development of process ontologies can be partially automated. However, a significant success factor in this automation is the language of the document and the tools available to interpret the written text in a given language. In this study, we conducted an exploratory case study in order to analyze the potential of process ontology discovery in different languages. We applied the process ontology discovery method we have developed for guidelines written in English and Turkish and compared the success rates. We also discuss the requirements for process discovery automation.

Enterprise Resource Planning (ERP) systems are large scale integrated systems covering most of the business processes of an enterprise. ERP projects differ from software projects with customization, modification, integration and data conversion phases. Most of the time effort and time estimations are performed in an ad-hoc fashion in ERP projects and as a result they frequently suffer from time and budget overruns. Although there is no consensus on a methodology to estimate size, effort and cost of ERP projects there are various research studies in the field. The purpose of this paper is to review the literature on effort estimation methods for ERP projects, their validations and limitations. The systematic literature review used online journal indexes between January 2000 and December 2016. Studies focusing on effort estimation for ERP projects were selected. Two reviewers assessed all studies and 41 were shortlisted. In most of the studies, cost factors for ERP projects were investigated and validated. Our findings showed that effort estimation methods have mostly used function points as an input. Validations of these methods were mostly done by using history-based validation approaches.

Agile software development methodologies have been widely adopted by the software industry during the last decade. Agility assessment is an approach to measure the success of this adoption as well as to satisfy the further demands. In response, a number of agile maturity self-assessment surveys have been developed. However, software organizations do not widely utilize existing self-assessment surveys. In this study we aim to identify the existing surveys and evaluate their strengths and weaknesses in agility assessment. Firstly, an exploratory case study is performed by using one of the surveys in order to determine the expected features of an agile maturity self-assessment survey. Then, a comprehensive case study is conducted to measure the sufficiency of the mostly referenced agile maturity self-assessment surveys according to the determined features. The study results reveal that current surveys do not meet the expected features comprehensively.

Enterprise Resource Planning (ERP) Systems evolve at a rapid pace based on customer and industry expectations. As a result, historic project data for these kinds of projects lose their value especially for analogy based estimation methods. In this rapidly evolving domain, function point based methods might provide a sound alternative for ERP effort estimation. This paper presents the results obtained by applying three methods published in the literature in which function points are used as an input for ERP effort estimation. The evaluation of these methods with respect to their measurement processes and estimation errors are presented; their advantages and disadvantages are discussed.

Agile methodologies have gained significant popularity among software development organizations during the last decade. Although agile methodologies are regarded as minimizing formal processes, they still utilize an estimation methodology for proper management. Story point is the most common input for agile effort estimation. Story point is an arbitrary measure; it reflects experiences of project participants. On the other hand, functional size is an alternative measure used in practice as an input for effort estimation. In this research, we collect and present the outcomes of three case studies which compared the effectiveness of COSMIC-based and story point based effort estimation in agile context. On selected projects of these organizations, software functional size was measured with COSMIC functional size measurement methodology. Effort prediction models were formed by using COSMIC size and actual effort spent; and the models were tested in terms of their effectiveness. The results show controversial outcomes. For all the cases, COSMIC based estimation was more precise. Therefore, COSMIC is an appropriate measure to estimate the effort in organizations that adopt agile software development. It is also observed that COSMIC allowed for computing productivity which has less disperse distribution than the productivity computed with SP. The data is also provided to help other researchers conduct their own studies.

Among the various approaches to software size measurement, Functional Size Measurement has been widely recognized for its usability in early phases of software development cycles and its independence from implementation language, development method and technology. Starting from its introduction with the original Function Point Analysis method in 1979, functional size has been a favored input to estimation and productivity models. As a result of the search for solutions to emerging measurement needs and the advancements in the discipline of software measurement, FSM concepts have been redefined and measurement methods have matured with notable contributions from the ISO standardization process. Despite the progress towards an unambiguously defined and versatile measure in software engineering, several misconceptions about FSM in software community keep on leading to misuse of functional size and unproductive measurement practices. While such misperceptions results in disappointment and wasted resources, an important consequence is the disinterest in FSM. In this paper, we elaborate seven misconceptions in FSM. We review functional size and FSM by discussing the misconceptions. Our purpose is to give a state-of-the-art presentation of functional size and to guide software practitioners and researchers in applying FSM principles properly in their practices and software engineering methods and models.

When business processes are to be automated by an information system, business process knowledge is required in many ways throughout the software development life cycle (SDLC). Frequently, analysis is repeated to recapture this knowledge resulting in duplicate effort and inconsistent artifacts. We present our unified business process modeling methodology, UPROM that is used to generate various artifacts from business process models developed in conformance with its notation and approach. Together with business process models, the artifacts are user requirements document, software size estimation, process metrics list, process definition document and business glossary. These artifacts are easily maintainable as they originate from a single source and generated automatically by a tool. The methodology is applied in three case studies and summary of results is provided.

Estimating functional size in early phases of software projects is essential for accurate effort and cost planning. When an organization requires its business processes to be automated by a business application software, business processes are analyzed in the initial phases and utilized in various ways through software development life cycle. In previous studies we presented a unified business process modelling methodology, UPROM for the practices of business process and user requirements analysis, COSMIC based functional size estimation (FSE) and process documentation. Applying UPROM notation, process and guidelines to implement these practices in an integrated way, a set of models are developed that can be used to generate related artifacts. As one of the artifacts, UPROM tool is used to automatically estimate the early COSMIC functional size conforming to UPROM FSE method. The procedures and rules of UPROM FSE method are described in this paper. The results of multiple case study and validation activities indicate that UPROM can be used to achieve reasonably accurate size estimation results in early phases and decreased subjectivity without significant extra work for estimation.

As various software management activities including cost estimation and project control are conducted based on the software size measurement, achieving high accuracy in functional size measurement (FSM) is critical. Several studies examined the relation between FSM training and improvement in the accuracy of FSM. However, those studies propose comprehensive frameworks and approaches that require fundamental changes in the training content. In this study, we analyzed the effect of highlighting error categories during training by extracting the errors throughout four years. We showed that, highlighting the frequent error categories during the same training period without a fundamental change in the content would significantly decrease the error rate. The results of the research we conducted are promising about the improvement of measurement accuracy.

Reliability of functional size measurement is very crucial since software management activities such as cost and budget estimations, process benchmarking and project control depend on software size measurements. In order to improve the reliability of functional size measurements, they should be controlled and reviewed at the end of the measurement process. However, manual inspection for detecting defects and errors of measurements is time and effort consuming and there is always a possibility of missing a defect. To overcome such problems we developed a tool, for detecting defects of COSMIC functional size measurements automatically. In this study we presented the process of developing the tool, R-COVER, and the results of the case studies conducted for analyzing the efficiency of the tool in terms of correctness and accuracy.

The reliability of Functional size is crucial for effort and cost estimation in software development projects. However, estimation results may differ because of the errors made in the measurement process. This research proposes an error prevention model for COSMIC Functional Size Measurement method to increase the reliability of the measurements. The model has two main suggestions, first, to use the types of the functional processes during the measurement to prevent errors, and the second, to use a cardinality table to prevent errors related with cascading processes.

The estimation of effort is at the heart of project tasks, since it is used for many purposes such as cost estimation, budgeting, monitoring, project planning, control and software investments. Researchers analyze problems of the estimation, propose new models and use new techniques to improve estimation accuracy. However, effort estimation problem is not only computational but also a managerial problem and we need a defined estimation methodology to guide companies in their effort estimation tasks. Management of such methodology requires estimation goals, execution steps, applied measurement methods and updating mechanisms to be properly specified. In this study we presented the development steps for developing such an Effort Estimation Methodology, called EFES. The methodology is based on the reported best practices, empirical results obtained from a medium scale company and solutions to problems & conflicts described in literature. The validation and applicability of EFES methodology is performed in a medium scale company.

Effort data is crucial for project management activities in software projects. This data is utilized in estimations that are required for project planning, in the formation of benchmarking data sets and as a main input for project monitoring and control activities. However, there are known problems regarding effort data collection in the industry. In this study we investigate the effort data collection practices in the industry and factors that lead to inaccurate effort data collection. A pilot study was conducted to observe problems in effort data collection and a survey is conducted to observe the existence of these problems in the industry as well as causes of these problems. In this paper, findings of these studies are presented along with certain suggestions to improve the quality of effort data.

Data repositories play critical role in software management practices. Construction of the estimation models, benchmark of software performance indicators, identification of the process improvement opportunities, and quality assessment are major utilization areas of software repositories. This paper presents the observed difficulties in utilization of external and multi-organizational software benchmark repositories for effort estimation model construction for a software organization in finance domain. ISBSG, Albrecht, China, Desharnais, Finnish, Maxwell and Kemerer repositories' data were utilized in this study. The approach was the utilization of these repositories and organization's own repository to estimate the software development effort and evaluate whether external and multi-organizational data be used for effort estimation.

A number of methods have been proposed to build a relationship between effort and size. These models are generally based on regression analysis and a widely accepted model is not yet available. Although in some sizing methods, such as MKII and IFPUG, different multipliers for the base functional components (BFC) exist, their origin and the purpose of their usage are undefined. The COSMIC method does not treat components separately and assigns the same measurement unit to each of them. In this study we used the Artificial Neural Network and regression based methods to create effort estimation models that take the four components of the COSMIC method into consideration. In the research we compared several functional size based effort models in terms of accuracy using a reliable company dataset. These models comprised not only the generic models proposed in the literature or currently in use, but also specific models that we generated using our dataset with a single and multi-variate regression analysis and the ANN method. We also explored the effect of functional similarity (FS) using our specific models. We found that using BFC instead of total size improved effort estimation models and the ANN method is a useful approach to calibrate these components according to the company characteristics.

There are various approaches to software size measurement. Among these, the metrics and methods based on measuring the functionality attribute have become widely used since the original method was introduced in 1979. Although functional size measurement methods have gone a long way, they still provide challenges for software managers. This article identifies improvement opportunities based on empirical studies we performed on ongoing projects. We also compare our findings with the extended dataset provided by the International Software Benchmarking Standards Group (ISBSG).

Although functional size measurement (FSM) methods have gone a long way, they still provide challenges for software managers. A major challenge is related to the comparison of the well established FSM methods. This paper identifies the conceptual similarities and differences among Mkll FPA, COSMIC FFP and IFPUG FPA methods based on a case study, which involves implementation of the methods to measure the functional size of a military inventory management project integrated with a document management system.

Software process improvement in small organizations is a challenging task where the "smallness" brings a number of unique problems. We report the status of our work on creating ISO 9001 compliant quality systems in a small software organization.

Software process improvement practice and research have predominantly assumed a vertical approach to definition and change. Our studies show that vertical approaches have inherent difficulties. We have developed a new approach called horizontal change (HOC-A) that provides a new environment for change in software development organizations. We describe some of the difficulties of the vertical process improvement approaches and demonstrate, with examples, how the horizontal change approach addresses these difficulties.