• A dynamic trend analysis framework for Stack Exchange communities is introduced. • Two indicators for measuring topic growth are introduced. • A classifier to identify ChatGPT-related questions is introduced. • Tag clustering combining Inclusion Index with Affinity Propagation is used. • An overhauled visualization tool from our previous work is presented. Technological and methodological trends emerge at unprecedented rates, attracting developers to explore their potential and seek advice in online social networks. In this spectrum, ChatGPT has become a popular technology used for generating content to satisfy user queries while developers also integrate its mechanisms into their applications. Social networks usually revolve around technological trends through relevant announcements, posts, and questions. The primary goal is to demystify and evaluate the content surrounding questions from developers on Stack Overflow (SO) regarding a trending technology or method, in this case, ChatGPT. We present a topic-oriented trend analysis framework for analyzing questions from Stack Exchange communities, formulating a case study with five Research Questions (RQs) adapted to ChatGPT-related queries posted on Stack Overflow. The proposed framework contains different components aimed at extracting the main topics of relevant questions, providing analytics and pipelines for evaluating and comparing topic popularity, difficulty, and trending ability, as well as filtering irrelevant questions. The analysis uncovers diverse topics referring to technologies, platforms, and programming languages associated with ChatGPT usage, as well as a variety of purposes related to textual, audio, and image data. Additionally, the framework helped in identifying one popular and one unpopular topic, along with one difficult and four rising topics. In the context of ChatGPT, statistical tests indicated that Langchain is a more popular framework than Flutter and that questions related to the ChatGPT API concentrate lower scores but more answers than questions associated with LLMs. Overall, this paper demonstrates that the introduced framework can be utilized for studying multiple objectives covering a trending subject, as its mechanisms rely exclusively on the standard characteristics of Stack Exchange (SE) questions. Also, the methodologies and findings can offer insights and ideas for future research and experiments.

Context Possessing up-to-date knowledge, skills and transversal competencies (KSTs) is essential for both the successful delivery of software projects and a career in software engineering (SE). However, the technological landscape is changing rapidly, posing continuous challenges: for professionals entering the market or pivoting careers, for organizations hiring and monitoring workforce expertise and for educational institutes designing or updating their curricula. Objectives We study job requirements within and across SE occupations (Applications Programmers, Software Developers, Systems Analysts, Web and Multimedia Developers) to assist software organizations to better face skill mismatch and skills’ gap problems, software engineers in upskilling and reskilling endeavors and software education institutes in providing more industrially relevant curricula. Method In this study, we leverage a large corpus of online job advertisements, which are jointly collected by CEDEFOP and Eurostat. The dataset is analyzed through the lens of concepts and techniques from the study of biodiversity of species to assess the variation of expertise and identify skills that are transferable or unique in these occupations. Specifically, we adopt established diversity indices, such as alpha diversity, beta diversity, ordination methods, and indicator species analysis, aiming to quantify both the variety of skills within occupations and the differences across them. This approach highlights both the breadth and distinctiveness of expertise across occupations, rendering the biodiversity perspective a central and practical part of our methodology. Results The results reveal that the complete list of KSTs that is used to characterize the profiles of OJAs for SE-related occupations is very broad and that skillset required for each occupation is quite distinct, since there are statistically significant differences in the composition of the skillsets. Transversal Skills and Competences (T) appear to be the most transferable qualification; or “adapt to change” and “work in teams” are the KSTs that appears more uniformly to all studied software occupations, and “computer programming” is the top hard-skill that appears more uniformly to all occupations. However, each occupation shows some specific qualifications. Conclusion The results are contrasted against the literature, are interpreted, various implications to researchers and practitioners are provided, and a retrospective analysis of the tailoring of the biodiversity approach to SE labor landscape is provided. Overall, the proposed biodiversity analysis adds value by providing a novel, theory-driven methodology to assess skill variation, identifying both common and occupation-specific KSTs, and supporting evidence-based workforce and curriculum design.

The use of Generative AI, and more specifically Large-Language Models (LLMs), is becoming an essential aid in the software development process. An increasing number of software engineers are using general-purpose or code-trained LLMs for writing code, formulating requirements or deriving test cases. However, since this practice has come so abruptly into the daily routines of developers, the research community is still lacking an in-depth evaluation of its effectiveness. A major aspect of software engineering using LLMs that is rather unexplored is the quality of the code that is generated. In this paper, we explore the ability of GenAI to assist developers in performing refactoring activities, employing well-established Object-Oriented Programming “ good-practices ” like GoF Design Patterns and SOLID principles. To achieve this goal, we have performed a controlled experiment on junior developers, relying on a cross-over experimental design, and asked them to complete development tasks with and without the use of an LLM. The results suggested that GenAI-Assisted solutions outperformed Humans-Only ones in terms of the correctness of implementing the selected practice (pattern or principle), whereas Humans-Only solutions were superior in cognitive steps of the refactoring process such as the identification of the problem and the compromised quality attributes.

Scientific Software Development (SSD) plays a pivotal role in accelerating research and solving real-world problems across disciplines. Yet, the field remains fragmented, lacking a unified framework for analyzing the skills embedded in open-source scientific software. To address this gap, we introduce SciESCO , a tri-phasic framework that combines bibliometric analysis, skill extraction and predictive modeling to provide actionable insights into evolving competencies. In the first phase, focusing on 2239 publications from 2015 to 2025 in SoftwareX and Software Impacts, we analyze scientific software contributions, as these are the only peer-reviewed journals that publish software tools accompanied by complete and openly available metadata, including source code and related documentation. In the second phase, skills are extracted using ESCOX, an open-source tool developed under the SKILLAB EU project that leverages the European Skills/Competences, Qualifications, and Occupations (ESCO) taxonomy. These extracted skills represent the supply side of the labour market and are subsequently classified into eight scientific domains using a large language model (LLM). Finally, a Graph Neural Network (GNN) predicts links between skills and domains. Our results highlight critical competencies such as machine learning, authoring software, UI design, deep learning and data visualization. The SciESCO framework offers a skill-oriented view of SSD, supporting HR professionals, researchers and institutions in aligning talent with scientific needs. This study also identifies gaps in mapping competencies to domains, opening avenues for future research in skill analytics and offering a forward-looking view of the future SSD researcher. Finally, SciESCO provides an open-source tool designed for implementation by stakeholders, practitioners and researchers.

Software artifacts and source code are often viewed as pure technical constructs aiming primarily at delivering specific functionality to the end users. However, almost each line of a computer program is the result of software engineer’s craftsmanship and thus reflects their skills and capabilities, but also their aesthetic view of how code should be written. Additionally, by nature, the code is not an artifact that is managed by a single person: the code is peer-reviewed, in some cases programmed in pairs, or maintained by different people. In this respect, the first impression for the quality of a code is usually a matter of “ reading ” the “ beauty ” of the code and then diving into the details of the actual implementation. This “ first-look ” impression can psychologically bias the software engineers, either positively or negatively and affect their evaluation. In this article we propose a novel code beauty model (accompanied with metrics) and empirically explore: (a) if different software engineers perceive code beauty in the same way; (b) if the proposed code beauty metrics are correlated to the perceived code beauty by individual software engineers; and (c) if code beauty metrics are correlated to software maintainability. The results of the study suggest: (a) that code beauty is highly subjective and different software engineers perceive a code chunk as beautiful or not in an inconsistent way; (b) that some code beauty metrics can be considered as correlated to maintainability; and therefore, the “ first-look ” impression might to some extent be representative of the quality of the reviewed code chunk.

As society is continuously adapting to technological change and progress, fast-moving digital transformations are the driving force for setting the necessary skillsets for the workforce. Furthermore, the advent of Industry 5.0 as a defining concept for the future, which advocates a human-centric coalescence of humans and technology or software, renders the skilled workforce the most important asset in any organization or business. The endgame of the digital transformation is to evoke the reshaping, evolution, or replacement of traditional and possibly obsolete processes at intra- or inter-organizational levels in multiple aspects, introducing innovative ways of re-defining the workforce. In this context SKILLAB will act as a smart tool for handling, honing, and widening the competencies of the personnel of companies, forecasting future skill gaps and providing European citizens with a tool for upskilling and reskilling.

Software artifacts and source code are often viewed as pure technical constructs aiming primarily at delivering specific functionality to the end users. However, almost each line of a computer program is the result of developers' craftsmanship and thus reflects their skills and capabilities, but also their aesthetic view of how code should be written. Additionally, by nature, the code is not an artifact that is managed by a single person: the code is peer-reviewed, in some cases programmed in pairs, or maintained by different people. In this respect, the first impression for the quality of a code is usually a matter of "reading" the aesthetics of the code and then, diving into the details of the actual implementation. This "first-look" impression can psychologically bias the software engineer, either positively or negatively and affect his/her evaluation. In this article we investigate whether code beauty (or code aesthetics) must be valued in software programs, as a proxy to the quality of the code. Specifically, we attempt to relate the notion of code beauty with code quality metrics. For this purpose, we catalogued existing beauty measures (assessing the aesthetics of images, objects, and alphanumeric displays), tailored them to match code beauty, and correlated them to structural properties that are related to Technical Debt Interest (such as coupling, cohesion, etc.). The results of the study suggest that some code beauty metrics can be considered as correlated to TD Interest; and therefore, the "first-look" impression might to some extent be representative of the quality of the reviewed code chunk.

In recent years, we have witnessed an important increase in research focusing on how machine learning (ML) techniques can be used for software quality assessment and improvement. However, the derived methodologies and tools lack transparency, due to the black-box nature of the employed machine learning models, leading to decreased trust in their results. To address this shortcoming, in this paper we extend the state-of-the-art and -practice by building explainable AI models on top of machine learning ones, to interpret the factors (i.e. software metrics) that constitute a module as in risk of having high technical debt (HIGH TD), to obtain thresholds for metric scores that are alerting for poor maintainability, and finally, we dig further to achieve local interpretation that explains the specific problems of each module, pinpointing to specific opportunities for improvement during TD management. To achieve this goal, we have developed project-specific classifiers (characterizing modules as HIGH and NOT-HIGH TD) for 21 open-source projects, and we explain their rationale using the SHapley Additive exPlanation (SHAP) analysis. Based on our analysis, complexity, comments ratio, cohesion, nesting of control flow statements, coupling, refactoring activity, and code churn are the most important reasons for characterizing classes as in HIGH TD risk. The analysis is complemented with global and local means of interpretation, such as metric thresholds and case-by-case reasoning for characterizing a class as in-risk of having HIGH TD. The results of the study are compared against the state-of-the-art and are interpreted from the point of view of both researchers and practitioners.

With medical technological advances being developed in a rapid pace, the need for effective Scientific Software Development (SSD), that can process, store and visualize medical data is ever growing. Particularly during the COVID-19 pandemic, the medical community came together to produce efficient solutions to tackle this global setback. Programmers and developers have an active role in the procurement of medical software, with many of them exchanging knowledge and opinions in Q&A portals like Stack Overflow (SO) about methodologies, techniques and programming queries. In this study we present a data-driven framework that collects, filters, stores and analyzes issues and questions for medical applications from SO, visualizing them in an intuitive manner. To highlight the functionalities of our framework, we present a case study with COVID-19 SSD related questions, providing insights and valuable information about the status of the domain.

One of the most well-known laws of software evolution suggests that code quality deteriorates over time. Following this law, recent empirical studies have brought evidence that Technical Debt (TD) Principal tends to increase (in absolute value) as the system grows, since more technical debt issues are added than resolved over time. To shed light into how technical debt accumulation occurs in practice, in this paper we explore specific maintenance activities (i.e., feature addition, bug fixing, and refactoring) and explore the balance between the technical debt that they introduce or resolve. To achieve this goal, we rely on studying Pull Requests (PR), which are the most established way to contribute code to an open-source project. A Pull Request is usually comprised by more than one commits, corresponding to a specific development / maintenance activity. In our study, we categorized Pull Requests, based on their labels, to find the effect that the different maintenance activities have on the accumulation of technical debt across evolution. In particular, we have analysed more than 13.5K pull requests (mined from 10 OSS projects), by calculating the TD Principal (calculated through SonarQube) before and after the Pull Requests. The results of the study suggested that several labels are used for tagging Pull Requests, out of which the most prevalent ones are new features, bug fixing, and refactoring. The effect of these activities on TD Principal accumulation is statistically different, and: (a) the addition of features tends to increase TD Principal; (b) refactoring is having an almost consistent positive effect (reducing TD Principal); and (c) bug fixing activity has undecisive impact on TD Principal. These results are compared to existing studies, interpreted, and various useful implications for researchers and practitioners have been drawn.

Abstract Design patterns are well‐known solutions to recurring design problems that are widely adopted in the software industry, either as formal means of communication or as a way to improve structural quality, enabling proper software extension. However, the adoption and correct instantiation of patterns is not a trivial task and requires substantial design experience. Some patterns are conceptually close or present similar design alternatives, leading novice developers to improper pattern selection, thereby reducing maintainability. Additionally, the mis‐instantiation of a GoF (Gang‐of‐Four) design pattern, leads to phenomena such as pattern grime or architecture decay. To alleviate this problem, in this work we propose an approach that can help software engineers to more easily and safely select the proper design pattern, for a given design problem. The approach relies on decision trees, which are constructed using domain knowledge, while options are conveyed to software engineers through an Eclipse Theia plugin. To assess the usefulness and the perceived benefits of the approach, as well as the usability of the tool support, we have conducted an industrial validation study, using various data collection methods, such as questionnaires, focus groups, and task analysis. The results of the study suggest that the proposed approach is promising, since it increases the probability of the proper pattern being selected, and various useful future work suggestions have been obtained by the practitioners.

Quality improvement can be performed at the: (a) micro-management level: interventions applied at a fine-grained level (e.g., at a class or method level, by applying a refactoring); or (b) macro-management level: interventions applied at a large-scale (e.g., at project level, by using a new framework or imposing a quality gate). By considering that the outcome of any activity can be characterized as the product ofimpactandscale, in this paper we aim at exploring the impact of Technical Debt (TD) Macro-Management, whose scale is by definition larger than TD Micro-Management. By considering that TD artifacts reside at the micro-level, the problem calls for a nested model solution; i.e., modeling the structure of the problem: artifacts have some inherent characteristics (e.g., size and complexity), but obey the same project management rules (e.g., quality gates, CI/CD features, etc.). In this paper, we use the Under-Bagging based Generalized Linear Mixed Models approach, to unveil project management activities that are associated with the existence of HIGH_TD artifacts, through an empirical study on 100 open-source projects. The results of the study confirm that micro-management parameters are associated with the probability of a class to be classified as HIGH_TD, but the results can be further improved by controlling some project-level parameters. Based on the findings of our nested analysis, we can advise practitioners on macro-technical debt management approaches (such as “control the number of commits per day”, “adopt quality control practices”, and “separate testing and development teams”) that can significantly reduce the probability of all software artifacts to concentrate HIGH_TD. Although some of these findings are intuitive, this is the first work that delivers empirical quantitative evidence on the relation between TD values and project- or process-level metrics.

Technical Debt estimation relies heavily on the use of static analysis tools looking for violations of pre-defined rules. Largely, Technical Debt principal is attributed to the presence of low-level code smells, unavoidably tying the effort for fixing the problems with mere coding inefficiencies. At the same time, despite their simple definition, the detection of most code smells is non-trivial and subjective, rendering the assessment of Technical Debt principal dubious. To this end, we have revisited the literature on code smell detection approaches backed by tools and developed an Eclipse plugin that incorporates six code smell detection approaches. The combined application of various smell detectors can increase the certainty of identifying actual code smells that matter to the development team. We also conduct a case study to investigate the agreement among the employed code smell detectors. To our surprise the level of agreement is quite low even for relatively simple code smells, threating the validity of existing TD analysis tools and calling for increased attention to the precise specification of code and design level issues.

To date, the identification and quantification of Technical Debt (TD) rely heavily on a few sophisticated tools that check for violations of certain predefined rules, usually through static analysis. Different tools result in divergent TD estimates calling into question the reliability of findings derived by a single tool. To alleviate this issue, we present a tool that employs machine learning on a dataset built upon the convergence of three widely-adopted TD Assessment tools to automatically assess the class-level TD for any arbitrary Java project. The proposed tool is able to classify software classes as high-TD or not, by synthesizing source code and repository activity information retrieved by employing four popular open source analyzers. The classification results are combined with proper visualization techniques, to enable the identification of classes that are more likely to be problematic. To demonstrate the proposed tool and evaluate its usefulness, a case study is conducted based on a real-world open-source software project. The proposed tool is expected to facilitate TD management activities and enable further experimentation through its use in an academic or industrial setting.

Technical Debt (TD) is a successful metaphor in conveying the consequences of software inefficiencies and their elimination to both technical and non-technical stakeholders, primarily due to its monetary nature. The identification and quantification of TD rely heavily on the use of a small handful of sophisticated tools that check for violations of certain predefined rules, usually through static analysis. Different tools result in divergent TD estimates calling into question the reliability of findings derived by a single tool. To alleviate this issue we use 18 metrics pertaining to source code, repository activity, issue tracking, refactorings, duplication and commenting rates of each class as features for statistical and Machine Learning models, so as to classify them as High-TD or not. As a benchmark we exploit 18.857 classes obtained from 25 Java projects, whose high levels of TD has been confirmed by three leading tools. The findings indicate that it is feasible to identify TD issues with sufficient accuracy and reasonable effort: a subset of superior classifiers achieved an F2-measure score of approximately 0.79 with an associated Module Inspection ratio of approximately 0.10. Based on the results a tool prototype for automatically assessing the TD of Java projects has been implemented.

As the industry is moving towards digitalized solutions and practices, a growth in remote working has been observed with companies embracing flexibility for their workforce. Global crises, such as the coronavirus pandemic, have also accelerated this process, transforming the labor market. This trend is reflected in job portals, that contain an increasing number of remote job advertisements. Recognizing this evolving change, we perform a thorough study in Stack Overflow, to examine the main characteristics of remote working that discriminate it from its on-site counterpart. By collecting and analyzing 8514 job posts and leveraging text mining and graph theory methodologies, we attempt to pinpoint the primary elements that define each category, from dominant technologies to job positions and top seeking industries. The findings suggest that remote working presents differences from traditional working, being mainly associated with the software engineering sector and with well-known software development and data analytics technologies.

The Covid-19 outbreak has changed to an unprecedented extent almost every aspect of human activity. At the same time, the pandemic has stimulated enormous amount of research by scientists across various disciplines, seeking to study the phenomenon itself, its epidemiological characteristics and ways to confront its consequences. Information Technology, and particularly Data Science, drive innovation in all related to Covid-19 biomedical fields. Acknowledging that software developers routinely resort to open `question & answer' communities like Stack Overflow to seek advice on solving technical issues, we have performed an empirical study to investigate the extent, evolution and characteristics of Covid-19 related posts. Through the study of 464 Stack Overflow questions posted in February and March 2020 and leveraging the power of text mining, we attempt to shed light into the interest of developers in Covid-19 related topics and the most popular problems for which the users seek information. The findings reveal that indeed this global crisis sparked off an intense activity in Stack Overflow with most post topics reflecting a strong interest on the analysis of Covid- 19 data, primarily using Python technologies.

The cornerstones of technical debt (TD) are two concepts borrowed from economics: principal and interest. Although in economics the two terms are related, in TD there is no study on this direction so as to validate the strength of the metaphor. We study the relation between Principal and Interest, and subsequently dig further into the ‘ingredients’ of each concept (since they are multi-faceted). In particular, we investigate if artifacts with similar levels of TD Principal exhibit a similar amount of TD Interest, and vice-versa. To achieve this goal, we performed an empirical study, analyzing the dataset using the Mantel test. Through the Mantel test, we examined the relation between TD Principal and Interest, and identified aspects that are able to denote proximity of artifacts, with respect to TD. Next, through Linear Mixed Effects (LME) modelling we studied the generalizability of the results. The results of the study suggest that TD Principal and Interest are related, in the sense that classes with similar levels of TD Principal tend to have similar levels of Interest. Additionally, we have reached the conclusion that aggregated measures of TD Principal or Interest are more capable of identifying proximate artifacts, compared to isolated metrics. Finally, we have provided empirical evidence on the fact that improving certain quality properties (e.g., size and coupling) should be prioritized while ranking refactoring opportunities in the sense that high values of these properties are in most of the cases related to artifacts with higher levels of TD Principal. The findings shed light on the relations between the two concepts, and can be useful for both researchers and practitioners: the former can get a deeper understanding of the concepts, whereas the latter can use our findings to guide their TD management processes such as prioritization and repayment.

Abstract Context The rapid evolvement of software development effort estimation models created the need for empirical evaluation of their quality. The empirical evaluation is based either on hypothesis tests with respect to a single criterion or on aggregating methods for multiple criteria. However, a model can be considered as a multidimensional entity performing differently on alternative datasets and its performance can be divergent when expressed by alternative criteria. Objective In this study, we explore this multidimensional nature of models by considering them as points in two different spaces (domain and criteria spaces). Method Introducing an alternative approach for data‐driven benchmarking, a new framework based on archetypal analysis is proposed for evaluation purposes of multiple models. Results The benefits of the framework are illustrated through a large‐scale experimental setup on a set of 93 effort estimation models, trained and tested on 10 datasets under 8 criteria providing answers to critical research questions. Conclusion The results indicate that a small minority of reference models is enough to define the performance of the bulk of all models. The framework focuses on models that have behavior close to archetypes and especially those that are close to a “best” archetype.

Software repositories and Question & Answer sites constitute promising and ever-increasing sources of information for software analytics and for the identification of the role of the human factor in software engineering. Empirical evidence suggests that studying the human factor in software-related issues is multifaceted. Although analyses on different levels of developer networks have been performed, there is no existing work that moves beyond developer technical skills to take into account other important factors that might affect development work, such as personality and non-technical skills. For this reason, in this work we propose an approach that aims at finding links between badges – a sort of gamification awarded for developer's participation – personality traits, and interpersonal skills from Stack Overflow developers' community. The Generalized Linear Models (GLM) approach is adopted in order to examine the effects of personality traits and interpersonal skills on the non-technical awarded badges. Experimental results from the application of the proposed framework revealed statistically significant effects of both personality traits and interpersonal skills on the award gamification process.

The set of concepts collectively known as Technical Debt (TD) assume that software liabilities set up a context that can make a future change more costly or impossible; and therefore repaying the debt should be pursued. However, software developers often disagree with an automatically generated list of improvement suggestions, which they consider not fitting or important for their own code. To shed light into the reasons that drive developers to adopt or reject refactoring opportunities (i.e. TD repayment), we have performed an empirical study on the potential factors that affect the developers' decision to agree with the removal of a specific TD liability. The study has been addressed to the developers of four well-known open-source applications. To increase the response rate, a personalized assessment has first been sent to each developer, summarizing his/her own contribution to the TD of the corresponding project. Responds have been collected through a custom built web application that presented code fragments suffering from violations as identified by SonarQube along with information that could possibly affect their level of agreement to the importance of resolving an issue. These factors include data such as the frequency of past changes in the module under study, the number of bugs, the type and intensity of the violation, the level of involvement of the developer and whether he/she is a contributor in the corresponding project. Multivariate statistical analysis methods have been used to understand the importance and the underlying relationships among these factors and the results are expected to be useful for researchers and practitioners in TD Management.

Skills and competences of people participating in online professional networks constitute an ever-increasing new source for data collection and analysis. An important sub-domain of human resources management (HRM) is the recruitment process. Job advertisements and people profiles are main parts of recruitment and since are now available online, they constitute a key factor of a new e-recruitment era. Data mining for erecruitment analysis is important in order to extract a knowledge base for people analytics. Skills and competences are the key variables for people analytics and can be drawn from job advertisements. Leveraging the raw information of online job offers, provides a rich source for people analytics. Detecting the appropriate skills and competences for a job from raw text data and associate them with a job seeker is an increasing challenge. The main objective of this paper is the proposal of a framework aiming to collect online job advertisements from a web source which concerns IT job offers and to extract from the raw text the required skills and competences for specific jobs. The selected professional networking web source is StackOverflow and multivariate statistical data analysis was used to test the correlations between skills and competences in the job offers dataset. The present work falls in a relatively new field of research, concerning the competence mining of peopleware data with special focus on software development.

The importance of providing accurate estimations of software cost in management life cycle has led to an overabundant pool of prediction candidates exhibiting certain advantages and limitations. Thus, there is an imperative need for well-established principles that will aid the right decision-making regarding the selection of the best candidate. Unfortunately, the choice of the most appropriate estimation technique is not a trivial task, due to the multi-faceted nature of error. Accuracy, bias and variance are notions describing different aspects of predictive power that someone has to take into consideration during the validation process. The main objective of this paper is the utilization of visual analytics for the evaluation of two fundamental ingredients of prediction accuracy: the bias and the variance. Through a bootstrap-based resampling algorithm, we provide an easy-to-interpret way in order to acquire significant knowledge about the quality of a prediction candidate and manage the uncertainty of the estimation process. Ensemble techniques utilizing the advantages of both simple and complex solo methods are possible balancing solutions to the problem of the bias-variance tradeoff.

Background: Despite the number of Web effort estimation techniques investigated, there is no consensus as to which technique produces the most accurate estimates, an issue shared by effort estimation in the general software estimation domain. A previous study in this domain has shown that using ensembles of estimation techniques can be used to address this issue. Aim: The aim of this paper is to investigate whether ensembles of effort estimation techniques will be similarly successful when used on Web project data. Method: The previous study built ensembles using solo effort estimation techniques that were deemed superior. In order to identify these superior techniques two approaches were investigated: The first involved replicating the methodology used in the previous study, while the second approach used the Scott-Knott algorithm. Both approaches were done using the same 90 solo estimation techniques on Web project data from the Tukutuku dataset. The replication identified 16 solo techniques that were deemed superior and were used to build 15 ensembles, while the Scott-Knott algorithm identified 19 superior solo techniques that were used to build two ensembles. Results: The ensembles produced by both approaches performed very well against solo effort estimation techniques. With the replication, the top 12 techniques were all ensembles, with the remaining 3 ensembles falling within the top 17 techniques. These 15 effort estimation ensembles, along with the 2 built by the second approach, were grouped into the best cluster of effort estimation techniques by the Scott-Knott algorithm. Conclusion: While it may not be possible to identify a single best technique, the results suggest that ensembles of estimation techniques consistently perform well even when using Web project data.

Software Cost Estimation can be described as the process of predicting the most realistic effort required to complete a software project. Due to the strong relationship of accurate effort estimations with many crucial project management activities, the research community has been focused on the development and application of a vast variety of methods and models trying to improve the estimation procedure. From the diversity of methods emerged the need for comparisons to determine the best model. However, the inconsistent results brought to light significant doubts and uncertainty about the appropriateness of the comparison process in experimental studies. Overall, there exist several potential sources of bias that have to be considered in order to reinforce the confidence of experiments. In this paper, we propose a statistical framework based on a multiple comparisons algorithm in order to rank several cost estimation models, identifying those which have significant differences in accuracy, and clustering them in nonoverlapping groups. The proposed framework is applied in a large-scale setup of comparing 11 prediction models over six datasets. The results illustrate the benefits and the significant information obtained through the systematic comparison of alternative methods.

Software Cost Estimation is the task of predicting the effort or productivity required to complete a software project. Two of the most known techniques appeared in literature so far are Regression Analysis and Estimation by Analogy. The results of the empirical studies show the lack of convergence in choosing the best prediction technique between the parametric Regression Analysis and the non-parametric Estimation by Analogy models. In this paper, we introduce the use of a semi-parametric model that achieves to incorporate some parametric information into a non-parametric model combining in this way regression and analogy. Furthermore, we demonstrate the procedure of building such a model on two well-known datasets and we present the comparative results based on the predictive accuracy of the new technique using several accuracy measures. We also perform statistical tests on the residuals in order to assess the improvement in the predictions attained through the new semi-parametric model in comparison to the accuracy of Regression Analysis and Estimation by Analogy when applied separately. Our results show that the semi-parametric model provides more accurate predictions than each one of the parametric and non-parametric approaches.