Context: Former literature revealed team performance is contingent on personality composition and interactive effects of team climate. While decades of research on personality prevails in software engineering, team climate remains sparsely researched. Objective: In agile software development, individuals and interactions are key sources of agility. This study replicates a previous study and analyzes the relationship between five-factor-model personality traits and team climate dimensions among agile teams in a telecom company. Method: A Web-based survey was replicated twice, first with 75 professionals from 12 teams in Sweden, followed by 46 professionals from seven teams in India. The data was used for correlation, regression analyses, and meta-analysis. Results: We observed significant negative correlations between neuroticism and all the team climate dimensions. Meta-analysis identified a significant medium-sized negative effect between neuroticism and participative safety. Regression analysis showed personality traits accounted for around 10 % of the variance in team climate dimensions. Conclusions: High neuroticism is not conducive to team climate as emotionally unstable members could impair team cohesion by being reactive and susceptible to stress. Managers assembling Scrum teams ought to mitigate higher neuroticism by counterbalancing it with an elevation of corresponding negatively correlated personality variables and providing support/training towards increasing the aforementioned variables. © 2023

Investigating factors influencing agile team climate is pressing given its impact on organizational performance. Despite previous studies, association between human aspects and team climate remains yet unexplored. Exploring association between two human aspects, namely team member's capability measures and personality traits, and member's perceived team climate. 75 professionals (12 teams) from one division participated in the first survey iteration, second iteration in another division included 46 professionals (7 teams). We employed correlation analyses to measure association between human aspects and climate, and regression analyses to identify team climate predictors. In relation to team climate dimensions, we observed a significant negative correlation with neuroticism and a significant positive correlation with responsibility. Linear regression analysis showed capability measures accounted for 19.2% of variance in team climate. Multivariable regression analysis revealed capability measures and personality traits together accounted for 25.7% of variance in team climate. An individual's propensity towards self-doubt (neuroticism) negatively affects perceived team climate, whereas individual's ability to be responsible and teamwork-oriented positively affect perceived team climate. The inclusion of both capability and personality variables as input for multivariable regression explained slightly more variance in team climate, compared to only capability measures (25.7% against 19.2%).

A good search strategy is essential for a successful systematic literature study. Historically, database searches have been the norm, which was later complemented with snowball searches. Our conjecture is that we can perform even better searches if combining these two search approaches, referred to as a hybrid search strategy. Our main objective was to compare and evaluate a hybrid search strategy. Furthermore, we compared four alternative hybrid search strategies to assess whether we could identify more cost-efficient ways of searching for relevant primary studies. To compare and evaluate the hybrid search strategy, we replicated the search procedure in a systematic literature review (SLR) on industry–academia collaboration in software engineering. The SLR used a more “traditional” approach to searching for relevant articles for an SLR, while our replication was executed using a hybrid search strategy. In our evaluation, the hybrid search strategy was superior in identifying relevant primary studies. It identified 30% more primary studies and even more studies when focusing only on peer-reviewed articles. To embrace individual viewpoints when assessing research articles and minimise the risk of missing primary studies, we introduced two new concepts, wild cards and borderline articles, when performing systematic literature studies. The hybrid search strategy is a strong contender for being used when performing systematic literature studies. Furthermore, alternative hybrid search strategies may be viable if selected wisely in relation to the start set for snowballing. Finally, the two new concepts were judged as essential to cater for different individual judgements and to minimise the risk of excluding primary studies that ought to be included.

Value-Based Software Engineering stresses the role of value in software related decisions. In the context of feature selection, software features judged to provide higher value take priority in the development process. This paper focuses on what value means when selecting software features. Using grounded theory, we conducted and analyzed semi-structured interviews with 21 key stakeholders (decision-makers) from three software/software-intensive companies, within a context where value-based decision-making was already established. Our analysis led to the building of a theory of value for value-based feature selection that identifies the nature of value propositions considered by key stakeholders when selecting software features (i.e., decision-making criteria for deciding upon software features, as suggested by Boehm (2003)). We found that some value propositions were common to all three company cases (core value propositions), whereas others were dependent upon the context in which a company operates, and the characteristics of the product under development (specific value propositions). Moreover, value propositions vary according to the stakeholder group and the type of feature being assessed. Our study provides significant insight into value in the context of feature selection, and generates new concepts around value-based feature selection such as new value propositions.

Software development involves many activities, and decision making is an essential one. Various factors can impact a decision-making process, and by understanding such factors, one can improve the process. Since people are the ones making decisions, some human-related aspects are amongst those influencing factors. One such aspect is the decision maker’s personality. This research investigates the relationship between decision-making style and personality within the context of software project development. We conducted a survey in a population of Brazilian software engineers to gather data on their personality and decision-making style. Data from 63 participants was gathered and resulted in the identification of seven statistically significant correlations between decision-making style and personality (personality factor and personality facets). Furthermore, we built a regression model in which decision-making style (DMS) was the response variable and personality factors the independent variables. The backward elimination procedure selected only agreeableness to explain 4.2% of DMS variation. The model accuracy was evaluated and deemed good enough. Regarding the moderation effect of demographic variables (age, educational level, experience, and role) on the relationship between DMS and Agreeableness, the analysis showed that only software engineers’ role has such effect. This paper contributes toward understanding the relationship between DMS and personality. Results show that the personality variable agreeableness can explain the variation in decision-making style. Furthermore, someone’s role in a software development project can impact the strength of the relationship between DMS and agreeableness.

Systematic Literature Reviews (SLRs) have been adopted within Software Engineering (SE) for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now potentially not fully up-to-date, and there are no standard proposals on how to update SLRs in SE. The objective of this paper is to propose guidelines on how to best search for evidence when updating SLRs in SE, and to evaluate these guidelines using an SLR that was not employed during the formulation of the guidelines. To propose our guidelines, we compare and discuss outcomes from applying different search strategies to identify primary studies in a published SLR, an SLR update, and two replications in the area of effort estimation. These guidelines are then evaluated using an SLR in the area of software ecosystems, its update and a replication. The use of a single iteration forward snowballing with Google Scholar, and employing as a seed set the original SLR and its primary studies is the most cost-effective way to search for new evidence when updating SLRs. Furthermore, the importance of having more than one researcher involved in the selection of papers when applying the inclusion and exclusion criteria is highlighted through the results. Our proposed guidelines formulated based upon an effort estimation SLR, its update and two replications, were supported when using an SLR in the area of software ecosystems, its update and a replication. Therefore, we put forward that our guidelines ought to be adopted for updating SLRs in SE.

Numerous software companies are adopting value-based decision making. However, what does value mean for key stakeholders making decisions? How do different stakeholder groups understand value? Without an explicit understanding of what value means, decisions are subject to ambiguity and vagueness, which are likely to bias them. This case study provides an in-depth analysis of key stakeholders' value propositions when selecting features for a large telecommunications company's software-intensive product. Stakeholders' value propositions were elicited via interviews, which were analyzed using Grounded Theory coding techniques (open and selective coding). Thirty-six value propositions were identified and classified into six dimensions: customer value, market competitiveness, economic value/profitability, cost efficiency, technology & architecture, and company strategy. Our results show that although propositions in the customer value dimension were those mentioned the most, the concept of value for feature selection encompasses a wide range of value propositions. Moreover, stakeholder groups focused on different and complementary value dimensions, calling to the importance of involving all key stakeholders in the decision making process. Although our results are particularly relevant to companies similar to the one described herein, they aim to generate a learning process on value-based feature selection for practitioners and researchers in general.

[Context] Systematic Literature Reviews (SLRs) have been adopted within the Software Engineering (SE) domain for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now outdated, and there are no standard proposals on how to update SLRs in SE. [Objective] The goal of this paper is to provide recommendations on how to best to search for evidence when updating SLRs in SE. [Method] To achieve our goal, we compare and discuss outcomes from applying different search strategies to identifying primary studies in a previously published SLR update on effort estimation. [Results] The use of a single iteration forward snowballing with Google Scholar, and employing the original SLR and its primary studies as a seed set seems to be the most cost-effective way to search for new evidence when updating SLRs. [Conclusions] The recommendations can be used to support decisions on how to update SLRs in SE.

Integrating value-oriented perspectives into the principles and practices of software engineering is critical to ensure that software development and management activities address all key stakeholders' views and also balance short-and-long-term goals. This is put forward in the discipline of Value-Based Software Engineering (VBSE). In this paper, a mapping study of VBSE is detailed. We classify evidence on VBSE principles and practices, research methods, and the research types. This mapping study includes 134 studies located from online searches, and backward snowballing of references. Our results show that VB Requirements Engineering (22%) and VB Planning and Control (19%) were the two principles and practices mostly investigated in the VBSE literature, whereas VB Risk Management, VB People Management and Value Creation (3% respectively) were the three less researched. In terms of the research method, the most commonly employed method is case-study research. In terms of research types, most of the studies (28%) proposed solution technique(s) without empirical validation.

Context: Companies must make a paradigm shift in which both short- and long-term value aspects are employed to guide their decision-making. Such need is pressing in innovative industries, such as ICT, and is the core of Value-based Software Engineering (VBSE). Objective: This paper details three case studies where value estimation models using Bayesian Network (BN) were built and validated. These estimation models were based upon value-based decisions made by key stakeholders in the contexts of feature selection, test cases execution prioritization, and user interfaces design selection. Methods: All three case studies were carried out according to a Framework called VALUE — improVing decision-mAking reLating to software-intensive prodUcts and sErvices development. This framework includes a mixed-methods approach, comprising several steps to build and validate company-specific value estimation models. Such a building process uses as input data key stakeholders’ decisions (gathered using the Value tool), plus additional input from key stakeholders. Results: Three value estimation BN models were built and validated, and the feedback received from the participating stakeholders was very positive. Conclusions: We detail the building and validation of three value estimation BN models, using a combination of data from past decision-making meetings and also input from key stakeholders.

Context From a point of view, software development is a set of decisions that need to be made while the software is developed. Many alternatives should be considered, such as the technology to employ, or the most important features to implement. However, many factors can influence one’s decision-making, such as the decision maker’s personality. Objective This paper reports the state of the art with regard to the relationship between decision-makers’ personality and decision-making aspects. Method We conducted a Systematic Literature Review to search and analyze published primary studies that discuss the abovementioned relationship in the context of companies that develop any kind of product or service. Results Despite the recognized influence of personality in decision-making activities, we were not able to find any study in Software Engineering field that discusses this relationship. We included 15 studies and most of them are from Management field, excluding one from Information System field. From these studies, we identified 75 reported relationships between 28 different personality aspects and 30 different decision-making aspects. Conclusion The interest in this topic born on 80’s and it has grown after 2002. However, despite the number of reported relationships, and the number of personalities and decision-making aspects investigated, more research on this topic is necessary. In particular, it is important to verify how someone’s personality influences the decision-making considering the software development context. This can help in improving how a decision is made in software engineering context.

The software industry's current decision-making relating to product/project management and development is largely done in a value neutral setting, in which cost is the primary driver for every decision taken. However, numerous studies have shown that the primary critical success factor that differentiates successful products/projects from failed ones lie in the value domain. Therefore, to remain competitive, innovative and to grow, companies must change from cost-based to value-based decisionmaking where the decisions taken are the best for that company's overall value creation. This paper details a case study where value-based decisions made by key stakeholders to select features for the next sprint of an Internet of Things (IoT) project, stored in a decisions database, were used to build and validate a value estimation model. This model's goal was to estimate the overall value contribution that each feature being discussed during a decision-making meeting would bring to the company, if selected for implementation. The estimation technique employed was Bayesian Network, and validation results were quite positive.

Several studies have shown that both individual and team capability can affect software development performance and project success; a deeper understating of such phenomena is crucial within the context of Agile Software Development (ASD), given that its workforce is a key source of agility. This paper contributes towards such understanding by means of a case study that uses data from 14 interviews carried out at a large telecommunications company, within the context of a mobile money transfer system developed in Sweden and India, to identify individual and team capability measures used to form productive teams. Our results identified 10 individual and five team capability measures, of which, respectively, five and four have not been previously characterised by a systematic literature review (SLR) on this same topic. Such review aggregated evidence for a total of 133 individual and 28 team capability measures. Further work entails extending our findings via interviewing other software/software-intensive industries practicing ASD.

To sustain growth, maintain competitive advantage and to innovate, companies must make a paradigm shift in which both short- and long-term value aspects are employed to guide their decision-making. Such need is clearly pressing in innovative industries, such as ICT, and is also the core of Value-based Software Engineering (VBSE). The goal of this paper is to detail a framework called VALUE—improving decision-making relating to software-intensive products and services development—and to show its application in practice to a large ICT company in Finland. The VALUE framework includes a mixed-methods approach, as follows: to elicit key stakeholders’ tacit knowledge regarding factors used during a decision-making process, either transcripts from interviews with key stakeholders are analysed and validated in focus group meetings or focus-group meeting(s) are directly applied. These value factors are later used as input to a Web-based tool (Value tool) employed to support decision making. This tool was co-created with four industrial partners in this research via a design science approach that includes several case studies and focus-group meetings. Later, data on key stakeholders’ decisions gathered using the Value tool, plus additional input from key stakeholders, are used, in combination with the Expert-based Knowledge Engineering of Bayesian Network (EKEBN) process, coupled with the weighed sum algorithm (WSA) method, to build and validate a company-specific value estimation model. The application of our proposed framework to a real case, as part of an ongoing collaboration with a large software company (company A), is presented herein. Further, we also provide a detailed example, partially using real data on decisions, of a value estimation Bayesian network (BN) model for company A. This paper presents some empirical results from applying the VALUE Framework to a large ICT company; those relate to eliciting key stakeholders’ tacit knowledge, which is later used as input to a pilot study where these stakeholders employ the Value tool to select features for one of their company’s chief products. The data on decisions obtained from this pilot study is later applied to a detailed example on building a value estimation BN model for company A. We detail a framework—VALUE framework—to be used to help companies improve their value-based decisions and to go a step further and also estimate the overall value of each decision.

[Context]: To remain competitive, innovative and to grow, companies should use a value-based decision-making where decisions are the best for that company's overall value creation. However, without tool support, the use of explicit value propositions and aggregation of different key stakeholders' decisions during decision-making may be a challenge for many companies. [Goal]: The goal of this paper is to investigate the extent to which a Web-based tool for value-based decision-making can successfully support stakeholders' decision-making process. [Method]: We conducted three case studies across four software projects, during six weeks, in the contexts of feature selection, test cases execution prioritization and user interfaces design selection. Prior to using the tool, stakeholders' value propositions were elicited via focus-group meetings; later, during a post-mortem phase, data was gathered via observation, semi-structured interviews and structured questionnaires. [Results]: Participants reported an improvement of their decision-making process and quality of decisions; further, they also felt confident about using the tool, and that it can be useful to their work. [Conclusions]: Results suggested that the use of tool support by the stakeholders in the investigated company for value-based decision-making improved their decision-making process and the quality of decisions.

[Background] Systematic Literature Reviews (SLRs) are one of the important pillars when employing an evidence-based paradigm in Software Engineering. To date most SLRs have been conducted using a search strategy involving several digital libraries. However, significant issues have been reported for digital libraries and applying such search strategy requires substantial effort. On the other hand, snowballing has recently arisen as a potentially more efficient alternative or complementary solution. Nevertheless, it requires a relevant seed set of papers. [Aims] This paper proposes and evaluates a hybrid search strategy combining searching in a specific digital library (Scopus) with backward and forward snowballing. [Method] The proposed hybrid strategy was applied to two previously published SLRs that adopted database searches. We investigate whether it is able to retrieve the same included papers with lower effort in terms of the number of analysed papers. The two selected SLRs relate respectively to elicitation techniques (not confined to Software Engineering (SE)) and to a specific SE topic on cost estimation. [Results] Our results provide preliminary support for the proposed hybrid search strategy as being suitable for SLRs investigating a specific research topic within the SE domain. Furthermore, it helps overcoming existing issues with using digital libraries in SE. [Conclusions] The hybrid search strategy provides competitive results, similar to using several digital libraries. However, further investigation is needed to evaluate the hybrid search strategy.

Context: Despite being defined during the first phase of the Systematic Literature Review (SLR) process, the protocol is usually refined when other phases are performed. Several researchers have reported their experiences in applying SLRs in Software Engineering (SE) however, there is still a lack of studies discussing the iterative nature of the protocol definition, especially how it should be perceived by researchers conducting SLRs. Objective: The main goal of this study is to perform a survey aiming to identify: (i) the perception of SE researchers related to protocol definition; (ii) the activities of the review process that typically lead to protocol refinements; and (iii) which protocol items are refined in those activities. Method: A survey was performed with 53 SE researchers. Results: Our results show that: (i) protocol definition and pilot test are the two activities that most lead to further protocol refinements; (ii) data extraction form is the most modified item. Besides that, this study confirmed the iterative nature of the protocol definition. Conclusions: An iterative pilot testcan facilitate refinements in the protocol.

Context: Software Engineering (SE) is an evolving discipline with new subareas being continuously developed and added. To structure and better understand the SE body of knowledge, taxonomies have been proposed in all SE knowledge areas. Objective: The objective of this paper is to characterize the state-of-the-art research on SE taxonomies. Method: A systematic mapping study was conducted, based on 270 primary studies. Results: An increasing number of SE taxonomies have been published since 2000 in a broad range of venues, including the top SE journals and conferences. The majority of taxonomies can be grouped into the following SWEBOK knowledge areas: construction (19.55%), design (19.55%), requirements (15.50%) and maintenance (11.81%). Illustration (45.76%) is the most frequently used approach for taxonomy validation. Hierarchy (53.14%) and faceted analysis (39.48%) are the most frequently used classification structures. Most taxonomies rely on qualitative procedures to classify subject matter instances, but in most cases (86.53%) these procedures are not described in sufficient detail. The majority of the taxonomies (97%) target unique subject matters and many taxonomy-papers are cited frequently. Most SE taxonomies are designed in an ad-hoc way. To address this issue, we have revised an existing method for developing taxonomies in a more systematic way. Conclusion: There is a strong interest in taxonomies in SE, but few taxonomies are extended or revised. Taxonomy design decisions regarding the used classification structures, procedures and descriptive bases are usually not well described and motivated.

Background: A Systematic Literature Review (SLR) is a methodology used to aggregate relevant evidence related to one or more research questions. Whenever new evidence is published after the completion of a SLR, this SLR should be updated in order to preserve its value. However, updating SLRs involves significant effort. Objective: The goal of this paper is to investigate the application of forward snowballing to support the update of SLRs. Method: We compare outcomes of an update achieved using the forward snowballing versus a published update using the search-based approach, i.e., searching for studies in electronic databases using a search string. Results: Forward snowballing showed a higher precision and a slightly lower recall. It reduced in more than five times the number of primary studies to filter however missed one relevant study. Conclusions: Due to its high precision, we believe that the use of forward snowballing considerably reduces the effort in updating SLRs in Software Engineering; however the risk of missing relevant papers should not be underrated.

Background: Survey is a method of research aiming to gather data from a large population of interest. Despite being extensively used in software engineering, survey-based research faces several challenges, such as selecting a representative population sample and designing the data collection instruments.

Context -- Developers' productivity plays an important role in software development organizations; however, in many cases the management of such human capital is mainly based on how project managers perceive productivity. Therefore, it is important to investigate what these perceptions are in practice. Goal -- This study's main goal is to understand project managers' perception regarding developers' productivity. Method -- We employed a qualitative research methodology using semi-structured interviews for data collection. We interviewed 12 managers from three software development organizations in the city of Manaus (Brazil). Results -- We identified that the managers' perceptions about developers' productivity are influenced by four different factors: (1) tasks delivered on time, (2) produced artifacts that do not need rework, (3) products that meet stakeholders' expectations, and (4) personal behavior such as focus and proactivity. Conclusions -- This qualitative study shows a perception of developers' productivity different from that presented in other research papers, and suggests that human factors play an important role in managers' perceptions about productivity. Future work will investigate how these perceptions concretely influence developers' productivity, and how they relate to the existing developers' productivity factors in the literature.

Numerous companies worldwide make their decisions related to software projects/products in a value neutral way, using only earned value systems, which represent short-term goals. Better decisions can be made using a value-based approach, achieving cost-effective results and reliable construction and maintenance of products. However, moving from a value-neutral to a value-based paradigm can be a challenge. We provide tool-support, which was co-created in collaboration with three software companies, to ease the paradigm shift. Our tool supports both individual and group-based decisions using several visualization mechanisms. Despite the co-creation process employed while developing the VALUE tool, there are specific issues relating to its usability that must also be assessed in order to reduce any possible drawbacks for its adoption by industry. This paper details three usability studies that were carried out to assess the VALUE tool's usability. The results also suggest that the tool is ready to be taken into use in the industry.

To facilitate the sharing and combination of knowledge by Global Software Engineering (GSE) researchers and practitioners, the need for a common terminology and knowledge classification scheme has been identified, and as a consequence, a taxonomy and an extension were proposed. In addition, one systematic literature review and a survey on respectively the state of the art and practice of effort estimation in GSE were conducted, showing that despite its importance in practice, the GSE effort estimation literature is rare and reported in an ad-hoc way. Therefore, this paper proposes a specialized GSE taxonomy for effort estimation, which was built on the recently proposed general GSE taxonomy (including the extension) and was also based on the findings from two empirical studies and expert knowledge. The specialized taxonomy was validated using data from eight finished GSE projects. Our effort estimation taxonomy for GSE can help both researchers and practitioners by supporting the reporting of new GSE effort estimation studies, i.e. new studies are to be easier to identify, compare, aggregate and synthesize. Further, it can also help practitioners by providing them with an initial set of factors that can be considered when estimating effort for GSE projects.

Presents an editorial discusses the current status and activities supported by this publication.

Context: There are numerous studies on effort estimation in Agile Software Development (ASD) and the state of the art in this area has been recently documented in a Systematic Literature Review (SLR). However, to date there are no studies on the state of the practice in this area, focusing on similar issues to those investigated in the above-mentioned SLR. Objectives: The aim of this paper is to report on the state of the practice on effort estimation in ASD, focusing on a wide range of aspects such as the estimation techniques and effort predictors used, to name a few. Method: A survey was carried out using as instrument an on-line questionnaire answered by agile practitioners who have experience in effort estimation. Results: Data was collected from 60 agile practitioners from 16 different countries, and the main findings are: 1) Planning poker (63%), analogy (47%) and expert judgment (38%) are frequently practiced estimation techniques in ASD; 2) Story points is the most frequently (62%) employed size metric, used solo or in combination with other metrics (e.g., function points); 3) Team's expertise level and prior experience are most commonly used cost drivers; 4) 52% of the respondents believe that their effort estimates on average are under/over estimated by an error of 25% or more; 5) Most agile teams take into account implementation and testing activities during effort estimation; and 6) Estimation is mostly performed at sprint and release planning levels in ASD. Conclusions: Estimation techniques that rely on experts' subjective assessment are the ones used the most in ASD, with effort underestimation being the dominant trend. Further, the use of multiple techniques in combination and story points seem to present a positive association with estimation accuracy, and team-related cost drivers are the ones used by most agile teams. Finally, requirements and management related issues are perceived as the main reasons for inaccurate estimates.

[Context]: The numerous challenges that can hinder software companies from gathering their own data have motivated over the past 15 years research on the use of cross-company (CC) datasets for software effort prediction. Part of this research focused on Web effort prediction, given the large increase worldwide in the development of Web applications. Some of these studies indicate that it may be possible to achieve better performance using CC models if some strategy to make the CC data more similar to the within-company (WC) data is adopted. [Goal]: This study investigates the use of a recently proposed approach called Dycom to assess to what extent Web effort predictions obtained using CC datasets are effective in relation to the predictions obtained using WC data when explicitly mapping the CC models to the WC context. [Method]: Data on 125 Web projects from eight different companies part of the Tukutuku database were used to build prediction models. We benchmarked these models against baseline models (mean and median effort) and a WC base learner that does not benefit of the mapping. We also compared Dycom against a competitive CC approach from the literature (NN-filtering). We report a company-by- company analysis. [Results]: Dycom usually managed to achieve similar or better performance than a WC model while using only half of the WC training data. These results are also an improvement over previous studies that investigated the use of different strategies to adapt CC models to the WC data for Web effort estimation. [Conclusions]: We conclude that the use of Dycom for Web effort prediction is quite promising and in general supports previous results when applying Dycom to conventional software datasets.

Effort estimation is a project management activity that is mandatory for the execution of software projects. Despite its importance, there have been just a few studies published on such activities within the Agile Global Software Development (AGSD) context. Their aggregated results were recently published as part of a secondary study that reported the state of the art on effort estimation in AGSD. This study aims to complement the above-mentioned secondary study by means of an empirical investigation on the state of the practice on effort estimation in AGSD. To do so, a survey was carried out using as instrument an on-line questionnaire and a sample comprising software practitioners experienced in effort estimation within the AGSD context. Results show that the effort estimation techniques used within the AGSD and collocated contexts remained unchanged, with planning poker being the one employed the most. Sourcing strategies were found to have no or a small influence upon the choice of estimation techniques. With regard to effort predictors, global challenges such as cultural and time zone differences were reported, in addition to factors that are commonly considered in the collocated context, such as team experience. Finally, many challenges that impact the accuracy of the effort estimates were reported by the respondents, such as problems with the software requirements and the fact that the communication effort between sites is not properly accounted.

Background: Effort estimation is the key for efficiently managing Web projects and achieving their success. In order to correctly estimate, it is necessary to have a broad knowledge of the factors that influence effort estimation in Web projects. Aim: In this research we aim to increase the understanding of Web effort estimation by using a set of factors identified in literature along with the knowledge from experts in effort estimation. Method: We have gathered data from two different sources: (a) our previous work, in which we applied Grounded Theory procedures to identify factors that influence Web effort estimation from the point of view of Web project estimation experts; and (b) a Systematic Literature Review (SLR) extension, in which we identified factors reported in research papers. We have used the qualitative results from these sources to make comparisons and draw conclusions on factors affecting Web effort estimation. Results: We identified a total of 90 factors that influence effort estimation in Web projects. From this set, 30 factors were identified only in the qualitative study with experts in effort estimation, not being present in the SLR extension. Conclusions: By integrating the factors found in both our qualitative study with effort estimation experts and the SLR extension, we managed to create a comprehensive list of factors influencing effort estimation. Also, this set can be a starting point in the proposal of effort estimation models. Finally, the results from our comparison can be considered an indication that it is necessary to increase the employment of qualitative research to capture evidences regarding the current state of practice in Software Engineering.

[Objective] The objective of this paper is to extend a previously conducted systematic literature review (SLR) that investigated under what circumstances individual organizations would be able to rely on cross-company based estimation models. [Method] We applied the same methodology used in the SLR we are extending herein (covering the period 2006-2013) based on primary studies that compared predictions from cross-company models with predictions from within-company models constructed from analysis of project data. [Results] We identified 11 additional papers; however two of these did not present independent results and one had inconclusive findings. Two of the remaining eight papers presented both, trials where cross-company predictions were not significantly different from within-company predictions and others where they were significantly different. Four found that cross-company models gave prediction accuracy significantly different from within-company models (one of them in favor of cross-company models), while two found no significant difference. The main pattern when examining the study related factors was that studies where cross-company predictions were significantly different from within-company predictions employed larger within-company data sets. [Conclusions] Overall, half of the analyzed evidence indicated that cross-company estimation models are not significantly worse than within-company estimation models. Moreover, there is some evidence that sample size does not imply in higher estimation accuracy, and that samples for building estimation models should be carefully selected/filtered based on quality control and project similarity aspects. The results need to be combined with the findings from the SLR we are extending to allow further investigating this topic.

Nowadays, software systems are a key factor in the success of many organizations as in most cases they play a central role helping them attain a competitive advantage. However, despite their importance, software systems may be quite costly to develop, so substantially decreasing companies' profits. In order to tackle this challenge, many organizations look for ways to decrease costs and increase profits by applying new software development approaches, like Global Software Development (GSD). Some aspects of the software project like communication, cooperation and coordination are more challenging in globally distributed than in collocated projects, since language, cultural and time zone differences are factors which can increase the required effort to globally perform a software project. Communication, coordination and cooperation aspects affect directly the effort estimation of a project, which is one of the critical tasks related to the management of a software development project. There are many studies related to effort estimation methods/techniques for colocated projects. However, there are evidences that the colocated approaches do not fit to GSD. So, this paper presents the results of a systematic literature review of effort estimation in the context of GSD, which aimed at help both researchers and practitioners to have a holistic view about the current state of the art regarding effort estimation in the context of GSD. The results suggest that there is room to improve the current state of the art on effort estimation in GSD.

Context: Reliable effort estimation is essential for better management of Web projects, hence the need to identify what are the key factors that affect effort estimates for new Web projects and how they are inter-related. Objective: This paper improves our understanding of Web effort estimation using as basis the knowledge of Web effort estimation experts. Method: We employed qualitative research with the participation of four different Web development companies in Manaus (Brazil) using semi-structured interviews for data collection; our data analysis was carried out using Grounded Theory-based procedures to identify and combine factors affecting the estimation effort of Web projects. Results: We identified four main groupings (categories) of factors - Web project, Web development complexity, Web development team, and Clients. Each of these groupings contains a set of factors that impact upon Web effort estimation. Conclusions: This is the first time that qualitative research is employed in the field of Web effort estimation to further understand and help improve this process. In addition, some of the factors found had never been identified in any of the previous studies in this field, thus suggesting that the use of Grounded Theory-based procedures may provide a way to enrich our understanding of the phenomenon under investigation via the identification of factors that overlap and also complement those from previous studies.

Background: Relational database-driven software applications have gained significant importance in modern software development. Given that software maintainability is an important quality attribute, predicting these applications' maintainability can provide various benefits to software organizations, such as adopting a defensive design and more informed resource management. Aims: The aim of this paper is to present the results from employing two well-known prediction techniques to estimate the maintainability of relational database-driven applications. Method: Case-based reasoning (CBR) and classification and regression trees (CART) were applied to data gathered on 56 software projects from software companies. The projects concerned development and/or maintenance of relational database-driven applications. Unlike previous studies, all variables (28 independent and 1 dependent) were measured on a 5-point bi-polar scale. Results: Results showed that CBR performed slightly better (at 76.8% correct predictions) in terms of prediction accuracy when compared to CART (67.8%). In addition, the two important predictors identified were documentation quality and understandability of the applications. Conclusions: The results show that CBR can be used by software companies to formalize and improve their process of maintainability prediction. Future work involves gathering more data and also employing other prediction techniques.

Background: Despite the number of Web effort estimation techniques investigated, there is no consensus as to which technique produces the most accurate estimates, an issue shared by effort estimation in the general software estimation domain. A previous study in this domain has shown that using ensembles of estimation techniques can be used to address this issue. Aim: The aim of this paper is to investigate whether ensembles of effort estimation techniques will be similarly successful when used on Web project data. Method: The previous study built ensembles using solo effort estimation techniques that were deemed superior. In order to identify these superior techniques two approaches were investigated: The first involved replicating the methodology used in the previous study, while the second approach used the Scott-Knott algorithm. Both approaches were done using the same 90 solo estimation techniques on Web project data from the Tukutuku dataset. The replication identified 16 solo techniques that were deemed superior and were used to build 15 ensembles, while the Scott-Knott algorithm identified 19 superior solo techniques that were used to build two ensembles. Results: The ensembles produced by both approaches performed very well against solo effort estimation techniques. With the replication, the top 12 techniques were all ensembles, with the remaining 3 ensembles falling within the top 17 techniques. These 15 effort estimation ensembles, along with the 2 built by the second approach, were grouped into the best cluster of effort estimation techniques by the Scott-Knott algorithm. Conclusion: While it may not be possible to identify a single best technique, the results suggest that ensembles of estimation techniques consistently perform well even when using Web project data.

One of the pillars for sound Software Project Management is reliable effort estimation. Therefore it is important to fully identify what are the fundamental factors that affect an effort estimate for a new project and how these factors are inter-related. This paper describes a case study where a Bayesian Network model to estimate effort for healthcare software projects was built. This model was solely elicited from expert knowledge, with the participation of seven project managers, and was validated using data from 22 past finished projects. The model led to numerous changes in process and also in business. The company adapted their existing effort estimation process to be in line with the model that was created, and the use of a mathematically-based model also led to an increase in the number of projects being delegated to this company by other company branches worldwide.

Recent studies have reported that Support Vector Regression (SVR) has the potential as a technique for software development effort estimation. However, its prediction accuracy is heavily influenced by the setting of parameters that needs to be done when employing it. No general guidelines are available to select these parameters, whose choice also depends on the characteristics of the dataset being used. This motivated the work described in (Corazza et al. 2010 ), extended herein. In order to automatically select suitable SVR parameters we proposed an approach based on the use of the meta-heuristics Tabu Search (TS). We designed TS to search for the parameters of both the support vector algorithm and of the employed kernel function, namely RBF. We empirically assessed the effectiveness of the approach using different types of datasets (single and cross-company datasets, Web and not Web projects) from the PROMISE repository and from the Tukutuku database. A total of 21 datasets were employed to perform a 10-fold or a leave-one-out cross-validation, depending on the size of the dataset. Several benchmarks were taken into account to assess both the effectiveness of TS to set SVR parameters and the prediction accuracy of the proposed approach with respect to widely used effort estimation techniques. The use of TS allowed us to automatically obtain suitable parameters’ choices required to run SVR. Moreover, the combination of TS and SVR significantly outperformed all the other techniques. The proposed approach represents a suitable technique for software development effort estimation.

Software maintainability is a very important quality attribute. Its prediction for relational database-driven software applications can help organizations improve the maintainability of these applications. The research presented herein adopts a survey-based approach where a survey was conducted with 40 software professionals aimed at identifying and ranking the important maintainability predictors for relational database-driven software applications. The survey results were analyzed using frequency analysis. The results suggest that maintainability prediction for relational database-driven applications is not the same as that of traditional software applications in terms of the importance of the predictors used for this purpose. The results also provide a baseline for creating maintainability prediction models for relational database-driven software applications.

To date most research in software effort estimation has not taken into account any form of chronological split when selecting projects for training and testing sets. A chronological split represents the use of a project's starting and completion dates, such that any model that estimates effort for a new project p only uses as its training set projects that were completed prior to p's starting date. Three recent studies investigated the use of chronological splits, using a type of chronological split called a moving window, which represented a subset of the most recent projects completed prior to a project p's starting date. They found some evidence in favour of using windows whenever projects were recent. These studies all defined window sizes as being fixed numbers of recent projects. In practice, we suggest that estimators are more likely to think in terms of elapsed time than the size of the data set, when deciding which projects to include in a training set. Therefore, this paper investigates the effect on accuracy when using moving windows of various durations to form training sets on which to base effort estimates. Our results show that the use of windows based on duration can affect the accuracy of estimates (in this data set, a window of about three years duration appears best), but to a lesser extent than windows based on a fixed number of projects.

OBJECTIVE - The objective of this paper is to describe a case study where Bayesian Networks (BNs) were used to construct an expert-based Web effort model. METHOD - We built a single-company BN model solely elicited from expert knowledge, where the domain expert was an experienced Web project manager from a small Web company in Auckland, New Zealand. This model was validated using data from 22 past finished Web projects. RESULTS - The BN model has to date been successfully used to estimate effort for numerous Web projects. CONCLUSIONS - Our results suggest that, at least for the Web Company that participated in this case study, the use of a model that allows the representation of uncertainty, inherent in effort estimation, can outperform expert-based estimates. Another nine companies have also benefited from using Bayesian Networks, with very promising results.

This paper details our experience building and validating six different expert-based Web effort estimation models for ICT companies in New Zealand and Brazil. All models were created using Bayesian networks, via eliciting knowledge from domain experts, and validated using data from past finished projects. Post-mortem interviews with the participating companies showed that they found the entire process extremely beneficial and worthwhile, and that all the models created remained in use by those companies.

This research adopts a new scenario-based black box testing methodology for testing web applications. It combines a black box testing strategy with the functions (scenarios) measured by the COSMIC-FFP measurement procedure (ISO/IEC 19761 standard) to produce an optimal set of test cases. This testing approach shows its applicability during all the development phases. Moreover, it can be applied during the early development phase once the specifications have been documented as well as after the development phase where we don't have the access to the code. This paper also considers the use of a functional complexity measure for assigning priorities to the generated test cases. Finally, those concepts have been applied on part of Online Banking System as a case study.

This paper describes a formal experiment carried out to investigate the effect of the personality factor Openness to experience on the academic performance of students who practiced pair programming (PP) in higher education. The experiment was carried out at the University of Auckland, using as subjects undergraduate students attending an introductory software programming course. Our results showed that differences in Openness level could significantly affect academic performance of students who pair programmed. In addition, our results also showed that most students gained higher satisfaction from the PP experience and their confidence level in solving programming exercises was also high.

Objective-The objective of this paper is to present the current state of research in Software Process Improvement (SPI) models and techniques used by small and medium Web development companies. Method-A previously conducted systematic review (Sulayman and Mendes, 2009) has been updated and extended to identify the current state of research on the mentioned topic. Results Eight studies applied SPI techniques or models to small and medium Web companies. The selected studies did not suggest any customized model or technique to measure the SPI of small and medium Web companies. The characteristics of some small and medium companies such as tight budget constraints, tight deadlines and a short term strategy were also identified. The measures of success for small and medium Web companies, as per SR results, include development team and client satisfaction, increase in productivity, compliance with standards and overall operational excellence. Conclusions-The results of this review, like our previously conducted systematic review (Sulayman and Mendes, 2009), show that very few studies have specifically focused on SPI for Web companies, despite the large number of existing Web companies worldwide, and the even larger number of Web applications being currently developed. This clearly indicates a research gap in this area, with numerous avenues for future work.

Background: A systematic literature review (SLR) is a methodology used to aggregate all relevant existing evidence to answer a research question of interest. Although crucial, the process used to select primary studies can be arduous, time consuming, and must often be conducted manually. Objective: We propose a novel approach, known as 'Systematic Literature Review based on Visual Text Mining' or simply SLR-VTM, to support the primary study selection activity using visual text mining (VTM) techniques. Method: We conducted a case study to compare the performance and effectiveness of four doctoral students in selecting primary studies manually and using the SLR-VTM approach. To enable the comparison, we also developed a VTM tool that implemented our approach. We hypothesized that students using SLR-VTM would present improved selection performance and effectiveness. Results: Our results show that incorporating VTM in the SLR study selection activity reduced the time spent in this activity and also increased the number of studies correctly included. Conclusions: Our pilot case study presents promising results suggesting that the use of VTM may indeed be beneficial during the study selection activity when performing an SLR.

The objective of this paper is to present the current evidence relative to the effectiveness of pair programming (PP) as a pedagogical tool in higher education CS/SE courses. We performed a systematic literature review (SLR) of empirical studies that investigated factors affecting the effectiveness of PP for CS/SE students and studies that measured the effectiveness of PP for CS/SE students. Seventy-four papers were used in our synthesis of evidence, and 14 compatibility factors that can potentially affect PP's effectiveness as a pedagogical tool were identified. Results showed that students' skill level was the factor that affected PP's effectiveness the most. The most common measure used to gauge PP's effectiveness was time spent on programming. In addition, students' satisfaction when using PP was overall higher than when working solo. Our meta-analyses showed that PP was effective in improving students' grades on assignments. Finally, in the studies that used quality as a measure of effectiveness, the number of test cases succeeded, academic performance, and expert opinion were the quality measures mostly applied. The results of this SLR show two clear gaps in this research field: 1) a lack of studies focusing on pair compatibility factors aimed at making PP an effective pedagogical tool and 2) a lack of studies investigating PP for software design/modeling tasks in conjunction with programming tasks.

This paper reports on an empirical study that investigates the effects of the personality trait of neuroticism on the academic performance of students who practiced pair programming during one academic semester. The experiment was conducted at The University of Auckland involving 270 first year undergraduate students enrolled in an introductory programming course. In this study, we hypothesized that neuroticism or lack of 'emotional stability' potentially affects pair students' academic performance. However, from the analysis of our results we found lack of evidence to support this. A correlation analysis showed significant positive associations between the conscientiousness personality trait and almost all performance criteria, thus corroborating evidence reported in the educational psychology literature.

This paper describes a formal experiment carried out to investigate the effect of the Personality factor conscientiousness on the effectiveness of Pair Programming as a pedagogical tool in higher Education. This experiment took place at the University of Auckland, using as subjects undergraduate students attending an introductory programming course. Conscientiousness was chosen because it has been shown to be the most consistent predictor of academic achievement. Our findings showed that differences in conscientiousness level did not significantly affect the academic performance of students who pair programmed, which could have been due to the short duration of the tasks used throughout the experiment. However, results revealed that another Personality factor - Openness to experience - presented a significant correlation with paired students' academic performance.

BACKGROUND-The systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE-The aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular, we wish to investigate the consistency of process and the stability of outcomes. METHOD-We compare the results of two independent reviews undertaken with a common research question. RESULTS-The two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS-In addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method.

Support vector regression (SVR) is a new generation of machine learning algorithms, suitable for predictive data modeling problems. The objective of this paper is to investigate the effectiveness of SVR for Web effort estimation, in particular when dealing with a cross-company dataset. To gain a deeper insight on the method, we carried out an empirical study using four kernels for SVR, namely linear, polynomial, Gaussian, and sigmoid. Moreover, we used two variables' preprocessing strategies (normalization and logarithmic), and two different dependent variables (effort and inverse effort). As a result, SVR was applied using six different configurations for each kernel. As for the dataset, we employed the Tukutuku database, which is widely adopted in Web effort estimation studies. A hold-out approach was adopted to evaluate the prediction accuracy for all the configurations, using two training sets, each containing data on 130 projects randomly selected, and two test sets, each containing the remaining 65 projects. As benchmark, SVR-based predictions were also compared to predictions obtained using manual stepwise regression, case-based reasoning, and Bayesian networks. Our results suggest that SVR performed well, since on the first hold-out, the linear kernel with a logarithmic transformation of variables provided significantly superior prediction accuracy than all the other techniques, while for the second hold-out, the Gaussian kernel achieved significantly superior predictions than all other techniques, except for manual stepwise regression.

Models for estimating software development effort are commonly built and evaluated using a set of historical projects. An important question is which projects to use as training data to build the model: should it be all of them, or a subset that seems particularly relevant? One factor to consider is project age: is it best to use the entire history of past projects, or is it more appropriate in a rapidly changing world to use a window of recent projects? We investigate the effect on estimation accuracy of using a moving window, using projects from the ISBSG data set. We find that using a moving window can improve accuracy, and we make some observations about factors that influence the range of possible window sizes and the best window size.

This paper presents the results of a systematic review conducted to collect evidence on software maintainability prediction and metrics. The study was targeted at the software quality attribute of maintainability as opposed to the process of software maintenance. The evidence was gathered from the selected studies against a set of meaningful and focused questions. 710 studies were initially retrieved; however of these only 15 studies were selected; their quality was assessed; data extraction was performed; and data was synthesized against the research questions. Our results suggest that there is little evidence on the effectiveness of software maintainability prediction techniques and models.

Pair Programming (PP) has been long researched in industry and academia. Although research evidence about its usefulness is somewhat inconclusive, previous studies showed that its use in an academic environment can benefit students in programming and design courses. In our study, we investigated the ldquohumanrdquo aspect of PP; in particular the effects that personality attributes may have on PP's effectiveness as a pedagogical tool. We conducted a formal experiment at the University of Auckland to investigate the influence of personality differences among paired students using the five-factor model as a personality measurement framework. The aim of our study was to improve the implementation of PP as a pedagogical tool through understanding the impact the variation in the personality profile of paired students has towards their academic performance. Our findings showed that differences in personality traits did not significantly affect the academic performance of students who pair programmed.

The objective of this paper is to further investigate the use of Bayesian Networks (BN) for Web effort estimation when using a cross-company dataset. Four BNs were built; two automatically using the Hugin tool with two training sets; two using a structure elicited by a domain expert, with parameters obtained from automatically fitting the network to the same training sets used in the automated elicitation (hybrid models). The accuracy of all four models was measured using two validation sets, and point estimates. As a benchmark, the BN-based predictions were also compared to predictions obtained using Manual StepWise Regression (MSWR), and Case-Based Reasoning (CBR). The BN model generated using Hugin presented similar accuracy to CBR and Mean effort-based predictions. Our results suggest that Hybrid BN models can provide significantly superior prediction accuracy. However, good results also seem to depend on characteristics of the training and validation sets used.

The objective of this paper is to compare, using a cross-company dataset, several Bayesian network (BN) models for Web effort estimation. Eight BNs were built; four automatically using Hugin and PowerSoft tools with two training sets, each with 130 Web projects from the Tukutuku database; four using a causal graph elicited by a domain expert, with parameters automatically fit using the same training sets used in the automated elicitation (hybrid models). Their accuracy was measured using two validation sets, each containing data on 65 projects, and point estimates. As a benchmark, the BN-based estimates were also compared to estimates obtained using manual stepwise regression (MSWR), case-based reasoning (CBR), mean- and median-based effort models. MSWR presented significantly better predictions than any of the BN models built herein, and in addition was the only technique to provide significantly superior predictions to a median-based effort model. This paper investigated data-driven and hybrid BN models using project data from the Tukutuku database. Our results suggest that the use of simpler models, such as the median effort, can outperform more complex models, such as BNs. In addition, MSWR seemed to be the only effective technique for Web effort estimation.

Given the growth in the number and size of Web Applications worldwide, Web quality assurance, and more specifically Web usability have become key success factors. Therefore, this work proposes a usability evaluation technique based on the combination of Web design perspectives adapted from existing literature, and heuristics. This new technique is assessed using a controlled experiment aimed at measuring the efficiency and effectiveness of our technique, in comparison to Nielsen's heuristic evaluation. Results indicated that our technique was significantly more effective than and as efficient as Nielsen's heuristic evaluation.

Size represents one of the most important attribute of software products used to predict software development effort. In the past nine years, several measures have been proposed to estimate the size of Web applications, and it is important to determine which one is most effective to predict Web development effort. To this aim in this paper we report on an empirical analysis where, using data from 15 Web projects developed by a software company, we compare four sets of size measures, using two prediction techniques, namely Forward Stepwise Regression (SWR) and Case-Based Reasoning (CBR). All the measures provided good predictions in terms of MMRE, MdMRE, and Pred(0.25) statistics, for both SWR and CBR. Moreover, when using SWR, length measures and Web Objects gave significant better results than Functional measures, however presented similar results to the Tukutuku measures. As for CBR, results did not show any significant differences amongst the four sets of size measures.

The objective of this paper is to extend the work by Mendes (2007), and to compare four techniques for Web effort estimation to identify which one provides best prediction accuracy. We employed four effort estimation techniques - Bayesian networks (BN), forward stepwise regression (SWR), case-based reasoning (CBR) and Classification and regression trees (CART) to obtain effort estimates. The dataset employed was of 150 Web projects from the Tukutuku dataset. Results showed that predictions obtained using a BN were significantly superior to those using other techniques. A model that incorporates the uncertainty inherent in effort estimation, can outperform other commonly used techniques, such as those used in this study.

Previous studies comparing the prediction accuracy of effort models built using Web cross- and single-company data sets have been inconclusive, and as such replicated studies are necessary to determine under what circumstances a company can place reliance on a cross-company effort model. This paper therefore replicates a previous study by investigating how successful a cross-company effort model is: i) to estimate effort for Web projects that belong to a single company and were not used to build the cross-company model; ii) compared to a single-company effort model. Our single-company data set had data on 15 Web projects from a single company and our cross-company data set had data on 68 Web projects from 25 different companies. The effort estimates used in our analysis were obtained by means of two effort estimation techniques, namely forward stepwise regression and case-based reasoning. Our results were similar to those from the replicated study, showing that predictions based on the single-company model were significantly more accurate than those based on the cross-company model.

The objective of this paper is to determine under what circumstances individual organizations would be able to rely on cross-company-based estimation models. We performed a systematic review of studies that compared predictions from cross-company models with predictions from within-company models based on analysis of project data. Ten papers compared cross-company and within-company estimation models; however, only seven presented independent results. Of those seven, three found that cross-company models were not significantly different from within-company models, and four found that cross-company models were significantly worse than within-company models. Experimental procedures used by the studies differed making it impossible to undertake formal meta-analysis of the results. The main trend distinguishing study results was that studies with small within-company data sets (i.e., $20 projects) that used leave-one-out cross validation all found that the within-company model was significantly different (better) from the cross-company model. The results of this review are inconclusive. It is clear that some organizations would be ill-served by cross-company models whereas others would benefit. Further studies are needed, but they must be independent (i.e., based on different data bases or at least different single company data sets) and should address specific hypotheses concerning the conditions that would favor cross-company or within-company models. In addition, experimenters need to standardize their experimental procedures to enable formal meta-analysis, and recommendations are made in Section 3.

OBJECTIVE – The objective of this paper is to determine under what circumstances individual organisations would be able to rely on cross-company based estimation models. METHOD – We performed a systematic review of studies that compared predictions from cross-company models with predictions from within-company models based on analysis of project data. RESULTS – Ten papers compared cross-company and within-company estimation models, however, only seven of the papers presented independent results. Of those seven, three found that cross-company models were as good as within-company models, four found cross-company models were significantly worse than within-company models. Experimental procedures used by the studies differed making it impossible to undertake formal meta-analysis of the results. The main trend distinguishing study results was that studies with small single company data sets (i.e. <20 projects) that used leave-one-out cross-validation all found that the within-company model was significantly more accurate than the cross-company model. CONCLUSIONS – The results of this review are inconclusive. It is clear that some organisations would be ill-served by cross-company models whereas others would benefit. Further studies are needed, but they must be independent (i.e. based on different data bases or at least different single company data sets). In addition, experimenters need to standardise their experimental procedures to enable formal meta-analysis.

The large growth in the number of large Web applications currently developed brings concerns related to their quality, and more specifically their testing and reliability. Web application testing is still in its infancy and relies mostly upon traditional software coverage testing processes, which are highly impractical. Recent studies have looked at usage-based statistical models for testing Web applications. One of these studies, conducted by Kallepalli and Tian [8], used Unified Markov Models (UMMs), built from Web server access logs, as basis for test case selection. In addition, server error logs were also used to measure a Web application's reliability, and consequently to investigate the effectiveness of UMMs as a suitable testing mechanism. This paper describes two experiments that replicated Kallepalli and Tian's work. Our results showed that, in contrast to [8], multiple set UMMs were needed for trustworthy test case generation. In addition, our reliability assessment corroborated results from [8], confirming that UMMs seem to be a suitable testing mechanism to use to test Web applications.

This paper presents the results of a replicated pair programming experiment conducted at the University of Auckland (NZ) during the first semester of 2005. It involved 190 second year Computer Science students attending a software design and construction course. We replicated the experiment described in [18], investigating similar issues to those reported in [32] and employing a subset of the questionnaires used in [32]. Our results confirm the use of pair programming as an effective programming/design learning technique.

This paper presents the results of a pair programming experiment conducted at the University of Auckland (NZ) during the first semester of 2004. It involved 300 second year Computer Science students attending a software design and construction course. We investigated similar issues to those reported in [26] and employed a subset of the questionnaires used by Laurie Williams et al. on the experiments presented in [26]. Our results support the use of pair programming as an effective programming/design learning technique.

Comprehension of an object-oriented (OO) system, its design and use of OO features such as aggregation, generalisation and other forms of association is a difficult task to undertake without the original design documentation for reference. In this paper, we describe the collection of high-level class metrics from the UML design documentation of five industrial-sized C++ systems. Two of the systems studied were libraries of reusable classes. Three hypotheses were tested between these high-level features and the low-level class features of a number of class methods and attributes in each of the five systems. A further two conjectures were then investigated to determine features of key classes in a system and to investigate any differences between library-based systems and the other systems studied in terms of coupling. Results indicated that, for the three application-based systems, no clear patterns emerged for hypotheses relating to generalisation. There was, however, a clear (positive) statistical significance for all three systems studied between aggregation, other types of association and the number of methods and attributes in a class. Key classes in the three application-based systems tended to contain large numbers of methods, attributes, and associations, significant amounts of aggregation but little inheritance. No consistent, identifiable key features could be found in the two library-based systems; both showed a distinct lack of any form of coupling (including inheritance) other than through the C++ friend facility.

Productivity measures based on a simple ratio of product size to project effort assume that size can be determined as a single measure. If there are many possible size measures in a data set and no obvious model for aggregating the measures into a single measure, we propose using the expression AdjustedSize/Effort to measure productivity. AdjustedSize is defined as the most appropriate regression-based effort estimation model, where all the size measures selected for inclusion in the estimation model have a regression parameter significantly different from zero (p<0.05). This productivity measurement method ensures that each project has an expected productivity value of one. Values between zero and one indicate lower than expected productivity, values greater than one indicate higher than expected productivity. We discuss the assumptions underlying this productivity measurement method and present an example of its use for Web application projects. We also explain the relationship between effort prediction models and productivity models.

Literature shows that over the years numerous techniques for estimating development effort have been suggested, derived from late project measures. However, to the successful management of software projects, estimates are necessary throughout the whole development life cycle. The objective is twofold. First, we describe the application of case-based reasoning (CBR) for estimating Web hypermedia development effort using measures collected at different stages in the development cycle. Second, we compare the prediction accuracy of those measures, obtained using different CBR configurations. Contrary to the expected, late measures did not show statistically significant better predictions than early measures.

Over the years software engineering researchers have suggested numerous techniques for estimating development effort. These techniques have been classified mainly as algorithmic, machine learning and expert judgement. Several studies have compared the prediction accuracy of those techniques, with emphasis placed on linear regression, stepwise regression, and Case-based Reasoning (CBR). To date no converging results have been obtained and we believe they may be influenced by the use of the same CBR configuration.The objective of this paper is twofold. First, to describe the application of case-based reasoning for estimating the effort for developing Web hypermedia applications. Second, comparing the prediction accuracy of different CBR configurations, using two Web hypermedia datasets.Results show that for both datasets the best estimations were obtained with weighted Euclidean distance, using either one analogy (dataset 1) or 3 analogies (dataset 2). We suggest therefore that case-based reasoning is a candidate technique for effort estimation and, with the aid of an automated environment, can be applied to Web hypermedia development effort prediction.

Hypermedia engineering constitutes the employment of an engineering approach to the development of hypermedia applications. Its main teaching objectives are for students to learn what an engineering approach means and how measurement can be applied.This paper presents the application of the Cognitive Flexibility Theory as an instructional theory to teach Hypermedia Engineering principles.Early results have shown that students presented a greater learning variability (suggested by their exam marks) when exposed to the CFT as a teaching practice, compared to conventional methods.