In supporting the development of high-quality software, especially necessary in the era of LLMs, automated program repair (APR) tools aim to improve code quality by automatically addressing violations detected by static analysis profilers. Previous research tends to evaluate APR tools only for their ability to clear violations, neglecting their potential introduction of new (sometimes severe) violations, changes to code functionality and degrading of code structure. There is thus a need for research to develop and assess comprehensive evaluation frameworks for APR tools. This study addresses this research gap, and evaluates Sorald (a state-of-the-art APR tool) as a proof of concept. Sorald’s effectiveness was evaluated in repairing 3,529 SonarQube violations across 30 rules within 2,393 Java code snippets extracted from Stack Overflow. Outcomes show that while Sorald fixes specific rule violations, it introduced 2,120 new faults (32 bugs, 2088 code smells), reduced code functional correctness—as evidenced by a 24% unit test failure rate—and degraded code structure, demonstrating the utility of our framework. Findings emphasize the need for evaluation methodologies that capture the full spectrum of APR tool effects, including side effects, to ensure their safe and effective adoption.

Abstract Software cross-project defect prediction (CPDP) makes use of cross-project (CP) data to overcome the lack of data necessary to train well-performing software defect prediction (SDP) classifiers in the early stage of new software projects. Since the CP data (known as the source) may be different from the new project’s data (known as the target), this makes it difficult for CPDP classifiers to perform well. In particular, it is a mismatch of data distributions between source and target that creates this difficulty. Transfer learning-based CPDP classifiers are designed to minimize these distribution differences. The first Transfer learning-based CPDP classifiers treated these differences equally, thereby degrading prediction performance. To this end, recent research has the Weighted Balanced Distribution Adaptation (W-BDA) method to leverage the importance of both distribution differences to improve classification performance. Although W-BDA has been shown to improve model performance in CPDP and tackle the class imbalance by balancing the class proportion of each domain, research to date has failed to consider model performance in light of increasing target data. We provide the first investigation studying the effects of increasing the target data when leveraging the importance of both distribution differences. We extend the initial W-BDA method and call this extension the W-BDA $$\mathbf {^{+}}$$ + method. To evaluate the effectiveness of W-BDA $$\mathbf {^{+}}$$ + for improving CPDP performance, we conduct eight experiments on 18 projects from four datasets, where data sampling was performed with different sampling methods. Data sampling was only performed on the baseline methods and not on our proposed W-BDA $$\mathbf {^{+}}$$ + and the original W-BDA because data sampling issues do not exist for these two methods. We evaluate our method using four complementary indicators (i.e., Balanced Accuracy, AUC, F-measure and G-Measure). Our findings reveal an average improvement of 6%, 7.5%, 10% and 12% for these four indicators when W-BDA $$\mathbf {^{+}}$$ + is compared to the original W-BDA and five other baseline methods (for all four of the sampling methods used). Also, as the target to source ratio is increased with different sampling methods, we observe a decrease in performance for the original W-BDA, with our W-BDA $$\mathbf {^{+}}$$ + approach outperforming the original W-BDA in most cases. Our results highlight the importance of having an awareness of the effect of the increasing availability of target data in CPDP scenarios when using a method that can handle the class imbalance problem.

Abstract Just-In-Time (JIT) defect prediction aims to identify defects early, at commit time. Hence, developers can take precautions to avoid defects when the code changes are still fresh in their minds. However, the utility of JIT defect prediction has not been investigated in relation to crashes of mobile apps. We therefore conducted a multi-case study employing both quantitative and qualitative analysis. In the quantitative analysis, we used machine learning techniques for prediction. We collected 113 reliability-related metrics for about 30,000 commits from 14 Android apps and selected 14 important metrics for prediction. We found that both standard JIT metrics and static analysis warnings are important for JIT prediction of mobile app crashes. We further optimized prediction performance, comparing seven state-of-the-art defect prediction techniques with hyperparameter optimization. Our results showed that Random Forest is the best performing model with an AUC-ROC of 0.83. In our qualitative analysis, we manually analysed a sample of 642 commits and identified different types of changes that are common in crash-inducing commits. We explored whether different aspects of changes can be used as metrics in JIT models to improve prediction performance. We found these metrics improve the prediction performance significantly. Hence, we suggest considering static analysis warnings and Android-specific metrics to adapt standard JIT defect prediction models for a mobile context to predict crashes. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.

Software developers make use of on crowdsourcing during development. Beyond learning from others, developers use online portals such as Stack Overflow as a vehicle for collaboration. However, little is known about developers’ experiences on such platforms, particularly around problems that are encountered online. Such insights could benefit software developers in terms of recommendations for pitfalls to avoid, ways to exploit crowdsourced knowledge, and the provision of insights to improve online code sharing communities. We interviewed 50 practitioners to fill this gap, where outcomes show that software developers’ use of online portals is targeted, and such portals are a lifeline to modern software development. Practitioners are facilitated with code solutions and debugging, often in a very timely fashion. While these experiences are largely positive, practitioners also encounter negative experiences online, some of which could be significantly deleterious to the community. We discuss the implications of these findings, such as creating awareness of the quality and reliability of code snippets, improving code searches, code validation and outdated code detection and attribution of code snippets.

Cross-project defect prediction (CPDP) models are used in new software project prediction tasks to improve defect prediction rates. The development of these CPDP models could be challenging in cases where there is little or no historical data. For this reason, researchers may need to rely on multiple sources and use transfer learning-based CPDP for building defect prediction models. These data are typically taken from similar and related projects, but their distributions can be different from the new software project (target data). Although, transfer learning-based CPDP models are designed to handle these distribution differences, but if not correctly handled by the model, may lead to negative transfer. To this end, recent works have focused on building transfer CPDP models, but little is known about how similar or dissimilar sources should be to avoid negative transfer. This paper provides the first empirical investigation to understand the effect of combining different sources with different levels of similarities in transfer CPDP. We introduce the use of the Population Stability Index (PSI) to interpret whether the distribution of the combined or single-source data is similar to the target data. This was validated using an adversarial approach. Experimental results on three public datasets reveal that when the source and target distribution are very similar, the probability of false alarm is improved by 3% to 7% and the recall indicator is reduced from 1% to 8%. Interestingly, we also found that when dissimilar source data are combined with different source datasets, the overall domain divergence is lowered, and the performance is improved. The results highlight the importance of using the right source to aid the learning process.

Stack Overflow is used to solve programming issues during software development. Research efforts have looked to identify relevant content on this platform. In particular, researchers have proposed various modelling techniques to predict acceptable Stack Overflow answers. Less interest, however, has been dedicated to examining the performance and quality of typically used modelling methods with respect to the model and feature complexity. Such insights could be of practical significance to the many practitioners who develop models for Stack Overflow. This study examines the performance and quality of two modelling methods, of varying degree of complexity, used for predicting Java and JavaScript acceptable answers on Stack Overflow. Our dataset comprised 249,588 posts drawn from years 2014-2016. Outcomes reveal significant differences in models’ performances and quality given the type of features and complexity of models used. Researchers examining model performance and quality and feature complexity may leverage these findings in selecting suitable modelling approaches for Q&A prediction.

Abstract Cross‐project defect prediction (CPDP), where data from different software projects are used to predict defects, has been proposed as a way to provide data for software projects that lack historical data. Evaluations of CPDP models using the Nearest Neighbour (NN) Filter approach have shown promising results in recent studies. A key challenge with defect‐prediction datasets is class imbalance, that is, highly skewed datasets where non‐buggy modules dominate the buggy modules. In the past, data resampling approaches have been applied to within‐projects defect prediction models to help alleviate the negative effects of class imbalance in the datasets. To address the class imbalance issue in CPDP, the authors assess the impact of data resampling approaches on CPDP models after the NN Filter is applied. The impact on prediction performance of five oversampling approaches (MAHAKIL, SMOTE, Borderline‐SMOTE, Random Oversampling and ADASYN) and three undersampling approaches (Random Undersampling, Tomek Links and One‐sided selection) is investigated and results are compared to approaches without data resampling. The authors examined six defect prediction models on 34 datasets extracted from the PROMISE repository. The authors' results show that there is a significant positive effect of data resampling on CPDP performance, suggesting that software quality teams and researchers should consider applying data resampling approaches for improved recall ( pd ) and g ‐measure prediction performance. However, if the goal is to improve precision and reduce false alarm ( pf ) then data resampling approaches should be avoided.

Together with many success stories, promises such as the increase in production speed and the improvement in stakeholders’ collaboration have contributed to making agile a transformation in the software industry in which many companies want to take part. However, driven either by a natural and expected evolution or by contextual factors that challenge the adoption of agile methods as prescribed by their creator(s), software processes in practice mutate into hybrids over time. Are these still agile? In this article, we investigate the question: what makes a software development method agile? We present an empirical study grounded in a large-scale international survey that aims to identify software development methods and practices that improve or tame agility. Based on 556 data points, we analyze the perceived degree of agility in the implementation of standard project disciplines and its relation to used development methods and practices. Our findings suggest that only a small number of participants operate their projects in a purely traditional or agile manner (under 15 percent). That said, most project disciplines and most practices show a clear trend towards increasing degrees of agility. Compared to the methods used to develop software, the selection of practices has a stronger effect on the degree of agility of a given discipline. Finally, there are no methods or practices that explicitly guarantee or prevent agility. We conclude that agility cannot be defined solely at the process level. Additional factors need to be taken into account when trying to implement or improve agility in a software company. Finally, we discuss the field of software process-related research in the light of our findings and present a roadmap for future research.

Abstract Hardly any software development process is used as prescribed by authors or standards. Regardless of company size or industry sector, a majority of project teams and companies use hybrid development methods (short: hybrid methods) that combine different development methods and practices. Even though such hybrid methods are highly individualized, a common understanding of how to systematically construct synergetic practices is missing. In this article, we make a first step towards a statistical construction procedure for hybrid methods. Grounded in 1467 data points from a large‐scale practitioner survey, we study the question: What are hybrid methods made of and how can they be systematically constructed? Our findings show that only eight methods and few practices build the core of modern software development. Using an 85% agreement level in the participants' selections, we provide examples illustrating how hybrid methods can be characterized by the practices they are made of. Furthermore, using this characterization, we develop an initial construction procedure, which allows for defining a method frame and enriching it incrementally to devise a hybrid method using ranked sets of practice.

We have come to rely on industrial-scale cyber-physical systems more and more to manage tasks and machinery in safety-critical situations. Efficient, reliable fault identification and management has become a critical factor in the design of these increasingly sophisticated and complex devices.Teams of co-operating software agents are one way to co-ordinate the flow of diagnostic information gathered during fault-finding. By wielding domain knowledge of the software architecture used to construct the system, agents build and refine their beliefs about the location and root cause of faults.This paper examines how agents constructed within the GORITE Multi-Agent Framework create and refine their beliefs. We demonstrate three different belief structures implemented within our Fault Diagnostic Engine, showing how each supports a distinct aspect of the agent's reasoning. Using domain knowledge of the IEC 61499 Function Block architecture, agents are able to examine and rigorously evaluate both individual components and entire sub-systems.

Integrating the design and creation of fault identification and diagnostic capabilities into Model-Driven Development methodologies is one approach to enhancing the resilience of Industrial Cyber-Physical Systems. We present a Fault Diagnostic Engine designed to recognise and diagnose faults in IEC 61499 Function Block Applications. Using diagnostic agents that interact directly with the target application, we demonstrate fault monitoring and analysis tech-niques and as well as failure scenario intervention. By designing and building fault diagnostic resources during early phases of Model-Driven Development, both iterative testing and long-term fault management capabilities can be created. While applying and refining appropriate model artifacts, we demonstrate that the concurrent development of function blocks alongside fault management capabilities is both feasible and worthwhile.

Neural saturation in Deep Neural Networks (DNNs) has been studied extensively, but remains relatively unexplored in Convolutional Neural Networks (CNNs). Understanding and alleviating the effects of convolutional kernel saturation is critical for enhancing CNN models classification accuracies. In this paper, we analyze the effect of convolutional kernel saturation in CNNs and propose a simple data augmentation technique to mitigate saturation and increase classification accuracy, by supplementing negative images to the training dataset. We hypothesize that greater semantic feature information can be extracted using negative images since they have the same structural information as standard images but differ in their data representations. Varied data representations decrease the probability of kernel saturation and thus increase the effectiveness of kernel weight updates. The two datasets selected to evaluate our hypothesis were CIFAR-10 and STL-10 as they have similar image classes but differ in image resolutions thus making for a better understanding of the saturation phenomenon. MNIST dataset was used to highlight the ineffectiveness of the technique for linearly separable data. The ResNet CNN architecture was chosen since the skip connections in the network ensure the most important features contributing the most to classification accuracy are retained. Our results show that CNNs are indeed susceptible to convolutional kernel saturation and that supplementing negative images to the training dataset can offer a statistically significant increase in classification accuracies when compared against models trained on the original datasets. Our results present accuracy increases of 6.98% and 3.16% on the STL-10 and CIFAR-10 datasets respectively.

Convolutional Neural Networks (CNNs) specialize in feature extraction rather than function mapping. In doing so they form complex internal hierarchical feature representations, the complexity of which gradually increases with a corresponding increment in neural network depth. In this paper, we examine the feature extraction capabilities of CNNs using Maximum Entropy (ME) and Signal-to-Noise Ratio (SNR) to validate the idea that, CNN models should be tailored for a given task and complexity of the input data. SNR and ME measures are used as they can accurately determine in the input dataset, the relative amount of signal information to the random noise and the maximum amount of information respectively.We use two well known benchmarking datasets, MNIST and CIFAR-10 to examine the information extraction and abstraction capabilities of CNNs. Through our experiments, we examine convolutional feature extraction and abstraction capabilities in CNNs and show that the classification accuracy or performance of CNNs is greatly dependent on the amount, complexity and quality of the signal information present in the input data. Furthermore, we show the effect of information overflow and underflow on CNN classification accuracies. Our hypothesis is that the feature extraction and abstraction capabilities of convolutional layers are limited and therefore, CNN models should be tailored to the input data by using appropriately sized CNNs based on the SNR and ME measures of the input dataset.

Selecting a suitable development method for a specific project context is one of the most challenging activities in process design. Every project is unique and, thus, many context factors have to be considered. Recent research took some initial steps towards statistically constructing hybrid development methods, yet, paid little attention to the peculiarities of context factors influencing method and practice selection. In this paper, we utilize exploratory factor analysis and logistic regression analysis to learn such context factors and to identify methods that are correlated with these factors. Our analysis is based on 829 data points from the HELENA dataset. We provide five base clusters of methods consisting of up to 10 methods that lay the foundation for devising hybrid development methods. The analysis of the five clusters using trained models reveals only a few context factors, e.g., project/product size and target application domain, that seem to significantly influence the selection of methods. An extended descriptive analysis of these practices in the context of the identified method clusters also suggests a consolidation of the relevant practice sets used in specific project contexts.

Software effort estimation models are typically developed based on an underlying assumption that all data points are equally relevant to the prediction of effort for future projects. The dynamic nature of several aspects of the software engineering process could mean that this assumption does not hold in at least some cases. This study employs three kernel estimator functions to test the stationarity assumption in five software engineering datasets that have been used in the construction of software effort estimation models. The kernel estimators are used in the generation of nonuniform weights which are subsequently employed in weighted linear regression modeling. In each model, older projects are assigned smaller weights while the more recently completed projects are assigned larger weights, to reflect their potentially greater relevance to present or future projects that need to be estimated. Prediction errors are compared to those obtained from uniform models. Our results indicate that, for the datasets that exhibit underlying nonstationary processes, uniform models are more accurate than the nonuniform models; that is, models based on kernel estimator functions are worse than the models where no weighting was applied. In contrast, the accuracies of uniform and nonuniform models for datasets that exhibited stationary processes were essentially equivalent. Our analysis indicates that as the heterogeneity of a dataset increases, the effect of stationarity is overridden. The results of our study also confirm prior findings that the accuracy of effort estimation models is independent of the type of kernel estimator function used in model development.

As Industrial Cyber–Physical Systems (ICPS) become more connected and widely-distributed, often operating in safety-critical environments, we require innovative approaches to detect and diagnose the faults that occur in them. We profile fault identification and diagnosis techniques employed in the aerospace, automotive, and industrial control domains. Each of these sectors has adopted particular methods to meet their differing diagnostic needs. By examining both theoretical presentations as well as case studies from production environments, we present a profile of the current approaches being employed and identify gaps. A scoping study was used to identify and compare fault detection and diagnosis methodologies that are presented in the current literature. We created categories for the different diagnostic approaches via a pilot study and present an analysis of the trends that emerged. We then compared the maturity of these approaches by adapting and using the NASA Technology Readiness Level (TRL) scale. Fault identification and analysis studies from 127 papers published from 2004 to 2019 reveal a wide diversity of promising techniques, both emerging and in-use. These range from traditional Physics-based Models to Data-Driven Artificial Intelligence (AI) and Knowledge-Based approaches. Hybrid techniques that blend aspects of these three broad categories were also encountered. Predictive diagnostics or prognostics featured prominently across all sectors, along with discussions of techniques including Fault trees, Petri nets and Markov approaches. We also profile some of the techniques that have reached the highest Technology Readiness Levels, showing how those methods are being applied in real-world environments beyond the laboratory. Our results suggest that the continuing wide use of both Model-Based and Data-Driven AI techniques across all domains, especially when they are used together in hybrid configuration, reflects the complexity of the current ICPS application space. While creating sufficiently-complete models is labor intensive, Model-free AI techniques were evidenced as a viable way of addressing aspects of this challenge, demonstrating the increasing sophistication of current machine learning systems. Connecting ICPS together to share sufficient telemetry to diagnose and manage faults is difficult when the physical environment places demands on ICPS. Despite these challenges, the most mature papers present robust fault diagnosis and analysis techniques which have moved beyond the laboratory and are proving valuable in real-world environments.

Context: Stack Overflow is a popular community question and answer portal used by practitioners to solve problems during software development. Developers can focus their attention on answers that have been accepted or where members have recorded high votes in judging good answers when searching for help. However, the latter mechanism (votes) can be unreliable, and there is currently no way to differentiate between an answer that is likely to be accepted and those that will not be accepted by looking at the answer's characteristics. Objective: In potentially providing a mechanism to identify acceptable answers, this study examines the features that distinguish an accepted answer from an unaccepted answer. Methods: We studied the Stack Overflow dataset by analyzing questions and answers for the two most popular tags (Java and JavaScript). Our dataset comprised 249,588 posts drawn from 2014-2016. We use random forest and neural network models to predict accepted answers, and study the features with the highest predictive power in those two models. Results: Our findings reveal that the length of code in answers, reputation of users, similarity of the text between questions and answers, and the time lag between questions and answers have the highest predictive power for differentiating accepted and unaccepted answers. Conclusion: Tools may leverage these findings in supporting developers and reducing the effort they must dedicate to searching for suitable answers on Stack Overflow.

Many prescriptive approaches to developing software intensive systems have been advocated but each is based on assumptions about context. It has been found that practitioners do not follow prescribed methodologies, but rather select and adapt specific practices according to local needs. As researchers, we would like to be in a position to support such tailoring. However, at the present time we simply do not have sufficient evidence relating practice and context for this to be possible. We have long understood that a deeper understanding of situated software practices is crucial for progress in this area, and have been exploring this problem from a number of perspectives. In this position paper, we draw together the various aspects of our work into a holistic model and discuss the ways in which the model might be applied to support the long term goal of evidence-based decision support for practitioners. The contribution specific to this paper is a discussion on model evaluation, including a proof-of-concept demonstration of model utility. We map Kernel elements from the Essence system to our model and discuss gaps and limitations exposed in the Kernel. Finally, we overview our plans for further refining and evaluating the model.

Context: It is not uncommon for a new team member to join an existing Agile software development team, even after development has started. This new team member faces a number of challenges before they are integrated into the team and can contribute productively to team progress. Ideally, each newcomer should be supported in this transition through an effective team onboarding program, although prior evidence suggests that this is challenging for many organisations. Objective: We seek to understand how Agile teams address the challenge of team onboarding in order to inform future onboarding design. Method: We conducted an interview survey of eleven participants from eight organisations to investigate what onboarding activities are common across Agile software development teams. We also identify common goals of onboarding from a synthesis of literature. A repertory grid instrument is used to map the contributions of onboarding techniques to onboarding goals. Results: Our study reveals that a broad range of team onboarding techniques, both formal and informal, are used in practice. It also shows that particular techniques that have high contributions to a given goal or set of goals. Conclusions: In presenting a set of onboarding goals to consider and an evidence-based mechanism for selecting techniques to achieve the desired goals it is expected that this study will contribute to better-informed onboarding design and planning. An increase in practitioner awareness of the options for supporting new team members is also an expected outcome.

Programmable Logic Controllers (PLCs) execute critical control software that drives Industrial Automation and Control Systems (IACS). PLCs can become easy targets for cyber-adversaries as they are resource-constrained and are usually built using legacy, less-capable security measures. Security attacks can significantly affect system availability, which is an essential requirement for IACS. We propose a method to make PLC applications more security-aware. Based on the well-known IEC 61499 function blocks standard for developing IACS software, our method allows designers to annotate critical parts of an application during design time. On deployment, these parts of the application are automatically secured using appropriate security mechanisms to detect and prevent attacks. We present a summary of availability attacks on distributed IACS applications that can be mitigated by our proposed method. Security mechanisms are achieved using IEC 61499 Service-Interface Function Blocks (SIFBs) embedding Intrusion Detection and Prevention System (IDPS), added to the application at compile time. This method is more amenable to providing active security protection from attacks on previously unknown (zero-day) vulnerabilities. We test our solution on an IEC 61499 application executing on Wago PFC200 PLCs. Experiments show that we can successfully log and prevent attacks at the application level as well as help the application to gracefully degrade into safe mode, subsequently improving availability.

Among the multitude of software development processes available, hardly any is used by the book. Regardless of company size or industry sector, a majority of project teams and companies use customized processes that combine different development methods--so-called hybrid development methods. Even though such hybrid development methods are highly individualized, a common understanding of how to systematically construct synergetic practices is missing. In this paper, we make a first step towards devising such guidelines. Grounded in 1,467 data points from a large-scale online survey among practitioners, we study the current state of practice in process use to answer the question: What are hybrid development methods made of? Our findings reveal that only eight methods and few practices build the core of modern software development. This small set allows for statistically constructing hybrid development methods. Using an 85% agreement level in the participants' selections, we provide two examples illustrating how hybrid development methods are characterized by the practices they are made of. Our evidence-based analysis approach lays the foundation for devising hybrid development methods.

BACKGROUND: In object oriented (OO) software systems, class size has been acknowledged as having an indirect effect on the relationship between certain artifact characteristics, captured via metrics, and fault-proneness, and therefore it is recommended to control for size when designing fault prediction models.

Cyber-attacks on Industrial Automation and Control Systems (IACS) are rising in numbers and sophistication. Embedded controller devices such as Programmable Logic Controllers (PLCs), which are central to controlling physical processes, must be secured against attacks on confidentiality, integrity and availability. The focus of this paper is to add design-level support for security in IACS applications, especially around inter-PLC communications. We propose an end-to-end solution to develop IACS applications with inherent, and parametric support for security. Built using the IEC 61499 Function Blocks standard, this solution allows us to annotate certain communications as ‘secure’ during design time. When the application is compiled, these annotations are transformed into a security layer that implements encrypted communication between PLCs. In this paper, we implement a part of this security layer focussed on confidentiality, called Confidentiality Layer for Function Blocks (CL4FB), which provides a range of encryption/decryption and secure key exchange functionalities. We study the impact of using CL4FB in IACS applications with real-time constraints. Through a case study focussing on protection functions in smart-grids, we show that varying levels of confidentiality can be achieved while also meeting hard real-time deadlines.

Design erosion is a persistent problem within the software engineering discipline. Software designs tend to deteriorate over time and there is a need for tools and techniques that support software architects when dealing with legacy systems. This paper presents an evaluation of a search-based software engineering (SBSE) approach intended to recover high-level architecture designs of software systems by structuring low-level artifacts into high-level architecture artifact configurations. In particular, this paper describes the performance evaluation of a number of metaheuristic search algorithms applied to architecture reconstruction problems with high dimensionality in terms of objectives. These problems have been selected as representative of the typical challenges faced by software architects dealing with legacy systems and the results inform the ongoing development of a software tool that supports the analysis of trade-offs between different reconstructed architectures.

Context: Contemporary software development is typically conducted in dynamic, resource-scarce environments that are prone to the accumulation of technical debt. While this general phenomenon is acknowledged, what remains unknown is how technical debt specifically manifests in and affects software processes, and how the software development techniques employed accommodate or mitigate the presence of this debt. Objectives: We sought to draw on practitioner insights and experiences in order to classify the effects of agile method use on technical debt management, given the popularity and perceived success of agile methods. We explore the breadth of practitioners’ knowledge about technical debt; how technical debt is manifested across the software process; and the perceived effects of common agile software development practices and processes on technical debt. In doing so, we address a research gap in technical debt knowledge and provide novel and actionable managerial recommendations. Method: We designed, tested and executed a multi-national survey questionnaire to address our objectives, receiving 184 responses from practitioners in Brazil, Finland, and New Zealand. Results: Our findings indicate that: 1) Practitioners are aware of technical debt, although, there was under utilization of the concept, 2) Technical debt commonly resides in legacy systems, however, concrete instances of technical debt are hard to conceptualize which makes it problematic to manage, 3) Queried agile practices and processes help to reduce technical debt; in particular, techniques that verify and maintain the structure and clarity of implemented artifacts (e.g., Coding standards and Refactoring) positively affect technical debt management. Conclusions: The fact that technical debt instances tend to have characteristics in common means that a systematic approach to its management is feasible. However, notwithstanding the positive effects of some agile practices on technical debt management, competing stakeholders’ interests remain a concern.

Context: In addressing how best to estimate how much effort is required to develop software, a recent study found that using exemplary and recently completed projects [forming Bellwether moving windows (BMW)] in software effort prediction (SEP) models leads to relatively improved accuracy. More studies need to be conducted to determine whether the BMW yields improved accuracy in general, since different sizing and aging parameters of the BMW are known to affect accuracy. Objective: To investigate the existence of exemplary projects (Bellwethers) with defined window size and age parameters, and whether their use in SEP improves prediction accuracy. Method: We empirically investigate the moving window assumption based on the theory that the prediction outcome of a future event depends on the outcomes of prior events. Sampling of Bellwethers was undertaken using three introduced Bellwether methods (SSPM, SysSam, and RandSam). The ergodic Markov chain was used to determine the stationarity of the Bellwethers. Results: Empirical results show that 1) Bellwethers exist in SEP and 2) the BMW has an approximate size of 50 to 80 exemplary projects that should not be more than 2 years old relative to the new projects to be estimated. Conclusion: The study's results add further weight to the recommended use of Bellwethers for improved prediction accuracy in SEP.

One of the purported ways to increase productivity and reduce development time is to reuse existing features and modules. If reuse is adopted, logically then, it will have a direct impact on a system's evolution. However, the evidence in the literature is not clear on the extent to which reuse is practiced in real-world projects, nor how it is practiced. In this paper we report the results of an investigation of reuse and evolution of software features in one of the largest open-source ecosystems - Eclipse. Eclipse provides a leading example of how a system can grow dramatically in size and number of features while maintaining its quality. Our results demonstrate the extent of feature reuse and evolution and also patterns of reuse across ten different Eclipse releases (from Europa to Neon).

Context: User involvement is generally considered to contributing to user satisfaction and project success and is central to Agile software development. In theory, the expectations about user involvement, such as the PO's, are quite demanding in this Agile way of working. But what are the expectations seen in practice, and are the expectations of user involvement aligned among the development team and users? Any misalignment could contribute to conflict and miscommunication among stakeholders that may result in ineffective user involvement. Objective: Our aim is to compare and contrast the expectations of two stakeholder groups (software development team, and software users) about user involvement in order to understand the expectations and assess their alignment. Method: We have conducted an exploratory case study of expectations about user involvement in an Agile software development. Qualitative data was collected through interviews to design a novel method for the assessing the alignment of expectations about user involvement by applying Repertory Grids (RG). Results: By aggregating the results from the interviews and RGs, varying degrees of expectation alignments were observed between the development team and user representatives. Conclusion: Alignment of expectations can be assessed in practice using the proposed RG instrument and can reveal misalignment between user roles and activities they participate in Agile software development projects. Although we used RG instrument retrospectively in this study, we posit that it could also be applied from the start of a project, or proactively as a diagnostic tool throughout a project to assess and ensure that expectations are aligned.

The DevOps phenomenon is gaining popularity through its ability to support continuous value delivery and ready accommodation of change. However, given the relative immaturity and general confusion about DevOps, a common view of expectations from a DevOps role is lacking. Through investigation of online job advertisements, combined with interviews, we identified key Knowledge Areas, Skills and Capabilities for a DevOps role and their relative importance in New Zealand's job market. Our analysis also revealed the global dimensions and the emerging nature of the DevOps role in GSE projects. This research adds a small advanced economy (New Zealand) perspective to the literature on DevOps job advertisements and should be of value to employers, job seekers, researchers as well educators and policy makers.

Outsourcing is typically considered to occur in three phases: decision, transition and operation. As outsourcing is now well established the switching of vendors and transitioning from one system to another is common. However, most of the research to date on outsourcing has focused on the decision and operation phases, leaving a gap between theory and practice concerning the transition phase. Transition in outsourcing entails the changing of systems, business processes and/or vendors. If a suitable transition approach is not applied pressures for another transition can immediately build. This paper presents results from a case study carried out on the 'Novopay Project' in which the Ministry of Education in New Zealand changed their vendor from an onshore to a near-shore provider. This project resulted in a sequence of three transitions, with each following a different approach as a direct result of the experiences encountered in the previous transition. In this research we made use of the rich 'data dump' of evidence provided by the Ministry of Education (MoE). Our analysis describes how a client organization can become trapped in a continuous transition cycle if a suitable approach is not applied. Transition1 involved the client - MoE - moving from complete outsourcing to selective insourcing. After realizing that they did not have the capabilities to manage insourcing, Transition2 was initiated. In Transition2 the sourcing approach reverted back to complete outsourcing. When it was realized that the new vendor in Transition2 could not in fact deliver a new service model or support end-users in following new business processes, Transition3 was initiated. In Transition3, the client established an internal company to insource service operations to support end-users. Transition can be a sound business strategy initiated for a range of reasons. However, if a flawed sourcing approach is chosen it can result in 'continuous transition'.

Bellwether effect refers to the existence of exemplary projects (called the Bellwether) within a historical dataset to be used for improved prediction performance. Recent studies have shown an implicit assumption of using recently completed projects (referred to as moving window) for improved prediction accuracy. In this paper, we investigate the Bellwether effect on software effort estimation accuracy using moving windows. The existence of the Bellwether was empirically proven based on six postulations. We apply statistical stratification and Markov chain methodology to select the Bellwether moving window. The resulting Bellwether moving window is used to predict the software effort of a new project. Empirical results show that Bellwether effect exist in chronological datasets with a set of exemplary and recently completed projects representing the Bellwether moving window. Result from this study has shown that the use of Bellwether moving window with the Gaussian weighting function significantly improve the prediction accuracy.

In seeking to complement consultants' and tool vendors' reports, there has been an increasing academic focus on understanding the adoption and use of software development methods and practices. We surveyed practitioners working in Brazil, Finland, and New Zealand in a transnational study to contribute to these efforts. Among our findings we observed that most of the 184 practitioners in our sample focused on a small portfolio of projects that were of short duration. In addition, Scrum and Kanban were used most; however, some practitioners also used conventional methods. Coding Standards, Simple Design and Refactoring were used most by practitioners, and these practices were held to be largely suitable for project and process management. Our evidence points to the need to properly understand and support a wide range of software methods.

While a substantial body of prior research has investigated the form and nature of production code, comparatively little attention has examined characteristics of test code, and, in particular, test smells in that code. In this paper, we explore the relationship between production code properties (at the class level) and a set of test smells, in five open source systems. Specifically, we examine whether complexity properties of a production class can be used as predictors of the presence of test smells in the associated unit test. Our results, derived from the analysis of 975 production class-unit test pairs, show that the Cyclomatic Complexity (CC) and Weighted Methods per Class (WMC) of production classes are strong indicators of the presence of smells in their associated unit tests. The Lack of Cohesion of Methods in a production class (LCOM) also appears to be a good indicator of the presence of test smells. Perhaps more importantly, all three metrics appear to be good indicators of particular test smells, especially Eager Test and Duplicated Code. The Depth of the Inheritance Tree (DIT), on the other hand, was not found to be significantly related to the incidence of test smells. The results have important implications for large-scale software development, particularly in a context where organizations are increasingly using, adopting or adapting open source code as part of their development strategy and need to ensure that classes and methods are kept as simple as possible.

Software has always been considered as malleable. Changes to software requirements are inevitable during the development process. Despite many software engineering advances over several decades, requirements changes are a source of project risk, particularly when businesses and technologies are evolving rapidly. Although effectively managing requirements changes is a critical aspect of software engineering, conceptions of requirements change in the literature and approaches to their management in practice still seem rudimentary. The overall goal of this study is to better understand the process of requirements change management. We present findings from an exploratory case study of requirements change management in a globally distributed setting. In this context we noted a contrast with the traditional models of requirements change. In theory, change control policies and formal processes are considered as a natural strategy to deal with requirements changes. Yet we observed that "informal requirements changes" (InfRc) were pervasive and unavoidable. Our results reveal an equally 'natural' informal change management process that is required to handle InfRc in parallel. We present a novel model of requirements change which, we argue, better represents the phenomenon and more realistically incorporates both the informal and formal types of change.

Context: Post-release user feedback plays an integral role in improving software quality and informing new features. Given its growing importance, feedback concerning security enhancements is particularly noteworthy. In considering the rapid uptake of Android we have examined the scale and severity of Android security threats as reported by its stakeholders. Objective: We systematically mine Android issue logs to derive insights into stakeholder perceptions and experiences in relation to certain Android security issues. Method: We employed contextual analysis techniques to study issues raised regarding confidentiality and privacy in the last three major Android releases, considering covariance of stakeholder comments, and the level of consistency in user preferences and priorities. Results: Confidentiality and privacy concerns varied in severity, and were most prevalent over Jelly Bean releases. Issues raised in regard to confidentiality related mostly to access, user credentials and permission management, while privacy concerns were mainly expressed about phone locking. Community users also expressed divergent preferences for new security features, ranging from more relaxed to very strict. Conclusions: Strategies that support continuous corrective measures for both old and new Android releases would likely maintain stakeholder confidence. An approach that provides users with basic default security settings, but with the power to configure additional security features if desired, would provide the best balance for Android's wide cohort of stakeholders.

Architectural challenges in migrating plan-driven projects to agile

Software effort estimation (SEE) is a core activity in all software processes and development lifecycles. A range of increasingly complex methods has been considered in the past 30 years for the prediction of effort, often with mixed and contradictory results. The comparative assessment of effort prediction methods has therefore become a common approach when considering how best to predict effort over a range of project types. Unfortunately, these assessments use a variety of sampling methods and error measurements, making comparison with other work difficult. This article proposes an automatically transformed linear model (ATLM) as a suitable baseline model for comparison against SEE methods. ATLM is simple yet performs well over a range of different project types. In addition, ATLM may be used with mixed numeric and categorical data and requires no parameter tuning. It is also deterministic, meaning that results obtained are amenable to replication. These and other arguments for using ATLM as a baseline model are presented, and a reference implementation described and made available. We suggest that ATLM should be used as a baseline of effort prediction quality for all future model comparisons in SEE.

Context: In order to preserve the value of Systematic Reviews (SRs), they should be frequently updated considering new evidence that has been produced since the completion of the previous version of the reviews. However, the update of an SR is a time consuming, manual task. Thus, many SRs have not been updated as they should be and, therefore, they are currently outdated. Objective: The main contribution of this paper is to support the update of SRs. Method: We propose USR-VTM, an approach based on Visual Text Mining (VTM) techniques, to support selection of new evidence in the form of primary studies. We then present a tool, named Revis, which supports our approach. Finally, we evaluate our approach through a comparison of outcomes achieved using USR-VTM versus the traditional (manual) approach. Results: Our results show that USR-VTM increases the number of studies correctly included compared to the traditional approach. Conclusions: USR-VTM effectively supports the update of SRs.

A growing number of empirical software engineering researchers suggest that a complementary focus on theory is required if the discipline is to mature. A first step in theory-building involves the establishment of suitable theoretical constructs. For researchers studying software projects, the lack of a theoretical construct for context is problematic for both experimentation and effort estimation. For experiments, insufficiently understood contextual factors confound results, and for estimation, unstated contextual factors affect estimation reliability. We have earlier proposed a framework that we suggest may be suitable as a construct for context i.e. represents a minimal, spanning set for the space of software contexts. The framework has six dimensions, described as Who, Where, What, When, How and Why. In this paper, we report the outcomes of a pilot study to test its suitability by categorising contextual factors from the software engineering literature into the framework. We found that one of the dimensions, Why, does not represent context, but rather is associated with objectives. We also identified some factors that do not clearly fit into the framework and require further investigation. Our contributions are the pursuing of a theoretical approach to understanding software context, the initial establishment and evaluation of a construct for context and the exposure of a lack of clarity of meaning in many 'contexts' currently applied as factors for estimating project outcomes.

Context: Individuals' personality traits have been shown to influence their behavior during team work. In particular, positive group attitudes are said to be essential for distributed and global software development efforts where collaboration is critical to project success. Objective: Given this, we have sought to study the influence of global software practitioners' personality profiles from a psycholinguistic perspective. Method: Artifacts from ten teams were selected from the IBM Rational Jazz repository and mined. We employed social network analysis (SNA) techniques to identify and group practitioners into two clusters based on the numbers of messages they communicated, Top Members and Others, and used standard statistical techniques to assess practitioners' engagement in task changes associated with work items. We then performed psycholinguistic analysis on practitioners' messages using linguistic dimensions of the LIWC tool that had been previously correlated with the Big Five personality profiles. Results: For our sample of 146 practitioners, we found that the Top Members demonstrated more openness to experience than the Other practitioners. Additionally, practitioners involved in usability-related tasks were found to be highly extroverted, and coders were most neurotic and conscientious. Conclusion: High levels of organizational and inter-personal skills may be useful for those operating in distributed settings, and personality diversity is likely to boost team performance.

It is generally acknowledged that software testing is both challenging and time-consuming. Understanding the factors that may positively or negatively affect testing effort will point to possibilities for reducing this effort. Consequently there is a significant body of research that has investigated relationships between static code properties and testability. The work reported in this paper complements this body of research by providing an empirical evaluation of the degree of association between runtime properties and class-level testability in object-oriented (OO) systems. The motivation for the use of dynamic code properties comes from the success of such metrics in providing a more complete insight into the multiple dimensions of software quality. In particular, we investigate the potential relationships between the runtime characteristics of production code, represented by Dynamic Coupling and Key Classes, and internal class-level testability. Testability of a class is considered here at the level of unit tests and two different measures are used to characterise those unit tests. The selected measures relate to test scope and structure: one is intended to measure the unit test size, represented by test lines of code, and the other is designed to reflect the intended design, represented by the number of test cases. In this research we found that Dynamic Coupling and Key Classes have significant correlations with class-level testability measures. We therefore suggest that these properties could be used as indicators of class-level testability. These results enhance our current knowledge and should help researchers in the area to build on previous results regarding factors believed to be related to testability and testing. Our results should also benefit practitioners in future class testability planning and maintenance activities.

Small to medium sized business enterprises (SMEs) generally thrive because they have successfully done something unique within a niche market. For this reason, SMEs may seek to protect their competitive advantage by avoiding any standardization encouraged by the use of packaged software (PS). Packaged software implementation at SMEs therefore presents challenges relating to how best to respond to mismatches between the functionality offered by the packaged software and each SME's business needs. An important question relates to which processes small software enterprises - or Small to Medium-Sized Software Development Companies (SMSSDCs) - apply in order to identify and then deal with these mismatches. To explore the processes of packaged software (PS) implementation, an ethnographic study was conducted to gain in-depth insights into the roles played by analysts in two SMSSDCs. The purpose of the study was to understand PS implementation in terms of requirements engineering (or 'PSIRE'). Data collected during the ethnographic study were analyzed using an inductive approach. Based on our analysis of the cases we constructed a theoretical model explaining the requirements engineering process for PS implementation, and named it the PSIRE Parallel Star Model. The Parallel Star Model shows that during PSIRE, more than one RE process can be carried out at the same time. The Parallel Star Model has few constraints, because not only can processes be carried out in parallel, but they do not always have to be followed in a particular order. This paper therefore offers a novel investigation and explanation of RE practices for packaged software implementation, approaching the phenomenon from the viewpoint of the analysts, and offers the first extensive study of packaged software implementation RE (PSIRE) in SMSSDCs.

Context: The utility of prediction models in empirical software engineering (ESE) is heavily reliant on the quality of the data used in building those models. Several data quality challenges such as noise, incompleteness, outliers and duplicate data points may be relevant in this regard. Objective: We investigate the reporting of three potentially influential elements of data quality in ESE studies: data collection, data pre-processing, and the identification of data quality issues. This enables us to establish how researchers view the topic of data quality and the mechanisms that are being used to address it. Greater awareness of data quality should inform both the sound conduct of ESE research and the robust practice of ESE data collection and processing. Method: We performed a targeted literature review of empirical software engineering studies covering the period January 2007 to September 2012. A total of 221 relevant studies met our inclusion criteria and were characterized in terms of their consideration and treatment of data quality. Results: We obtained useful insights as to how the ESE community considers these three elements of data quality. Only 23 of these 221 studies reported on all three elements of data quality considered in this paper. Conclusion: The reporting of data collection procedures is not documented consistently in ESE studies. It will be useful if data collection challenges are reported in order to improve our understanding of why there are problems with software engineering data sets and the models developed from them. More generally, data quality should be given far greater attention by the community. The improvement of data sets through enhanced data collection, pre-processing and quality assessment should lead to more reliable prediction models, thus improving the practice of software engineering.

Context: Interest in software engineering (SE) methodologies and tools has been complemented in recent years by research efforts oriented towards understanding the human processes involved in software development. This shift has been imperative given reports of inadequately performing teams and the consequent growing emphasis on individuals and team relations in contemporary SE methods. Objective: While software repositories have frequently been studied with a view to explaining such human processes, research has tended to use primarily quantitative analysis approaches. There is concern, however, that such approaches can provide only a partial picture of the software process. Given the way human behavior is nuanced within psychological and social contexts, it has been asserted that a full understanding may only be achieved through deeper contextual enquiries. Method: We have followed such an approach and have applied data mining, SNA, psycholinguistic analysis and directed content analysis (CA) to study the way core developers at IBM Rational Jazz contribute their social and intellectual capital, and have compared the attitudes, interactions and activities of these members to those of their less active counterparts. Results: Among our results, we uncovered that Jazz's core developers worked across multiple roles, and were crucial to their teams' organizational, intra-personal and interpersonal processes. Additionally, although these individuals were highly task- and achievement-focused, they were also largely responsible for maintaining positive team atmosphere. Further, we uncovered that, as a group, Jazz developers spent a large amount of time providing context awareness in support of their colleagues. Conclusion: Our results suggest that high-performing distributed agile teams rely on both individual and collective efforts, as well as organizational environments that promote informal and organic work structures.

Context: Given the acknowledged need to understand the people processes enacted during software development, software repositories and mailing lists have become a focus for many studies. However, researchers have tended to use mostly mathematical and frequency-based techniques to examine the software artifacts contained within them. Objective: There is growing recognition that these approaches uncover only a partial picture of what happens during software projects, and deeper contextual approaches may provide further understanding of the intricate nature of software teams' dynamics. We demonstrate the relevance and utility of such approaches in this study. Method: We use psycholinguistics and directed content analysis (CA) to study the way project tasks drive teams' attitudes and knowledge sharing. We compare the outcomes of these two approaches and offer methodological advice for researchers using similar forms of repository data. Results: Our analysis reveals significant differences in the way teams work given their portfolio of tasks and the distribution of roles. Conclusion: We overcome the limitations associated with employing purely quantitative approaches, while avoiding the time-intensive and potentially invasive nature of field work required in full case studies.

It is widely acknowledged by researchers and practitioners that software development methodologies are generally adapted to suit specific project contexts.Research into practices-as-implemented has been fragmented and has tended to focus either on the strength of adherence to a specific methodology or on how the efficacy of specific practices is affected by contextual factors.We submit the need for a more holistic, integrated approach to investigating context-related best practice.We propose a six-dimensional model of the problem-space, with dimensions organisational drivers (why), space and time (where), culture (who), product life-cycle stage (when), product constraints (what) and engagement constraints (how).We test our model by using it to describe and explain a reported implementation study.Our contributions are a novel approach to understanding situated software practices and a preliminary model for software contexts.

Agile software developers are required to self-organize, occupying various informal roles as needed in order to successfully deliver software features.However, previous research has reported conflicting evidence about the way teams actually undertake this activity.The ability to self-organize is particularly necessary for software development in globally distributed environments, where distance has been shown to exacerbate human-centric issues.Understanding the way successful teams self-organise should inform distributed team composition strategies and software project governance.We have used psycholinguistics to study the way IBM Rational Jazz practitioners enacted various roles, expressed attitudes and shared competencies to successfully self-organize in their global projects.Among our findings, we uncovered that practitioners enacted various roles depending on their teams' cohort of features; and that team leaders were most critical to IBM Jazz teams' self-organisation.We discuss these findings and highlight their implications for software project governance.

Global Software Engineering (GSE) research contains few examples consciously applying what Glass and colleagues have termed an 'evaluative-critical' approach. In this study we apply dilemma analysis to conduct a critical review of a major (and ongoing) near shore Business Process Outsourcing project in New Zealand. The project has become so troubled that a Government Minister has recently been assigned responsibility for troubleshooting it. The 'Novo pay' project concerns the implementation of a nationwide payroll system responsible for the payment of some 110,000 teachers and education sector staff. An Australian company won the contract for customizing and implementing the Novo pay system, taking over from an existing New Zealand service provider. We demonstrate how a modified form of dilemma analysis can be a powerful technique for highlighting risks and stakeholder impacts from empirical data, and that adopting an evaluative-critical approach to such projects can usefully highlight tensions and barriers to satisfactory project outcomes.

Global software development practices are shaped by the challenges of time and 'distance', notions perceived to separate sites in a multi-site collaboration. Yet while sites may be fixed, the actors in global projects are mobile, so distance becomes a dynamic spatial dimension rather than a static concept. This empirical study applies grounded theory to unpack the nature of mobility within a three site globally distributed team setting. We develop a model for mapping the movements of team members in local and global spaces, and demonstrate its operation through static snapshots and dynamic patterns evolving over time. Through this study we highlight the complexity of 'mobility' as one facet of 'space' in globally distributed teams and illuminate its tight coupling with the accompanying dimensions of accessibility and context awareness.

Several important aspects of software product quality can be evaluated using dynamic metrics that effectively capture and reflect the software's true runtime behavior. While the extent of research in this field is still relatively limited, particularly when compared to research on static metrics, the field is growing, given the inherent advantages of dynamic metrics. The aim of this work is to systematically investigate the body of research on dynamic software metrics to identify issues associated with their selection, design and implementation. Mapping studies are being increasingly used in software engineering to characterize an emerging body of research and to identify gaps in the field under investigation. In this study we identified and evaluated 60 works based on a set of defined selection criteria. These studies were further classified and analyzed to identify their relativity to future dynamic metrics research. The classification was based on three different facets: research focus, research type and contribution type. We found a strong body of research related to dynamic coupling and cohesion metrics, with most works also addressing the abstract notion of software complexity. Specific opportunities for future work relate to a much broader range of quality dimensions.

This report describes the experiences of one organization's adoption of Test Driven Development (TDD) practices as part of a medium-term software project employing Extreme Programming as a methodology. Three years into this project the team's TDD experiences are compared with their non-TDD experiences on other ongoing projects. The perceptions of the benefits and challenges of using TDD in this context are gathered through five semi-structured interviews with key team members. Their experiences indicate that use of TDD has generally been positive and the reasons for this are explored to deepen the understanding of TDD practice and its effects on code quality, application quality and development productivity. Lessons learned are identified to aid others with the adoption and implementation of TDD practices, and some potential further research areas are suggested.

Background: A systematic literature review (SLR) is a methodology used to aggregate all relevant existing evidence to answer a research question of interest. Although crucial, the process used to select primary studies can be arduous, time consuming, and must often be conducted manually. Objective: We propose a novel approach, known as 'Systematic Literature Review based on Visual Text Mining' or simply SLR-VTM, to support the primary study selection activity using visual text mining (VTM) techniques. Method: We conducted a case study to compare the performance and effectiveness of four doctoral students in selecting primary studies manually and using the SLR-VTM approach. To enable the comparison, we also developed a VTM tool that implemented our approach. We hypothesized that students using SLR-VTM would present improved selection performance and effectiveness. Results: Our results show that incorporating VTM in the SLR study selection activity reduced the time spent in this activity and also increased the number of studies correctly included. Conclusions: Our pilot case study presents promising results suggesting that the use of VTM may indeed be beneficial during the study selection activity when performing an SLR.

BACKGROUND: In reality project managers are constrained by the incremental nature of data collection. Specifically, project observations are accumulated one project at a time. Likewise within-project data are accumulated one stage or phase at a time. However, empirical researchers have given limited attention to this perspective.

Background: In spite of their limited scope, measures reflecting adherence to schedule, budget and specification continue to dominate the assessment and reporting of project outcomes.

The prevailing notion of time which pervades reports on global software development practice is the linear notion of time as a scarce commodity to be optimized through working across global boundaries. This `temponomic' view of time provides a useful but limited model through which to understand how time operates in practice within globally distributed teams. We report findings from an in depth empirical study which employed a grounded analysis of the many dimensions of time in action within a global team setting. A situated analysis of the actions at each of three globally distributed sites, demonstrates how the differing aspects of time interact, and how some of the known challenges in working globally, can be viewed from a temporal viewpoint. We argue that this more nuanced understanding of how time functions in globally distributed teams may help managers and researchers develop more appropriate practices and models for managing such teams.

BACKGROUND-The systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE-The aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular, we wish to investigate the consistency of process and the stability of outcomes. METHOD-We compare the results of two independent reviews undertaken with a common research question. RESULTS-The two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS-In addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method.

The collaborative development of shared understanding is crucial to the success of software development projects. It is also a challenging and volatile process in practice. Small organizations may be especially vulnerable due to reliance on key individuals and insufficient resource to employ several domain specialists. There is, however, minimal empirical research on sharing domain understanding in the context of small software organizations. In this paper we present the results of a field study of commercial software development practice in which we conducted semi-structured interviews with practitioners from ten such organizations. The study provides insights into practices, perceptions, and challenges related to developing shared domain understanding. Our results show that smaller organizations place particular emphasis on the use of prototypes or existing products to refine and verify domain understanding. Furthermore they perceive the biggest challenge to developing shared understanding as the quality of the client representative(s).

Software effort prediction clearly plays a crucial role in software project management. In keeping with more dynamic approaches to software development, it is not sufficient to only predict the whole-project effort at an early stage. Rather, the project manager must also dynamically predict the effort of different stages or activities during the software development process. This can assist the project manager to reestimate effort and adjust the project plan, thus avoiding effort or schedule overruns. This paper presents a method for software physical time stage-effort prediction based on grey models GM(1,1) and Verhulst. This method establishes models dynamically according to particular types of stage-effort sequences, and can adapt to particular development methodologies automatically by using a novel grey feedback mechanism. We evaluate the proposed method with a large-scale real-world software engineering dataset, and compare it with the linear regression method and the Kalman filter method, revealing that accuracy has been improved by at least 28% and 50%, respectively. The results indicate that the method can be effective and has considerable potential. We believe that stage predictions could be a useful complement to whole-project effort prediction methods.

The availability of multi-organisation data sets has made it possible for individual organisations to build and apply management models, even if they do not have data of their own. In the absence of any data this may be a sensible option, driven by necessity. However, if both cross-company (or global) and within-company (or local) data are available, which should be used in preference? Several research papers have addressed this question but without any apparent convergence of results. We conduct a systematic review of empirical studies comparing global and local effort prediction systems. We located 10 relevant studies: 3 supported global models, 2 were equivocal and 5 supported local models. The studies do not have converging results. A contributing factor is that they have utilised different local and global data sets and different experimental designs thus there is substantial heterogeneity. We identify the need for common response variables and for common experimental and reporting protocols.

There is a growing body of evidence to suggest that significant benefits may be gained from augmenting current approaches to software development effort estimation, and indeed other project management activities, with models developed using fuzzy logic and other soft computing methods. The tasks undertaken by project managers early in a development process would appear to be particularly amenable to such a strategy, particularly if fuzzy logic models are used in a complementary manner with other algorithmic approaches, thus providing a range of predictions as opposed to a single point value. As well as providing a more intuitively acceptable set of estimates, this would help to reduce or remove the unwarranted level of certainty associated with a point estimate. Furthermore, such an approach would enable organizations to "store" their project management knowledge, making them less susceptible to employee resignations and the like. If fuzzy logic modeling is to be implemented in industry, however, managers must first believe it to be a realistic and workable option. This issue is addressed here by considering two related questions: one, what expectations do project managers have in relation to effort estimation? And two, what is their opinion of the methods that might be useful in this regard? This is followed by a discussion of the results of two surveys of project managers aimed at deriving membership functions using polling methods, the first using an interval declaration approach and the second using votes on fixed points. It is concluded that there is indeed support in the software engineering practitioner community for the use of methods based on the principles of fuzzy logic modeling.

This paper describes ongoing research directed at formulating a set of appropriate measures for assessing and ultimately predicting effort requirements for multimedia systems development. Whilst significant advances have been made in the determination of measures for both transaction-based and process-intensive systems, very little work has been undertaken in relation to measures for multimedia systems. A small preliminary empirical study is reviewed as a precursor to a more exploratory investigation of the factors that are considered by industry to be influential in determining development effort. This work incorporates the development and use of a goal-based framework to assist the measure selection process from a literature basis, followed by an industry questionnaire. The results provide a number of preliminary but nevertheless useful insights into contemporary project management practices with respect to multimedia systems.