The pull-based development model allows collaborators to develop and propose changes to a codebase. However, pull requests can often offer duplicate functionality and therefore duplicate effort. Users can also request changes via issues, the text of which could provide clues, useful in determining duplicate pull requests. This research investigates combining pull requests with issues with a view to better detecting duplicate pull requests. The paper reviews existing related work and then extends this by investigating the use of natural language processing (NLP) on combined issues and pull requests in order to detect duplicates. Using data taken from 15 popular GitHub repositories, an NLP model was trained to predict duplicates by comparing the title and description of issues and pull requests. An evaluation of this model shows that duplicates can be detected with an accuracy of 93.9% and recall rate of 90.5%, while an exploratory study shows that the volume of duplicates detected can be increased dramatically by combining issues and pull requests into a single dataset. These results show a significant improvement on previous studies and demonstrate the value in detecting duplicates from issues and pull requests combined.

In the field of natural language processing (NLP), Large Language Models (LLMs) such as MISTRAL have been pre-trained on diverse tasks and datasets to achieve comprehensive language understanding. Nevertheless, fine-tuning small closed-domain datasets presents limitations, including overfitting, restricted generalization, data representation bias, and catastrophic forgetting issues. To tackle these challenges and align LLMs with ethical standards and specific requirements within closed-domain datasets, we apply a methodology that involves training four prominent LLMs - GEMMA, LLaMA, MISTRAL, and SOLAR - on a small closed-domain dataset using three distinct optimization techniques: Contrastive Preference Optimization (CPO), Direct Preference Optimization (DPO), and Kahneman-Tversky Optimization (KTO). We evaluate their proficiency by employing the Retrieval Augmented Generation (RAG) benchmark system to assess accuracy. Our research provides researchers with comprehensive insights into the effectiveness of these training methods in improving language model performance within closed-domain datasets while addressing the aforementioned challenges faced by LLMs in specific domains with limited datasets.

Context The COVID‐19 pandemic has had a disruptive impact on how people work and collaborate across all global economic sectors, including software business. While remote working is not new for software engineers, forced WFH situations come with both limitations and opportunities. As the ‘new normal’ for working might be based on the current state of Work‐from‐home (WFH), it is useful to understand what has happened and learn from that. Objective This study aims to gain insights into how their WFH arrangement impacts project management and software engineering. We are also interested in exploring these impacts in different contexts, such as startups and established companies. Method We conducted a global‐scale, cross‐sectional survey during the spring and summer 2021. Our results are based on quantitative and qualitative analysis of 297 valid responses. Results We characterize the profile of WFH in both spatial and temporal aspects, together with a set of common collaborative tools and coordination and control mechanisms. We revealed some areas of project management that are relatively more challenging during WFH situations, such as coordination, communication and project planning. We also revealed a mixed picture of the perceived impact of WFH on different software engineering activities. Conclusion WFH is a situational phenomenon which can have both negative and positive impact on software teams. For practitioners, we suggest a unified approach to consider the context of WFH, collaborative tools, associated coordination and control approaches and a process that resolve those aspects that are sensitive to physical interaction.

Context: Programming education keeps facing chal-lenges. A significant challenge is the mismatch between the increasing student demand and the shortage of teaching workforce on personal tutoring. Objective: The aim of this research is to address the demand-workforce mismatch and relieve the challenges of programming education. Method: After theoretically discussing a set of methodological principles (e.g., active learning from suitable code examples), a prototyping research method is employed to iteratively verify whether or not the proposed principles can be practically implemented. Results: A methodology, namely code example-aided just-in-time learning, gradually emerged. A code example hunting tool was developed for enabling this methodology. Conclusion: The code example-aided just-in-time learning can effectively and efficiently facilitate both experienced learners and beginners to study programming by practising, without requiring extensive guidance from teachers.

Code smells indicate weaknesses in software design that may slow down development or increase the risk of bugs or failures in the future. This paper aims to investigate the correlation of code smells with defects within classes. The method used uses a tool to automatically detect code smells in selected projects and then assesses the correlation of these to the number of defects found in the code. Most existing articles determine that software modules/classes with more smells tend to have more defects. However, while the experiments in this paper covering a range of languages agreed with this, the correlation was found to be weak. There remains a need for further investigation of the types of code smells that tend to indicate or predict defects occurring. Future work will perform more detailed experiments by investigating a larger quantity and variety of software systems as well as more granular studies into types of code smell and defects arising.

Takagi-Sugeno-Kang (TSK) rule-based fuzzy systems struggle to deal with high dimensional data and suffer from the curse of dimensionality. As the number of input features increases, the number of rules increases exponentially, which reduces the model's interpretability rapidly. This paper presents a novel fuzzy weighted forest aggregation method to effectively model high dimensional data by reducing the number of fuzzy rules, without sacrificing accuracy. The fuzzy weighted forest is comprised of several fuzzy weighted trees. Each tree is created based on a subset of features captured across different parts of the input space. Given n input features and N samples, every fuzzy tree is randomly assigned n′ features and N′ samples, where n′ and N′ are significantly smaller than n and N respectively. Each path within a tree, from root to leaf, forms a fuzzy rule. The non-leaf nodes represent the antecedents of rules, and the leaf node represents the consequents. This study shows how the proposed method utilizes pruning to significantly reduce the number of fuzzy rules. This method therefore creates a less complex model whilst achieving high accuracy comparable with, and sometimes better than, existing state-of-the-art TSK-based fuzzy models.

Fuzzy aggregation operators commonly rely on expert information to solve multi-attribute decision-making problems. Such expert input may contain human bias or may sometimes be unavailable. This paper proposes a novel data-driven fuzzy aggregation method for Takagi-Sugeno-Kang fuzzy neural networks (TSKFNN) based upon the ensemble learning algorithm, AdaBoost. The objective of this research is to investigate whether ensemble learning is an effective tool for data-driven fuzzy aggregation. Our hypothesis is that ensemble learning would improve model performance and explainability. In this study, AdaBoost is applied to get a weighted combination of fuzzy rules in the TSKFNN and calculate the weighted average of these fuzzy rules to generate model predictions. Existing fuzzy aggregation operators are used as benchmarks to evaluate the proposed model. The results show that the proposed model is capable of yielding higher accuracy and greater interpretability than the existing methods through the identification of the most significant fuzzy rules used in the decision-making process.

Requirements Engineering (RE) has established itself as a software engineering discipline over the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that was first piloted in Germany and, after making substantial modifications, has now been replicated in 10 countries worldwide. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from the replication studies conducted in 10 countries with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirical theory of RE practice. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents are common, that requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly internally driven. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.

Genetic algorithms have become popular in automating software refactoring and an increasing level of attention is being given to the use of multi-objective approaches. This paper investigated the use of a multi-objective genetic algorithm to automate software refactoring using a purpose built tool, MultiRefactor. The tool used a metric function to measure quality in a software system and tested a second objective to measure the importance of the classes being refactored. This priority objective takes as input a set of classes to favor and, optionally, a set of classes to disfavor as well. The multi-objective setup refactors the input program to improve its quality using the quality objective, while also focusing on the classes specified by the user. An experiment was constructed to measure the multi-objective approach against the alternative mono-objective approach that does not use an objective to measure priority of classes. The two approaches were tested on six different open source Java programs. The multi-objective approach was found to give significantly better priority scores across all inputs in a similar time, while also generating improvements in the quality scores.

This paper proposes a multi-objective genetic algorithm to automate software refactoring and validates the approach using a tool, MultiRefactor, and a set of open source Java programs. The tool uses a metric function to measure quality in a software system and tests a second objective to measure the recentness of the code elements being refactored. Previous versions of the software project are analyzed and a recentness measure is then calculated with respect to previous versions of code. The multi-objective setup refactors the input program to improve its quality using the quality objective, while also focusing on the recentness of the code elements inspected. An experiment has been constructed to measure the multi-objective approach against an alternative mono-objective approach that does not use an objective to measure element recentness. The two approaches are validated using six different open source Java programs. The multi-objective approach is found to give significantly better recentness scores across all inputs in a similar time, while also generating improvements in the quality score.

Much previous research has indicated that where a student sits in a university lecture theatre has a correlation with their final grade. Frequently those students that sit regularly in the front rows have been reported to achieve the highest grades. However most of the research restricted student seat movement, which is both unnatural and may have adversely influenced the research results. A previously reported unique unrestricted seat tracking investigation by the authors of this paper used a web and mobile software tracking application (PinPoint) to investigate student seating related performances in a 12 week Java programming university module. The PinPoint investigation concluded that the best assessment results were achieved by the students in the front rows and that assessment scores degraded the further students sat from the front. Additionally while the most engaged students were found to regularly sit at the front the same was not true for the most academically able or those with the greatest prior programming experience. This paper presents a further analysis of the PinPoint data, focusing on assessment performances within similar groups (academic ability, engagement and prior programming experiences) and additionally presents results of a temporal movement study and a qualitative analysis of the group and individual student seating decisions. It concludes that a comparison of student assessment performances within each of the peer groups, in every instance, found that the front row students outperformed their peers sitting further back. This strongly suggests that there is a benefit to sitting at the front regardless of academic ability, engagement or prior subject knowledge. It also points to other untested factors that may be positively influencing the front row performances.

We propose simple models to predict the performance degradation of disk requests due to storage device contention in consolidated virtualized environments. Model parameters can be deduced from measurements obtained inside Virtual Machines (VMs) from a system where a single VM accesses a remote storage server. The parameterized model can then be used to predict the effect of storage contention when multiple VMs are consolidated on the same server. We first propose a trace-driven approach that evaluates a queueing network with fair share scheduling using simulation. The model parameters consider Virtual Machine Monitor level disk access optimizations and rely on a calibration technique. We further present a measurement-based approach that allows a distinct characterization of read/write performance attributes. In particular, we define simple linear prediction models for I/O request mean response times, throughputs and read/write mixes, as well as a simulation model for predicting response time distributions. We found our models to be effective in predicting such quantities across a range of synthetic and emulated application workloads.

We propose a novel admission control policy for database queries. Our methodology uses system measurements of CPU utilization and query backlogs to determine interference between queries in execution on the same database server. Query interference may arise due to the concurrent access of hardware and software resources and can affect performance in positive and negative ways. Specifically our admission control considers the mix of jobs in service and prioritizes the query classes consuming CPU resources more efficiently. The policy ignores I/O subsystems and is therefore highly appropriate for in-memory databases. We validate our approach in trace-driven simulation and show performance increases of query slowdowns and throughputs compared to first-come first-served and shortest expected processing time first scheduling. Simulation experiments are parameterized from system traces of a SAP HANA in-memory database installation with TPC-H type workloads.

Abstract Changes to software requirements not only pose a risk to the successful delivery of software applications but also provide opportunity for improved usability and value. Increased understanding of the causes and consequences of change can support requirements management and also make progress towards the goal of change anticipation. This paper presents the results of two case studies that address objectives arising from that ultimate goal. The first case study evaluated the potential of a change source taxonomy containing the elements ‘market’, ‘organisation’, ‘vision’, ‘specification’, and ‘solution’ to provide a meaningful basis for change classification and measurement. The second case study investigated whether the requirements attributes of novelty, complexity, and dependency correlated with requirements volatility. While insufficiency of data in the first case study precluded an investigation of changes arising due to the change source of ‘market’, for the remainder of the change sources, results indicate a significant difference in cost, value to the customer and management considerations. Findings show that higher cost and value changes arose more often from ‘organisation’ and ‘vision’ sources; these changes also generally involved the co-operation of more stakeholder groups and were considered to be less controllable than changes arising from the ‘specification’ or ‘solution’ sources. Results from the second case study indicate that only ‘requirements dependency’ is consistently correlated with volatility and that changes coming from each change source affect different groups of requirements. We conclude that the taxonomy can provide a meaningful means of change classification, but that a single requirement attribute is insufficient for change prediction. A theoretical causal account of requirements change is drawn from the implications of the combined results of the two case studies.

Although a number of requirements change classifications have been proposed in the literature, there is no empirical assessment of their practical value in terms of their capacity to inform change monitoring and management. This paper describes an investigation of the informative efficacy of a taxonomy of requirements change sources which distinguishes between changes arising from `market', `organisation', `project vision', `specification' and `solution'. This investigation was effected through a case study where change data was recorded over a 16 month period covering the development lifecycle of a government sector software application. While insufficiency of data precluded an investigation of changes arising due to the change source of `market', for the remainder of the change sources, results indicate a significant difference in cost, value to the customer and management considerations. Findings show that higher cost and value changes arose more often from `organisation' and `vision' sources; these changes also generally involved the co-operation of more stakeholder groups and were considered to be less controllable than changes arising from the `specification' or `solution' sources. Overall, the results suggest that monitoring and measuring change using this classification is a practical means to support change management, understanding and risk visibility.

We propose a trace-driven approach to predict the performance degradation of disk request response times due to storage device contention in consolidated virtualized environments. Our performance model evaluates a queueing network with fair share scheduling using trace-driven simulation. The model parameters can be deduced from measurements obtained inside Virtual Machines (VMs) from a system where a single VM accesses a remote storage server. The parameterized model can then be used to predict the effect of storage contention when multiple VMs are consolidated on the same virtualized server. The model parameter estimation relies on a search technique that tries to estimate the splitting and merging of blocks at the the Virtual Machine Monitor (VMM) level in the case of multiple competing VMs. Simulation experiments based on traces of the Postmark and FFSB disk benchmarks show that our model is able to accurately predict the impact of workload consolidation on VM disk IO response times.

Abstract This focus section contains papers related to agile software development. The papers address a range of research areas including the application of agile methods to safety critical software development, the relationship of agile development with user experience design and how to measure flow in lean software development.

Various design patterns have become known for increasing the adaptability of software, so reducing the cost of maintenance and evolution. In this study the focus was placed on the performance cost of using such patterns with the research question 'How does the use of popular design patterns impact performance?' To answer this, an experiment was designed were a number of music players were developed using popular patterns, and the performance compared to a functionally identical player developed avoiding the patterns under investigation. The results provide evidence that in all cases the design patterns had an impact on performance. However, except in the case of severe hardware constraints, this impact is expected to be so minimal that any performance downside is outweighed by the advantages of using the pattern.

This paper reports results from a survey of experienced project managers on their perception of software risk management. From a sample of 18 experienced project managers, we have found good awareness of risk management, but low tool usage. We offer evidence that the main barriers to performing risk management are related to its perceived high cost and comparative low value. Psychological issues are also important, but less so. Risk identification and monitoring, in particular, are perceived as effort intensive and costly. The perception is that risk management is not prioritised highly enough. Our conclusion is that more must be done to visibly prove the value : cost ratio for risk management activities.

Requirements changes during software development pose a risk to cost, schedule and quality while at the same time providing an opportunity to add value. Provision of a generic change source taxonomy which makes the distinction between factors contributing to requirements uncertainty and events that trigger change will support requirements change risk visibility, and also facilitate richer recording of change data. In this paper we present a collaborative study to investigate and support the management of software requirements volatility within the development lifecycle. Previously published change `causes' are elicited from the literature, consolidated using expert knowledge and classified using card sorting. The resulting change trigger taxonomy constructs were initially validated using a small set of requirements change data, and deemed sufficient and practical as a means to collect common requirements change source statistics across multiple projects.

Model-Driven Architecture (MDA) supports the transformation from reusable models to executable software. Business representations, however, cannot be fully and explicitly represented in such models for direct transformation into running systems. Thus, once business needs change, the language abstractions used by MDA (e.g. object constraint language/action semantics), being low level, have to be edited directly. We therefore describe an agent-oriented MDA (AMDA) that uses a set of business models under continuous maintenance by business people, reflecting the current business needs and being associated with adaptive agents that interpret the captured knowledge to behave dynamically. Three contributions of the AMDA approach are identified: (1) to Agent-oriented Software Engineering, a method of building adaptive Multi-Agent Systems; (2) to MDA, a means of abstracting high-level business-oriented models to align executable systems with their requirements at runtime; (3) to distributed systems, the interoperability of disparate components and services via the agent abstraction.

This paper proposes an approach to the prioritisation of system changes that takes account of the relative costs and benefits of those changes and the risks that they reduce or introduce. This is part of the SERUM methodology (Software Engineering Risk: Understanding and Management), which is being developed to help identify effective ways of using risk analysis and control in software production. SERUM introduces risk management at the initial business analysis stage of requirements investigation, and assumes an evolutionary approach to software delivery. Prioritisation is determined from five factors: benefits, costs and risk exposure in the current system, target system, and development process. The relative importance of these factors is adjustable. Results from a case study at NEC illustrate the prioritisation process and a supporting software tool is also described.