Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. • International survey gathering insights from 188 practitioners across 25 countries. • Overview of current practices and challenges in engineering ML-enabled systems. • Inferential quantitative analysis reporting the status quo with confidence intervals. • Qualitative analysis mapping ML life cycle challenges and causes of project failure.

Automated software defect prediction (SDP) methods are increasingly applied, often with the use of machine learning (ML) techniques. Yet, the existing ML-based approaches require manually extracted features, which are cumbersome, time consuming and hardly capture the semantic information reported in bug reporting tools. Deep learning (DL) techniques provide practitioners with the opportunities to automatically extract and learn from more complex and high-dimensional data. The purpose of this study is to systematically identify, analyze, summarize, and synthesize the current state of the utilization of DL algorithms for SDP in the literature. We systematically selected a pool of 102 peer-reviewed studies and then conducted a quantitative and qualitative analysis using the data extracted from these studies. Main highlights include: (1) most studies applied supervised DL; (2) two third of the studies used metrics as an input to DL algorithms; (3) Convolutional Neural Network is the most frequently used DL algorithm. Based on our findings, we propose to (1) develop more comprehensive DL approaches that automatically capture the needed features; (2) use diverse software artifacts other than source code; (3) adopt data augmentation techniques to tackle the class imbalance problem; (4) publish replication packages.

Abstract Just-in-time defect prediction (JITDP) research is increasingly focused on program changes instead of complete program modules within the context of continuous integration and continuous testing paradigm. Traditional machine learning-based defect prediction models have been built since the early 2000s, and recently, deep learning-based models have been designed and implemented. While deep learning (DL) algorithms can provide state-of-the-art performance in many application domains, they should be carefully selected and designed for a software engineering problem. In this research, we evaluate the performance of traditional machine learning algorithms and data sampling techniques for JITDP problems and compare the model performance with the performance of a DL-based prediction model. Experimental results demonstrated that DL algorithms leveraging sampling methods perform significantly worse than the decision tree-based ensemble method. The XGBoost-based model appears to be 116 times faster than the multilayer perceptron-based (MLP) prediction model. This study indicates that DL-based models are not always the optimal solution for software defect prediction, and thus, shallow, traditional machine learning can be preferred because of better performance in terms of accuracy and time parameters.

Abstract This paper proposes a novel method for assessing text coherence. Central to this approach is an ontology‐based representation of text, which captures the level of relatedness between consecutive sentences via ontologies. Our method encompasses annotating text using ontological concepts and assessing text coherence based on relatedness measurement among these concepts. The ontology‐based relatedness measurement method used in this study considers various types of relationships in ontologies and derived relationships via an inference engine for computing relatedness. We hypothesized that rich variety of relationships and inferred facts in ontologies would improve the success of text coherence assessment. Our results demonstrate that the use of ontologies yields to coherence values that have a higher correlation with human ratings.

Context:Knowledge level and productivity of the software engineering (SE) workforce are the subject of regular discussions among practitioners, educators, and researchers. There have been many efforts to measure and improve the knowledge gap between SE education and industrial needs. Objective:Although the existing efforts for aligning SE education and industrial needs have provided valuable insights, there is a need for analyzing the SE topics in a more “fine-grained” manner; i.e., knowing that SE university graduates should know more about requirements engineering is important, but it is more valuable to know the exact topics of requirements engineering that are most important in the industry. Method:We achieve the above objective by assessing the knowledge gaps of software engineers by designing and executing an opinion survey on levels of knowledge learned in universities versus skills needed in industry. We designed the survey by using the SE knowledge areas (KAs) from the latest version of the Software Engineering Body of Knowledge (SWEBOK v3), which classifies the SE knowledge into 12 KAs, which are themselves broken down into 67 subareas (sub-KAs) in total. Our analysis is based on (opinion) data gathered from 129 practitioners, who are mostly based in Turkey. Results:Based on our findings, we recommend that educators should include more materials on software maintenance, software configuration management, and testing in their SE curriculum. Based on the literature as well as the current trends in industry, we provide actionable suggestions to improve SE curriculum to decrease the knowledge gap.

Large-scale software development organizations generally carry out multiple software development projects simultaneously. Teams use various software development processes and tools to implement these projects. In this context, the main challenges of the practitioners are (1) keeping track of the status of a single project where hybrid set of tools exist for different software life cycle activities (2) effectively monitoring a consolidated status of multiple projects that use hybrid processes and tools. To address these challenges, it is vital to have a unified view of these projects independent from these hybrid processes and tools. To this end, we report on our preliminary experiences on the development of a unified project monitoring solution and a corresponding tool support based on the Essence framework's language and kernel. Our solution provides an up-to-date and unified view of projects by collecting data from various tools automatically as well as allowing manual data entry.

Software process assessment (SPA) is the foundation step for software process improvement. ISO/IEC 15504 defines the term process assessment as "the systematic evaluation of an organization's processes against a process reference model (PRM)". In process assessment, there is a need to set and maintain a mapping between an organization's processes and a PRM, where process experts transform the gap between the two into opportunities for process improvement. To maintain such a mapping requires a continuous tracking and alignment between the organization's processes and the PRM(s). The use of ontologies might be a suitable solution to provide computerized tool support for SPA that becomes erroneous and time-consuming if done manually. With an aim to understand the use and usefulness of ontologies in SPA, in this study, we have performed a systematic literature review (SLR). We have searched the most known digital libraries and selected 14 studies out of 54 initially selected and 571 initially retrieved. We analyzed the selected studies with respect to a number of research questions that address; contribution facet, targeted software processes, research facet, process improvement model used, process assessment model used, ontology representation language, purpose of ontology use, qualitative and quantitative benefits reported, and challenges faced. As a result, we synthesized a conceptual model of ontology-based support in SPA. We hope the results of our work will be useful for researchers and practitioners to direct their future studies on the use of ontologies for SPA.