[Context] Technical debt (TD) in machine learning (ML) systems, much like its counterpart in software engineering (SE), holds the potential to lead to future rework, posing risks to productivity, quality, and team morale. Despite growing attention to TD in SE, the understanding of ML-specific code-related TD remains underexplored. [Objective] This paper aims to identify and discuss the relevance of code-related issues that lead to TD in ML code throughout the ML workflow. [Method] The study first compiled a list of 34 potential issues contributing to TD in ML code by examining the phases of the ML workflow, their typical associated activities, and problem types. This list was refined through two focus group sessions involving nine experienced ML professionals, where each issue was assessed based on its occurrence contributing to TD in ML code and its relevance. [Results] The list of issues contributing to TD in the source code of ML systems was refined from 34 to 30, with 24 of these issues considered highly relevant. The data pre-processing phase was the most critical, with 14 issues considered highly relevant. Shortcuts in code related to typical pre-processing tasks (e.g., handling missing values, outliers, inconsistencies, scaling, rebalancing, and feature selection) often result in “patch fixes” rather than sustainable solutions, leading to the accumulation of TD and increasing maintenance costs. Relevant issues were also found in the data collection, model creation and training, and model evaluation phases. [Conclusion] We have made the final list of issues available to the community and believe it will help raise awareness about issues that need to be addressed throughout the ML workflow to reduce TD and improve the maintainability of ML code.

Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. • International survey gathering insights from 188 practitioners across 25 countries. • Overview of current practices and challenges in engineering ML-enabled systems. • Inferential quantitative analysis reporting the status quo with confidence intervals. • Qualitative analysis mapping ML life cycle challenges and causes of project failure.

[Context] Applying design principles has long been acknowledged as beneficial for understanding and maintainability in traditional software projects. These benefits may similarly hold for Machine Learning (ML) projects, which involve iterative experimentation with data, models, and algorithms. However, ML components are often developed by data scientists with diverse educational backgrounds, potentially resulting in code that doesn't adhere to software design best practices. [Goal] In order to better understand this phenomenon, we investigated the impact of the SOLID design principles on ML code understanding. [Method] We conducted a controlled experiment with three independent trials involving 100 data scientists. We restructured real industrial ML code that did not use SOLID principles. Within each trial, one group was presented with the original ML code, while the other was presented with ML code incorporating SOLID principles. Participants of both groups were asked to analyze the code and fill out a questionnaire that included both open-ended and closed-ended questions on their understanding. [Results] The study results provide statistically significant evidence that the adoption of the SOLID design principles can improve code understanding within the realm of ML projects. [Conclusion] We put forward that software engineering design principles should be spread within the data science community and considered for enhancing the maintainability of ML code.

Machine learning (ML) has become a core feature for today’s real-world applications, making it a trending topic for the software engineering community. Requirements Engineering (RE) is no stranger to this and its main conferences have included workshops aiming at discussing RE in the context of ML. However, current research on the intersection between RE and ML mainly focuses on using ML techniques to support RE activities rather than on exploring how RE can improve the development of ML-based systems. This paper concerns a systematic mapping study aiming at characterizing the publication landscape of RE for ML-based systems, outlining research contributions and contemporary gaps for future research. In total, we identified 35 studies that met our inclusion criteria. We found several different types of contributions, in the form of analyses, approaches, checklists and guidelines, quality models, and taxonomies. We discuss gaps by mapping these contributions against the RE topics to which they were contributing and their type of empirical evaluation. We also identified quality characteristics that are particularly relevant for the ML context (e.g., data quality, explainability, fairness, safety, and transparency). Main reported challenges are related to the lack of validated RE techniques, the fragmented and incomplete understanding of NFRs for ML, and difficulties in handling customer expectations. There is a need for future research on the topic to reveal best practices and to propose and investigate approaches that are suitable to be used in practice.

Abstract This work presents a new neuro-evolutionary model, called NEVE (Neuroevolutionary Ensemble), based on an ensemble of Multi-Layer Perceptron (MLP) neural networks for learning in nonstationary environments. NEVE makes use of quantum-inspired evolutionary models to automatically configure the ensemble members and combine their output. The quantum-inspired evolutionary models identify the most appropriate topology for each MLP network, select the most relevant input variables, determine the neural network weights and calculate the voting weight of each ensemble member. Four different approaches of NEVE are developed, varying the mechanism for detecting and treating concepts drifts, including proactive drift detection approaches. The proposed models were evaluated in real and artificial datasets, comparing the results obtained with other consolidated models in the literature. The results show that the accuracy of NEVE is higher in most cases and the best configurations are obtained using some mechanism for drift detection. These results reinforce that the neuroevolutionary ensemble approach is a robust choice for situations in which the datasets are subject to sudden changes in behaviour.

Most drift detection mechanisms designed for classification problems works in a post-event manner: after receiving the data set completely (patterns and class labels of the train and test set), they apply a sequence of procedures to identify some change in the class-conditional distribution - a concept drift. However, detecting changes after its occurrence can be in some situations harmful for the process under supervision. This paper proposes a pre-event approach for abrupt drift detection, called by A2D2. Briefly, this method is composed of three steps: (i) label the patterns from the test set, using an unsupervised method; (ii) compute some statistics from the train and test set, conditioned on the given class labels; and (iii) compare the train and test statistics using a multivariate hypothesis test. Also, it has been proposed a procedure for creating datasets with abrupt drift. This procedure was used in the sensivity analysis of A2D2, in order to understand the influence degree of each parameter on its final performance.

This work presents a Genetic Fuzzy Controller (GFC), called Genetic Programming Fuzzy Inference System for Control tasks (GPFIS-Control). It is based on Multi-Gene Genetic Programming, a variant of canonical Genetic Programming. The main characteristics and concepts of this approach are described, as well as its distinctions from other GFCs. Two benchmarks application of GPFIS-Control are considered: the Cart-Centering Problem and the Inverted Pendulum. In both cases results demonstrate the superiority and potentialities of GPFIS-Control in relation to other GFCs found in the literature.

In our previous works [1, 2], we proposed NEVE, a model that uses a weighted ensemble of neural network classifiers for adaptive learning, trained by means of a quantum-inspired evolutionary algorithm (QIEA). We showed that the neuro-evolutionary classifiers were able to learn the dataset and to quickly respond to any drifts on the underlying data. Now, we are particularly interested on analyzing the influence of an unlimited ensemble, instead of the limited ensemble from NEVE. For that, we modified NEVE to work with unlimited ensembles, and we call this new algorithm NEVE++. To verity how the unlimited ensemble influences the results, we used four different datasets with concept drift in order to compare the accuracy of NEVE and NEVE++, using two other existing algorithms as reference.

This work presents a Genetic Fuzzy Classification System (GFCS) called Genetic Programming Fuzzy Classification System (GPF-CLASS). This model differs from the traditional approach of GFCS, which uses the metaheuristic as a way to learn “if-then” fuzzy rules. This classical approach needs several changes and constraints on the use of genetic operators, evaluation and selection, which depends primarily on the metaheuristic used. Genetic Programming makes this implementation costly and explores few of its characteristics and potentialities. The GPF-CLASS model seeks for a greater integration with the metaheuristic: Multi-Gene Genetic Programming (MGGP), exploring its potential of terminals selection (input features) and functional form and at the same time aims to provide the user with a comprehension of the classification solution. Tests with 22 benchmarks datasets for classification have been performed and, as well as statistical analysis and comparisons with others Genetic Fuzzy Systems proposed in the literature.

This work describes the use of a weighted ensemble of neural network classifiers for adaptive learning. We train the neural networks by means of a quantum-inspired evolutionary algorithm (QIEA). The QIEA is also used to determine the best weights for each classifier belonging to the ensemble when a new block of data arrives. We show that the neuroevolutionary classifiers are able to learn the data set and to quickly respond to any drifts on the underlying data. We also compare the results reached by our model with an existing algorithm, Learn++.NSE, in two different nonstationary scenarios.

This work describes the use of a weighted ensemble of neural network classifiers for adaptive learning. We train the neural networks by means of a quantum-inspired evolutionary algorithm (QIEA). The QIEA is also used to determine the best weights for each classifier belonging to the ensemble when a new block of data arrives. After running several simulations using two different datasets and performing two different analysis of the results, we show that the proposed algorithm, named neuro-evolutionary ensemble (NEVE), was able to learn the data set and to quickly respond to any drifts on the underlying data, indicating that our model can be a good alternative to address concept drift problems. We also compare the results obtained by our model with an existing algorithm, Learn++.NSE, in two different nonstationary scenarios.

The relevance of business process modeling and information technology is growing. That happens because, in general, information systems automate, assist and control those processes. So, the business understanding can be crucial for an appropriate requirements definition. However, there are still few methods that exploit the contributions that the Business Process Engineering can provide for requirements elicitation to produce results more compliant to the company needs. Thus, this work presents a method and its application in a real case that uses the business processes as a way to extract the requirements.