Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. • International survey gathering insights from 188 practitioners across 25 countries. • Overview of current practices and challenges in engineering ML-enabled systems. • Inferential quantitative analysis reporting the status quo with confidence intervals. • Qualitative analysis mapping ML life cycle challenges and causes of project failure.

[Context] Applying design principles has long been acknowledged as beneficial for understanding and maintainability in traditional software projects. These benefits may similarly hold for Machine Learning (ML) projects, which involve iterative experimentation with data, models, and algorithms. However, ML components are often developed by data scientists with diverse educational backgrounds, potentially resulting in code that doesn't adhere to software design best practices. [Goal] In order to better understand this phenomenon, we investigated the impact of the SOLID design principles on ML code understanding. [Method] We conducted a controlled experiment with three independent trials involving 100 data scientists. We restructured real industrial ML code that did not use SOLID principles. Within each trial, one group was presented with the original ML code, while the other was presented with ML code incorporating SOLID principles. Participants of both groups were asked to analyze the code and fill out a questionnaire that included both open-ended and closed-ended questions on their understanding. [Results] The study results provide statistically significant evidence that the adoption of the SOLID design principles can improve code understanding within the realm of ML projects. [Conclusion] We put forward that software engineering design principles should be spread within the data science community and considered for enhancing the maintainability of ML code.

Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In this paper, we present PerSpecML , a perspective-based approach for specifying ML-enabled systems that helps practitioners identify which attributes, including ML and non-ML components, are important to contribute to the overall system’s quality. The approach involves analyzing 60 concerns related to 28 tasks that practitioners typically face in ML projects, grouping them into five perspectives: system objectives, user experience , infrastructure, model, and data. Together, these perspectives serve to mediate the communication between business owners, domain experts, designers, software and ML engineers, and data scientists. The creation of PerSpecML involved a series of formative evaluations conducted in different contexts: (i) in academia, (ii) with industry representatives, and (iii) in two real industrial case studies . As a result of the diverse validations and continuous improvements, PerSpecML stands as a promising approach, poised to positively impact the specification of ML-enabled systems, particularly helping to reveal key components that would have been otherwise missed without using PerSpecML . Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board .

The precedent is a textual citation of prior court decisions. This undoubtedly offers great value in a common-law-based judicial system where courts are bound to their previous rulings, such as in the United States, Canada, and India. In those countries, precedent relevance detection is an issue that has attracted considerable attention where studies propose Network Science techniques for relevance measurement --- where decisions and their relationships are represented in network structures. However, those methods fail to capture the precedent relevance in the Brazilian scenario due to the massive and increasing number of decisions issued yearly. The Brazilian Supreme Court (STF), the highest judicial body in Brazil, has produced more than a million decisions over the last decade. Therefore, we propose an interpretable and cost-effective process to explore the precedent through latent topics that emerge, evolve, and fade over time in a collection of historical documents. To do so, we explore dynamic topic modeling with tensor decomposition as a tool to investigate the legal changes embedded in those decisions over time. We base our study on the individual decisions published by STF between 2000 and 2018. Additionally, through experiments, we explore the proposed process within different scenarios to investigate the precedent citations over the STF's recent history, and how those citations correlates with the legal named entities, such as legislative references. The experiments showed the process' capability to produce coherent and interpretable results for temporal analysis of the precedent citations in larger collections of documents. Also, it presents the potential to support further studies in the legal domain.

This work investigates the use of smooth neural networks for modeling dynamic variations of implicit surfaces under the level set equation (LSE). For this, it extends the representation of neural implicit surfaces to the space-time ℝ3× ℝ, which opens up mechanisms for continuous geometric transformations. Examples include evolving an initial surface towards general vector fields, smoothing and sharpening using the mean curvature equation, and interpolations of initial conditions.The network training considers two constraints. A data term is responsible for fitting the initial condition to the corresponding time instant, usually ℝ3× {0}. Then, a LSE term forces the network to approximate the underlying geometric evolution given by the LSE, without any supervision. The network can also be initialized based on previously trained initial conditions, resulting in faster convergence compared to the standard approach.

Machine learning (ML) teams often work on a project just to realize the performance of the model is not good enough. Indeed, the success of ML-enabled systems involves aligning data with business problems, translating them into ML tasks, experimenting with algorithms, evaluating models, capturing data from users, among others. Literature has shown that ML-enabled systems are rarely built based on precise specifications for such concerns, leading ML teams to become misaligned due to incorrect assumptions, which may affect the quality of such systems and overall project success. In order to help addressing this issue, this paper describes our work towards a perspective-based approach for specifying ML-enabled systems. The approach involves analyzing a set of 45 ML concerns grouped into five perspectives: objectives, user experience, infrastructure, model, and data. The main contribution of this paper is to provide two new artifacts that can be used to help specifying ML-enabled systems: (i) the perspective-based ML task and concern diagram and (ii) the perspective-based ML specification template.

The large-scale adoption of systems that automate classifications using Machine Learning (ML) algorithms raises pressing challenges as they support or make decisions with profound consequences for human beings. It is important to understand how users' trust is affected by ML models' suggestions, even when those models are wrong. Many research efforts have focused on the user's ability to interpret what a model has learned. In this paper, we seek to understand another aspect of ML interpretability: how the presence of classification probabilities affects users' trust in the model outcomes, especially in ambiguous instances. To this end, we conducted an online survey in which we asked participants to evaluate their agreement with an automatic classification made by an ML model before and after presenting them the model classification probabilities. Surprisingly, we found that, in ambiguous instances, respondents agreed more with incorrect model outcomes than with correct ones, requiring further analyses.

Petrobras is Brazil's largest publicly-held company, operating in the oil, natural gas, and energy industry. Internal efforts enabled Petrobras to identify Digital Transformation (DT) opportunities to further promote their operational excellence. While addressing these opportunities typically requires Research and Development (R&D) uncertainties that could lead traditional R&D cooperation terms to be negotiated in years, there are time-to-market constraints for fast-paced deliveries to experiment solution options. Having this in mind, they partnered up with PUC-Rio to establish a new DT initiative. [Goal] The goal of this paper is to present the Lean R&D approach, tailored within the new initiative to meet the aforementioned DT needs. [Method] We designed Lean R&D integrating the following building blocks: (i) Lean Inceptions, to allow stakeholders to jointly outline a Minimal Viable Product (MVP); (ii) parallel technical feasibility assessment and conception phases, allowing to `fail fast'; (iii) scrum-based development management; and (iv) strategically aligned continuous experimentation to test business hypotheses. We report on first experiences of applying Lean R&D in practice. [Results] Lean R&D enabled addressing research-related uncertainties early and to efficiently deliver valuable MVPs within fast-paced four months cycles. [Conclusions] In our first experiences Lean R&D showed itself suitable for supporting DT initiatives. However, more formal case studies are needed. The business strategy alignment and the continuous support of a highly qualified research team were considered key success factors.

Data privacy and protection has been a trending topic in recent years. The COVID 19 pandemic has brought about additional challenges and tensions. For example, sharing health data across several organizations is crucial for significant control and reduction of massive infection and death risks. This implies the need for broadly collecting and using personal and sensitive data, which raises the complexity of data protection and privacy challenges. Permissioned blockchain technology is one way to empower users in controlling how their data flows through the net, in a transparent and secure way, through an immutable, unified, and distributed database ruled by smart contracts. Given this background, we developed a second layer data governance model for permissioned blockchains based on the Governance Analytical Framework principles to be applied in pandemic situations. The model has been designed to organize the relationship between data subjects, data controller, and data processor. Regarding privacy concerns, our proposal complies with the Brazilian General Data Protection Law.

Globoplay is Globo Group’s digital video streaming platform and offers a very diverse video content catalogue ranging from international to brazilian productions such as movies, series, soap operas, and TV programs produced by Globo Group. One of the challenges with such large and diverse content collection is its distribution to the user base in order to help our subscribers with finding relevant content that meets their expectations and to increase their engagement with the product. In this work, we show the result of a content-based recommendation approach based on multi-modal features such as visual characteristics and audio patterns found in the video content. Using techniques applied to short videos, we model it as a similarity problem based on the content of the video, where, given a video, we establish the top-n videos most similar to it in the collection. For the evaluation, we conducted a study through interviews with a group of users to understand their perception of recommendations based on audiovisual characteristics. For the future, we plan to: explore and define the best approach to combine text, audio and video features for video recommendations; explore audiovisual features with other recommendation approaches such as session based and collaborative filtering; perform AB testing in production; and evaluate the proposal impact in business metrics.

In petroleum reservoirs, parameters such as permeability, porosity, extension, and skin effects, have a high level of uncertainty. Different techniques attempt to improve the estimation of these parameters to improve production forecasting and cost reduction. The conventional production test, for example, requires the analysis of reservoir well pressure data. However, these data have several noise sources, such as the tidal effect, that may compromise the accuracy of test results. This letter proposes a method to smooth the noise generated by the tidal effect from well pressure data, using mathematical tools, such as the Fourier transform. We apply the discrete sine transform type I (DST-I) to filter the signal, and we use the pressure derivative to verify the results.

The new Brazilian Code of Civil Procedure (CPC) has elevated the importance of precedents in the legal decision-making process. This increased the need to find relevant precedents for a given issue or dispute. Precedents play a central role in judicial thinking by providing information to judges about the legal relevance of particular facts and by establishing legal rules. Precedents are also an important argumentative tool, enabling lawyers to present arguments based on previous decisions. The automated search for relevant precedents is an unattended issue in the Brazilian scenario, partly due to the court's massive production of decisions -- only in 2018 the Brazilian Supreme Court (STF) produced more than 121.000 new rulings -- and partly due to the technical challenges arising from the unstructured nature of the court's practices. In this paper, we present a study of precedent relevance, taking into account the uniqueness of the Brazilian legal system and of STF. To do so, we conducted an exploratory investigation over the precedent network extracted from 1.152.963 decisions published by the STF between 2008 and 2018. This exploratory analysis, although interesting in itself, reveals important challenges that need to be overcome by future research in order for the technology to have the kind of impact it can have on legal practice and academia. In our conclusion, we set out possible paths forward, briefly considering some of the most promising ways to sort out the signal from the noise.

Fault detection is a crucial step in reservoir characterization. Despite the many tools developed in the past decades, automation of this task remains a challenge. We investigate the application of convolutional neural networks (CNNs) to seismic fault detection. CNN is a deep learning method growing in interest in the computer vision community, due to its high performances in a great variety of object detection tasks. One of the constraints of this method is the need to provide a massive number of interpreted data, a requirement particularly difficult to attend in the seismic area. To this end, we built a synthetic data set with simple fault geometries. The input of our network is the seismic amplitude only; the method does not require computing any seismic attribute. We apply a strategy of patch classification along the images, which requires a simple postprocess to extract the exact fault location. Our network shows good results on synthetic data and encouraging results when tested on regions of a real section of The Netherland offshore F3 block in the North Sea.

Background: Unhandled exceptions affect the reliability of web applications. Several studies have measured the reliability of web applications in use against unhandled exceptions, showing a recurrence of the problem during the maintenance phase. Detecting latent unhandled exceptions automatically is difficult and application-specific. Hence, general approaches to deal with defects in web applications do not treat unhandled exceptions appropriately. Aims: To design and evaluate a method that can support finding, correcting, and preventing unhandled exceptions in web applications. Method: We applied the design science engineering cycle to design a method called Pattern-Driven Maintenance (PDM). PDM relies on identifying defect patterns based on application server logs and producing static analysis rules that can be used for prevention. We applied PDM to two industrial web applications involving different companies and technologies, measuring the reliability improvement and the precision of the produced static analysis rules. Results: In both cases, our approach allowed identifying defect patterns and finding latent unhandled exceptions to be fixed in the source code, enabling to completely eliminate the pattern-related failures and improving the application reliability. The static analysis rules produced by PDM achieved a precision of 59-68% in the first application and 89-100% in the second, where lessons learnt from the first evaluation were addressed. Conclusions: The results strengthen our confidence that PDM can help maintainers to improve the reliability for unhandled exceptions in other existing web applications.

This letter proposes an algorithm for solving the sparse-spike deconvolution problem, named Lobbes (Lasso-based binary search for parameter selection). It improves the fast iterative shrinkage and threshold algorithm for Toeplitz-sparse matrix factorization by performing three steps to find a suitable regularization parameter: 1) a normalization procedure over the input data; 2) a binary search step based on the least absolute shrinkage and selection operator; and 3) the elimination of consecutive peaks similar to non-maximum suppression. Such parameter allows us to find a solution with a specified sparsity. We compare our results against the original algorithm and with the known sparse-inducing greedy approach of orthogonal matching pursuit. Relative to state-of-the-art, results demonstrate that Lobbes generates better results: better signal-to-noise ratio of the reconstructed signal and better result for reflectivity peaks. We also derive a new way to measure the quality of the deconvolution.

The main input data for a traffic management system are trajectories generated by active GPS devices installed in vehicles. In this case, such trajectories can be understood as mobile traffic sensors. A vehicle raw trajectory may be considered as a continuous data stream generated by a GPS device, installed in the vehicle, that continuously transmit its position. In some cities, such as the City of Rio de Janeiro, each bus circulating in the city has to be equipped with such device. Using this fact, the City Hall of Rio de Janeiro offers a public Web service that publishes, at about every minute, the current GPS position of all buses. However, this public data service offers only the instantaneous data, in other words, there is no historical data available. For this reason, it is necessary to develop another service that periodically queries this data and stores its entries for future processing and for useful applications, such as traffic analysis and planning. The objective of this paper is to present the Buses In Rio tool that implements this functionality and provides access to bus trajectory data since June, 2014. The current database has approximately 2 billion GPS samples.

Abstract Geometry processing applications frequently rely on octree structures, since they provide simple and efficient hierarchies for discrete data. However, octrees do not guarantee direct continuous interpolation of this data inside its nodes. This motivates the use of the octree's dual structure, which is one of the simplest continuous hierarchical structures. With the emergence of pointerless representations, with their ability to reduce memory footprint and adapt to parallel architectures, the generation of duals of pointerless octrees becomes a natural challenge. This work proposes strategies for dual generation of static or dynamic pointerless octrees. Experimentally, those methods enjoy the memory reduction of pointerless representations and speed up the execution by several factors compared to the usual recursive generation.

Vector fields analysis traditionally distinguishes conservative (curl-free) from mass preserving (divergence-free) components. The Helmholtz-Hodge decomposition allows separating any vector field into the sum of three uniquely defined components: curl free, divergence free and harmonic. This decomposition is usually achieved by using mesh-based methods such as finite differences or finite elements. This work presents a new meshless approach to the Helmholtz-Hodge decomposition for the analysis of 2D discrete vector fields. It embeds into the SPH particle-based framework. The proposed method is efficient and can be applied to extract features from a 2D discrete vector field and to multiphase fluid flow simulation to ensure incompressibility.

Abstract The definition of a good view of a 3D scene is highly subjective and strongly depends on both the scene content and the 3D application. Usually, camera placement is performed directly by the user, and that task may be laborious. Existing automatic virtual cameras guide the user by optimizing a single rule, e.g. maximizing the visible silhouette or the projected area. However, the use of a static pre‐defined rule may fail in respecting the user's subjective understanding of the scene. This work introduces intelligent design galleries, a learning approach for subjective problems such as the camera placement. The interaction of the user with a design gallery teaches a statistical learning machine. The trained machine can then imitate the user, either by pre‐selecting good views or by automatically placing the camera. The learning process relies on a Support Vector Machines for classifying views from a collection of descriptors, ranging from 2D image quality to 3D features visibility. Experiments of the automatic camera placement demonstrate that the proposed technique is efficient and handles scenes with occlusion and high depth complexities. This work also includes user validations of the intelligent gallery interface.

Abstract This work emerged from the following observation: usual search procedures for octrees start from the root to retrieve the data stored at the leaves. But as the leaves are the farthest nodes to the root, why start from the root? With usual octree representations, there is no other way to access a leaf. However, hashed octrees allow direct access to any node, given its position in space and its depth in the octree. Search procedures take the position as an input, but the depth remains unknown. This work proposes to estimate the depth of an arbitrary node through a statistical optimization of the average cost of search procedures. As the highest costs of these algorithms are obtained when starting from the root, this method improves on both the memory footprint by the use of hashed octrees, and execution time through the proposed optimization.

Abstract Performances of actual mesh compression algorithms vary significantly depending on the type of model it encodes. These methods rely on prior assumptions on the mesh to be efficient, such as regular connectivity, simple topology and similarity between its elements. However, these priors are implicit in usual schemes, harming their suitability for specific models. In particular, connectivity‐driven schemes are difficult to generalize to higher dimensions and to handle topological singularities. GEncode is a new single‐rate, geometry‐driven compression scheme where prior knowledge of the mesh is plugged into the coder in an explicit manner. It encodes meshes of arbitrary dimension without topological restrictions, but can incorporate topological properties, such as manifoldness, to improve the compression ratio. Prior knowledge of the geometry is taken as an input of the algorithm, represented by a function of the local geometry. This suits particularly well for scanned and remeshed models, where exact geometric priors are available. Compression results surfaces and volumes are competitive with existing schemes.

Abstract This paper proposes the stellar mesh simplification method, a fast implementation of the Four‐Face Cluster (FFC) algorithm. In this method, a probabilistic optimization heuristic substitutes the priority queue of the original method, which results in a 40% faster algorithm with the same order of distortion. It extends naturally to a progressive and/or multiresolution scheme for combinatorial surfaces. This work also presents a simple way to encode the hierarchy of the resulting multiresolution meshes. This work also focuses on important aspects for the development of a practical and robust implementation of this simplification technique, and on the analysis of the influence of the parameters.

Morse theory is a powerful tool for investigating the topology of smooth manifolds. It has been widely used by the computational topology, computer graphics, and geometric modeling communities to devise topology-based algorithms and data structures. Forman introduced a discrete version of this theory which is purely combinatorial. This work aims to build, visualize, and apply the basic elements of Forman's discrete Morse theory. It intends to use some of those concepts to visually study the topology of an object. As a basis, an algorithmic construction of optimal Forman's discrete gradient vector fields is provided. This construction is then used to topologically analyze mesh compression schemes, such as Edgebreaker and Grow&Fold. In particular, this paper proves that the complexity class of the strategy optimization of Grow&Fold is MAX-SNP hard.