• We introduced a fully automated framework that quantifies the relationship between microservice architecture complexity, derived from multiple types of statically detected dependencies, and performance-related quality attributes. • We analyzed five complex benchmark systems, including four structurally distinct releases of the Train-Ticket microservice benchmark and one instance of the DeathStarBench suite. • Regarding the relationship between microservice-architecture complexity and overall performance, our analysis demonstrated that systems with poor structural complexity, reflected in high propagation cost and high Clique ratios, exhibited reduced scalability and supported fewer user requests. • Regarding the relationship between individual service complexity and their performance, our results show that endpoints with higher coupling scores tend to have longer response times. Using Spearman’s rank correlation to assess the correlation between each endpoint’s coupling score and its performance score, we find a statistically significant positive correlation. A microservice architecture is intended to promote modularity and evolvability. In this paper, we present an automated framework for assessing the relationship between microservice architecture complexity and performance-related quality attributes. In this framework, we use PPTAM, a performance testing tool, to evaluate system response time under varying user loads, and DV8, an architecture analysis tool, to assess architectural complexity and the complexity of individual services using coupling scores, propagation cost, and architectural antipatterns derived from various types of dependency relations. Using this approach, we evaluated five benchmark systems, including four releases of a microservice system that share similar functionalities but differ in structural design. The results show that microservice architectures with poor complexity scores also exhibited degraded performance outcomes. This automated framework, for the first time, enables a comprehensive measurement of microservice architecture complexity, formed through multiple types of statically extracted dependencies, and its correlation with dynamically obtained performance metrics.

User interfaces serve as the primary interaction point between users and software systems, yet designing intuitive and effective interfaces remains a challenge, particularly when visualizing complex technical data. In this study, we introduce the concept of visualization impedance mismatch, which describes the misalignment between the way data and mechanisms are visualized and the mental models users employ when interacting with a system. Through a case study involving ten developers, we identify key difficulties in UI design and extract $\mathbf{1 5}$ user interface patterns aimed at reducing this mismatch. These patterns provide structured solutions to common UI design challenges, addressing issues such as feedback mechanisms, action tracking, contextual messaging, and error prevention. We further discuss internal and external threats to validity when identifying such patterns. Our findings contribute to a deeper understanding of visualization challenges in software design and offer practical guidelines for developers seeking to improve user experience in data-heavy applications.

Software quality is crucial in software development: if not addressed in early phases of the software development life cycle, it may even lead to technical bankruptcy, i.e., a situation in which modifications cost more than redeveloping the application from scratch. In addition, code security must also be addressed to reduce software vulnerabilities and to comply with legal requirements. In this work, we aim to investigate the relationship between refactoring code quality and software security, with the purpose of understanding whether and to what extent improving software quality could have a positive impact on software security as well. Specifically, we investigate to what extent rule violations of a software quality tool such as SonarQube overlap with rule violations of a software vulnerability tool like Fortify Static Code Analyzer. We first compared the rules encoded in the quality models of both tools, to discover possible overlapping cases. Later, we compared the issues raised by both tools on a set of open source Java projects; we also investigated the cases in which a quality refactoring process impacts over software security (thus removing one or more vulnerabilities). We furthermore validated our results statistically. Our results show that resolving software quality issues might also resolve security issues but only in part: many security issues still persist in the source code; also, some quality aspects are more likely to be improved in respect to others. In addition, this empirical study uncovers rule co-occurrences between the two tools. This study confirms the need for using a security-oriented static analysis tool to enforce software security instead of relying only on a quality-oriented one. Results have highlighted important insights for practitioners.

Methods for Optical Flow (OF) estimation based on Deep Learning have considerably improved traditional approaches in challenging and realistic conditions. However, data-driven approaches can inherently be biased, leading to unexpected under-performance in real application scenarios. In this paper, we first observe that the OF estimation accuracy varies with motion direction, and name this phenomenon ‘OF sign imbalance’. The sign imbalance cannot be assessed by means of the endpoint-error (EPE), the typical training and evaluation metric for Deep Optical Flow estimators. This paper tackles this issue by proposing a new metric to assess the sign imbalance, which is compared to the endpoint-error. We provide an extensive evaluation of the sign imbalance for the state-of-the-art optical flow estimators. Based on the evaluation, we propose two strategies to mitigate the phenomenon, i) by constraining the model estimations during inference, and, ii) by constraining the loss function during training. Testing and training code is available at: www.github.com/stsavian/equivariant_of_estimation.

Various works investigated microservice anti-patterns and bad smells in the past few years. We identified seven secondary publications that summarize these, but they have little overlap in purpose and often use different terms to describe the identified anti-patterns and smells. This work catalogs recurring bad design practices known as anti-patterns and bad smells for microservice architectures, and provides a classification into categories as well as methods for detecting these practices. We conducted a systematic literature review in the form of a tertiary study targeting secondary studies identifying poor design practices for microservices. We provide a comprehensive catalog of 58 disjoint anti-patterns, grouped into five categories, which we derived from 203 originally identified anti-patterns for microservices. The results provide a reference to microservice developers to design better-quality systems and researchers who aim to detect system quality based on anti-patterns. It also serves as an anti-pattern catalog for development-aiding tools, which are not currently available for microservice system development but could mitigate quality degradation throughout system evolution.

Coping with the rapid growing complexity in contemporary software architecture, tracing has become an increasingly critical practice and been adopted widely by software engineers. By adopting tracing tools, practitioners are able to monitor, debug, and optimize distributed software architectures easily. However, with excessive number of valid candidates, researchers and practitioners have a hard time finding and selecting the suitable tracing tools by systematically considering their features and advantages. To such a purpose, this paper aims to provide an overview of popular Open tracing tools via comparison. Herein, we first identified 30 tools in an objective, systematic, and reproducible manner adopting the Systematic Multivocal Literature Review protocol. Then, we characterized each tool looking at the (1) measured features, (2) popularity both in peer-reviewed literature and online media, and (3) benefits and issues. As a result, this paper presents a systematic comparison amongst the selected tracing tools in terms of their features, popularity, benefits and issues. The result mainly shows that each tracing tool provides a unique combination of features with also different pros and cons. The contribution of this paper is to provide the practitioners better understanding of the tracing tools facilitating their adoption.

Continuous quality assurance for extra-functional properties of modern software systems is today a big challenge as their complexity is constantly increasing to satisfy market demands. This is the case of microservice systems. They provide high control on the scale of operation by means of fine-grained service decomposition, but this demands careful consideration of the relations between performance of individual microservices and service failures.

Regression testing wants to prevent that errors, which have already been corrected once, creep back into a system that has been updated. A naïve approach consists of re-running the entire test suite (TS) against the changed version of the software under test (SUT). However, this might result in a time-and resource-consuming process; e.g., when dealing with large and/or complex SUTs and TSs. To avoid this problem, Test Case Selection (TCS) approaches can be used. This kind of approaches build a temporary TS comprising only those test cases (TCs) that are relevant to the changes made to the SUT, so avoiding executing unnecessary TCs. In this paper, we introduce CATTO (Commit Adaptive Tool for Test suite Optimization), a tool implementing a TCS strategy for SUTs written in Java as well as a wrapper to allow developers to use CATTO within IntelliJ IDEA and to execute CATTO just-in-time before committing changes to the repository. We conducted a preliminary evaluation of CATTO on seven open-source Java projects to evaluate the reduction of the test-suite size, the loss of fault-revealing TCs, and the loss of fault-detection capability. The results suggest that CATTO can be of help to developers when performing TCS. The video demo and the documentation of the tool is available at: https://catto-tool.github.io/

Effective and automated verification techniques able to provide assurances of performance and scalability are highly demanded in the context of microservices systems. In this paper, we introduce a methodology that applies specification-driven load testing to learn the behavior of the target microservices system under multiple deployment configurations. Testing is driven by realistic workload conditions sampled in production. The sampling produces a formal description of the users’ behavior through a Discrete Time Markov Chain. This model drives multiple load testing sessions that query the system under test and feed a Bayesian inference process which incrementally refines the initial model to obtain a complete specification from run-time evidence as a Continuous Time Markov Chain. The complete specification is then used to conduct automated verification by using probabilistic model checking and to compute a configuration score that evaluates alternative deployment options. This paper introduces the methodology, its theoretical foundation, and the toolchain we developed to automate it. Our empirical evaluation shows its applicability, benefits, and costs on a representative microservices system benchmark. We show that the methodology detects performance issues, traces them back to system-level requirements, and, thanks to the configuration score, provides engineers with insights on deployment options. The comparison between our approach and a selected state-of-the-art baseline shows that we are able to reduce the cost up to 73% in terms of number of tests. The verification stage requires negligible execution time and memory consumption. We observed that the verification of 360 system-level requirements took ∼1 minute by consuming at most 34 KB. The computation of the score involved the verification of ∼7k (automatically generated) properties verified in ∼72 seconds using at most ∼50 KB.

Context. Software Performance Antipatterns (SPAs) research has focused on algorithms for the characterization, detection, and solution of antipatterns. However, existing algorithms are based on the analysis of runtime behavior to detect trends on several monitored variables (e.g., response time, CPU utilization, and number of threads) using pre-defined thresholds. Objective. In this paper, we introduce a new approach for SPA characterization and detection designed to support continuous integration/delivery/deployment (CI/CDD) pipelines, with the goal of addressing the lack of computationally efficient algorithms.

Many of today's software systems are maintained over years or even decades. To ensure that software remains useful, new features have to be added or old features have to be adapted to respond to new or changed requirements. As time goes on, some of the features become obsolete, i.e., are not needed anymore. Typically, these features are not removed because of various reasons, e.g., because removing them might be considered too costly, the costs of keeping unused features is considered low, or because of the "sunk cost fallacy", i.e., that a feature is considered worth to keep because of the previously invested resources (time, money or effort) to build it. The consequences of keeping unused source code can impact maintainability, technical debt, performance, and extensibility of the system. This can lead to lower development productivity and to a reduced innovation ability, consequently reducing competitiveness on the market. This paper aims to present an approach to identify features based on their value and on costs for keeping or removing them.

Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with these techniques is that code introduces new vocabulary at a far higher rate than natural language, as new identifier names proliferate. Both large vocabularies and out-of-vocabulary issues severely affect Neural Language Models (NLMs) of source code, degrading their performance and rendering them unable to scale.

The task “recommend a video to watch next?” has been in the focus of recommender systems’ research for a long time. However, adequately exploiting the clues hidden in the sequences of actions of user sessions in order to reveal users’ short-term intentions moved only recently into the focus of research. Based on a real-world application scenario, in this paper, we propose a Markov Chain-based transition probability matrix to efficiently reveal the short-term preferences of individuals. We experimentally evaluated our proposed method by comparing it against state-of-the-art algorithms in an offline as well as a live evaluation setting. In both cases our method not only demonstrated its superiority over its competitors, but exposed a clearly stronger engagement of users on the platform. In the online setting, our method improved the click-through rate by up to 93.61%. This paper therefore contributes real-world evidence for improving the recommendation effectiveness, by considering sequence-awareness, since capturing the short-term preferences of users is crucial in the light of items with a short life span such as tv programs (news, tv shows, etc.).

Microservices have emerged as an architectural style for developing distributed applications. Assessing the performance of architecture deployment configurations — e.g., with respect to deployment alternatives — is challenging and must be aligned with the system usage in the production environment. In this paper, we introduce an approach for using operational profiles to generate load tests to automatically assess scalability pass/fail criteria of microservice configuration alternatives. The approach provides a Domain-based metric for each alternative that can, for instance, be applied to make informed decisions about the selection of alternatives and to conduct production monitoring regarding performance-related system properties, e.g., anomaly detection. We have evaluated our approach using extensive experiments in a large bare metal host environment and a virtualized environment. First, the data presented in this paper supports the need to carefully evaluate the impact of increasing the level of computing resources on performance. Specifically, for the experiments presented in this paper, we observed that the evaluated Domain-based metric is a non-increasing function of the number of CPU resources for one of the environments under study. In a subsequent series of experiments, we investigate the application of the approach to assess the impact of security attacks on the performance of architecture deployment configurations.

As a developer, it is often complex to foresee the impact of changes in source code on usage, e.g., it is time-consuming to find out all components that will be impacted by a change or estimate the impact on the usability of a failing piece of code. It is therefore hard to decide how much effort in quality assurance is justifiable to obtain the desired business goals. In this paper, to reduce the difficulty for developers to understand the importance of source code, we propose an automated way to provide this information to developers as they are working on a given piece of code. As a proof-of-concept, we developed a plug-in for Microsoft Visual Studio Code that informs about the importance of source code methods based on the frequency of usage by the end-users of the developed software. The plug-in aims to increase the awareness developers have about the importance of source code in an unobtrusive way, helping them to prioritize their effort to quality assurance, technical excellence, and usability. code_call_lens can be downloaded from GitHub at https://github.com/xxMUROxx/vscode.code_call_lens.

Background. Process mining is a technique to build process models from "execution logs" (i.e., events triggered by the execution of a process). State-of-the-art tools can provide process managers with different graphical representations of such models. Managers use these models to compare them with an ideal process model or to support process improvement. They typically select the representation based on their experience and knowledge of the system. Aim. This work studies how to automatically build process models representing the actual intents (or uses) of users while interacting with a software system. Such intents are expressed as a set of actions performed by a user to a system to achieve specific use goals. Method. This work applies the theory of Hidden Markov Models to mine use logs and automatically model the use of a system. Results. Unlike the models generated with process mining tools, the Hidden Markov Models automatically generated in this study provide the intents of a user and can be used to recommend managers with a faithful representation of the use of their systems. Conclusions. The automatic generation of the Hidden Markov Models can achieve a good level of accuracy in representing the actual user's intents provided the log dataset is carefully chosen. In our study, the information contained in one-month set of logs helped automatically build Hidden Markov Models with superior accuracy and similar expressiveness of the models built together with the company's stakeholder.

This paper describes a novel design of a graphical user interface (GUI) to recommend useful command within an integrated development environment. The recommendation GUI contains a description of the suggested command, an explanation why the command is recommended, and a command usage example. The proposed design is based on the analysis of relevant guidelines identified in the literature. Its perceived usability and acceptance were evaluated in a live user study with 36 software developers. Our findings, partially contradicting existing literature, indicate that the presentation of the command-the description and the example-is perceived as more useful than the explanation of the rationale for the recommendation.

Context: Eliciting requirements from customers is a complex task. In Agile processes, the customer talks directly with the development team and often reports requirements in an unstructured way. The requirements elicitation process is up to the developers, who split it into user stories by means of different techniques. Objective: We aim to compare the requirements decomposition process of an unstructured process and three Agile processes, namely XP, Scrum, and Scrum with Kanban. Method: We conducted a multiple case study with a replication design, based on the project idea of an entrepreneur, a designer with no experience in software development. Four teams developed the project independently, using four different development processes. The requirements were elicited by the teams from the entrepreneur, who acted as product owner and was available to talk with the four groups during the project. Results: The teams decomposed the requirements using different techniques, based on the selected development process. Conclusion: Scrum with Kanban and XP resulted in the most effective processes from different points of view. Unexpectedly, decomposition techniques commonly adopted in traditional processes are still used in Agile processes, which may reduce project agility and performance. Therefore, we believe that decomposition techniques need to be addressed to a greater extent, both from the practitioners’ and the research points of view.

The software engineering community is continuously producing tools to tackle software construction problems. This paper presents a research study to identify which tools, artifacts, and commands developers use during task solving and how one can design software that can suggest and convince the developer to use specific software construction techniques. We want to understand under which conditions developers accept suggestions for a more efficient and effective usage of the available instruments, and if observed usage patterns correlate with observable improvements in the process or product. The expected results include detailed logs of how developers construct software during XP 2016, their preferences for software construction recommendations, and which effects accepted suggestions have on task execution and outcome.

Software Quality Assurance is a complex and time-expensive task. In this study we want to observe how agile developers react to just-in-time metrics about the code smells they introduce, and how the metrics influence the quality of the output.

Characterizing how users interact with software has many applications. For example, to understand which features are used, in which sequence operations are performed, etc. can help to understand how the user interface could be improved, to identify missing features, or to identify scenarios which are good candidates for test cases. This paper presents an industrial case study in which we investigate how users interact with an enterprise resource planning software using process mining. Our case study illustrates how we identify user interaction processes, the encountered advantages, and the faced challenges. One of the major findings is that the decision how to group events into cases is crucial for the application of the method.

Microenterprises (companies with less than 10 employees) are the dominating form of organizations in Europe. Unfortunately, many approaches to improve software development processes based on measurement are not tailored for such small companies. This poster proposes a measurement infrastructure that is developed with the goal to support microenterprises in measuring their process, product, and usage of the developed software to provide feedback to the entire development team. What the here presented tool wants to propose is to automate not only the data collection, but to be consequent in the rest of the feedback loop: to setup the interpretation and visualization of the data that, once this is done, no more intervention is needed.

The reputation of lightweight software development processes such as Agile and Lean is damaged by practitioners that claim benefits of such processes that are not true. Teams that want to demonstrate their seriousness, could benefit from matching their processes to the CMMI model, a recognized model by industry and the public administration. CMMI stands for Capability Maturity Model Integration and provides a reference model to improve and evaluate processes according to their maturity based on best practices. On the other hand, particularly in a lightweight software development process, the costs of a CMMI appraisal are hard to justify since its advantages are not directly related to the creation of value for the customer. This paper presents Jidoka4CMMI, a tool that — once a CMMI appraisal has been conducted — allows the documentation of the assessment criteria in form of executable test cases. The test cases, and so the CMMI appraisal, can be repeated anytime, without additional costs. The use of Jidoka4CMMI increases the benefits of conducting a CMMI appraisal. We hope that this encourages practitioners using lightweight software development processes to assess their processes using a CMMI model.

ABSTRACT This paper describes the challenges of handling changing requirements in software companies. This empirical investigation deals with the different sources of changes and with the different approaches to requirements evolution. We have considered the point of view of 35 managers of software companies interviewed through a questionnaire. The final results highlight some areas to improve requirements engineering and requirements management. Copyright © 2013 John Wiley & Sons, Ltd.

To understand and improve processes in organizations, six key questions need to be answered, namely, what, how, where, who, when, why. Organizations with established processes have IT system(s) that gather(s) information about some or all of the key questions. Software organizations usually have defined processes, but they usually lack information about how processes are actually executed. Moreover, there is no explicit information about process instances and activities. Existing process mining techniques face problems in coping with such environment. We propose a tool, Egidio, which uses non-invasively collected data and builds organizational models. In particular, we explain the tool within a software company, which is able to extract different aspects of development processes. The main contribution of Egidio is the ability to mine processes and organizational models from fine-grained data collected in a non-invasive manner, without interrupting the developers' work.