Prototyping of machine learning (ML) solutions represents a pivotal stage in developing ML-enabled systems. In this course, the prototype serves as a means of communication and should demonstrate the technical feasibility and value to technical and non-technical stakeholders. But, in the context of the current ML solution prototyping process and tooling environment, non-technical stakeholders are limited in their ability to participate effectively, primarily due to the difficulty of understanding the specific ML solution strategy being implemented. In addition, valuable knowledge is lost during the prototype development process because the process is not sufficiently documented, preserved, or made easily accessible for future projects. To significantly improve the development of ML prototypes, we propose an extended ML prototyping process and tool support in the form of a toolbox. Preliminary implementations of some tools of the toolbox are presented.

When experimenting on solutions to Machine Learning problems, data scientists often integrate nonlinear workflows into Jupyter Notebooks to explore different approaches and evaluate the impact of changes such as using different Machine Learning models or adjusting parameters. This mode of working leads to Notebooks that are cluttered and difficult to navigate which complicates refining and reusing previous experiments later on. Jupyter Notebooks lack inherent support for such a mode of working. Therefore, we propose the JupyterLab extension HisTreethat provides an interactive tree-based representation of the experiment history in Jupyter Notebooks. Hereby, Note-book versions triggered by specified Notebook operations, are automatically saved and arranged in a tree structure. In this way, HisTreeallows data scientists to explore, compare, organize, and refine their past experiment approaches. In this paper, first, we introduce the concept of an experiment history tree model. This is followed by a comprehensive description of the functionality of HisTree,which aims to support data scientists in organizing experiments in Jupyter Notebooks. Initial feedback from user experiments shows that the tree-based experiment model is very promising and that the HisTreeextension is both useful and usable to conduct ML experiments in Jupyter Notebooks.

Effective and efficient testing of complex component based software systems is hard. Isolated test cases that test isolated components are focused and efficient but are not effective in detecting integration faults. Integration test cases on the other hand are hard to develop and maintain. In this paper we present an unit test based integration meta-model and testing approach, to test the integrateability of component-based systems based on structural and behavioral information derived from test executions of the respective components' unit test cases. The meta-model is formalized using the property graph model and integration criteria are presented to detect certain types of integration faults early on. Last but not least we propose an approach to derive integration tests from the data contained in the model.

Continuous Delivery (CD) aims at reducing the cycle time from changes to software release while also increasing the software quality. To automate CD, delivery process models, defining all delivery activities need to be designed. Quality properties of delivery process models, such as maintainability, still oppose challenges. Previous research indicates that the quality of such models can be improved by aligning them with the software architecture. While software architecture knowledge is only incorporated implicitly, deep technical and process knowledge is required. On this basis, this paper introduces a new kind of delivery process models that focus mainly on software architecture knowledge. Hereby, we discard the current activity-centric view and shift to an artifact-centric view. Moreover, we outsource the required process- and technical knowledge to a transformation activities knowledge base. In order to make an artifact-based delivery process model executable, we provide a model-to-model transformation which constructs a CD pipeline from an artifact-based model with the help of the transformation activities knowledge base. We evaluated our approach by conducting a small industrial qualitative user study. It showed that low-experienced developers benefit from the reduced knowledge requirements of the artifact-based modeling approach.

Combinatorial robustness testing is an approach to generate separate test inputs for positive and negative test scenarios. The test model is enriched with semantic information to distinguish valid from invalid values and value combinations. Unfortunately, it is easy to create over-constrained models and invalid values or invalid value combinations do not appear in the final test suite. In this paper, we extend previous work on manual repair and develop a technique to semi-automatically repair over-constrained models. The technique is evaluated with benchmark models and the results indicate a small computational overhead.

Error-handling is an important means to improve the robustness of a system and testing error-handling is crucial to ensure its correctness. In this paper, we argue that error-handling leads to input masking which requires special treatment in for combinatorial testing. Therefore, we propose an extension to combinatorial testing including a robustness fault model and robustness combination strategy. We also provide an evaluation which compares its efficiency to normal combinatorial testing.

Enterprise Architecture (EA) is a widely accepted means to ease the alignment of IS projects with enterprise-wide objectives. One central artifact of EA are EA models, which provide a holistic view on the organization and support EA's stakeholder to create added value. As EA collects its data from different sources, the data can be contradictory. This work contributes to existing research by proposing a novel approach to deal with contradictory data without solving the thereby caused conflicts. In order to achieve this objective, we refine the Predictive, Probabilistic Architecture Modeling Framework (P2AMF) introduced by Johnson et al., which already incorporates a way to represent uncertainty regarding the existence of modelled entities. To make our technique usable, we generalize P2AMF from its UML/OCL notation to a graph presentation in order to apply it to EA models notated in arbitrary notations like ArchiMate. Furthermore, we add alternative scenarios in different versions along a time series to meet the requirements of a distributed EA evolution. To show the applicability of our approach, we developed a proof of concept prototype by implementing the proposed calculations and guidelines on a Neo4j graph database. Last, we argue that our approach meets the stated requirements of a distributed EA evolution.

Architecture conformance checks are important to control the inevitable drift between the prescriptive and descriptive architectures of a software system during its evolution. To this end, behavior-based architecture conformance checks should be employed in addition to static ones. But behavior-based analyses suffer from an important shortcoming: their results depend on the adequateness of the monitored behavior. Our claim is that a behavior-based architecture conformance check is adequate if (1) the architectural rules relevant from a behavior viewpoint are expressible and can be checked against and (2) the set of captured scenarios are relevant for exhibiting the overall behavior of the system. First, using ARAMIS, our approach to behavior-based architecture reconstruction and conformance checking, we exemplify how conformance rules can be expressed. Then, we propose a metric to investigate the relevance of the monitored scenarios. Last we present two case studies, in which we defined and checked communication rules and discuss the relevance of the monitored scenarios.

The reconstruction of software architectures and the evaluation of architecture conformance of software systems is a long-studied research topic. Although up-to-date architecture descriptions are necessary to understand and evolve systems, they are rarely available. Consequently, many software architecture reconstruction approaches and tools have been proposed. Despite this, software architects still do not extensively employ these tools and suffer from negative effects when relying only on outdated descriptions. In this paper we present ARAMIS, an approach and associated toolbox that aims to support the behavior-based reconstruction of up-to-date architecture descriptions based on the correction of possibly outdated prescriptive ones. Additionally, ARAMIS addresses the so-called meta-model incompatibility problem by allowing architects to use their own architecture description language instead of the one that the reconstruction tool requires. ARAMIS checks the behavior-based architecture conformance of a system based on pre-specified communication rules, derives up-to-date descriptions and enables their exploration. We have evaluated ARAMIS during several combined surveys and interviews with subjects from both the industry and academia and have obtained positive results.

Abstract. Model evolution in model libraries di↵ers from general model evolution. It limits the scope to the manageable and allows to develop clear concepts, approaches, solutions, and methodologies. Looking at model quality in evolving model libraries, we focus on quality concerns related to reusability. In this paper, we put forward our proactive quality guidance approach for model evolution in model libraries. It uses an editing-time assessment linked to a lightweight quality model, corresponding metrics, and simplified reviews. All of which help to guide model evolution by means of quality gates fostering model reusability.

Software architecture lies at the backbone of any software system and its choice directly influences important non-functional characteristics such as maintainability, extensibility, etc. Up-to-date software architecture descriptions should be at any time available to support the analysis and evaluation of the current state of the architecture. However the current state of the art lacks both methodologies and tools for ensuring availability of architecture descriptions and fails to offer objective means for evaluating software architectures. Currently, no generally accepted method for comparing software from an architecture point of view exists. In this paper, we present our current results towards creating a so-called architecture quality index that includes a bidirectional architecture quality model as well as a quality benchmark created for the context of the ARAMIS research project. The proposed architecture quality index aims to support the architects to evaluate and compare the architecture of software systems based on information extracted during the considered systems' run-time.

Architecture descriptions greatly contribute to the understanding, evaluation and evolution of software but despite this, up-to-date software architecture views are rarely available. Typically only initial descriptions of the static view are created but during the development and evolution process the software drifts away from its description. Methods and corresponding tool support for reconstructing and evaluating the current architecture views have been developed and proposed, but they usually address the reconstruction of static and dynamic views separately. Especially the dynamic views are usually bloated with low-level information (e.g., Object interactions) making the understanding and evaluation of the behavior very intricate. To overcome this, we presented ARAMIS, a general architecture for building tool-based approaches that support the architecture-centric evolution and evaluation of software systems with a strong focus on their behavior. This work presents ARAMIS-CICE, an instantiation of ARAMIS. Its goal is to automatically test if the run-time interactions between architecture units match the architecture description. Furthermore, ARAMIS-CICE characterizes the intercepted behavior using two newly-defined architecture metrics. We present the fundamental concepts of ARAMIS-CICE: its meta-model, metrics and implementation. We then discuss the results of a two-folded evaluation. The evaluation shows very promising results.

Even though a lot of work was contributed to extend and enhance metric requirements gathering techniques, metric systems are often developed chaotically and a solid dedicated metric system engineering approach is still missing. This paper provides our experiences at developing a metric system together with a large IT service provider and presents an overview on our reference architecture for enterprise measurement infrastructures. Furthermore we give some insights into our metric systems engineering approach which integrates software engineering best practices, modern ideas like micro services, and well established metric related techniques such as GQM.

In our former work we proposed a micro service-based reference architecture for Enterprise Measurement Infrastructures (EMI) which received encouraging feedback. The reference architecture supports the systematic development of measurement systems. This paper provides deeper insight into the application of the reference architecture by presenting the results of two field studies after an examination of the most important requirements that drove the development of the reference architecture. The two selected field studies were conducted with large cooperation partners from industry and research and addressed real problems. Using our reference architecture, development process, and requirements gathering techniques we were able to successfully build the EMIs presented in this paper. These results further ease the application of micro service inside our reference architecture and support practitioners with specific examples.

Content-assist systems and code completion are nicely accessible in integrated development environments (IDEs). Using multiple data sources and performing sophisticated completion in several editors is quite common. However, no such supporting system exists for modeling environments, e.g., a completion mechanism in class diagrams is only existent for textual items like names, if at all. We designed a framework to bolster model recommendation research and present the requirements, concepts, architecture, and the realization below. Last of which is easily extendable and adaptable to either new data recommendation strategies or new environments like editors. As additional tool support, we provide a simulation environment, which ease development as well as implementing recommendation algorithm. Accordingly, researchers get all the conceptual groundwork and a realized infrastructure that ease the initial burden to start recommendations in modeling environments.

Recommender systems for source code artifacts are newly emerging and are now successfully supporting programmers. Their underlying knowledge bases, recommender algorithms, and user interfaces are well studied. Integrated into the development environment, they do a fairly good job in reducing complexity and development time.

The documentation of architecture and design decisions lies at the backbone of building a comprehensive architectural knowledge basis within a company. As a consequence, a plethora of supporting frameworks has been lately proposed by the research community. The existing frameworks focus on capturing the rationale that lies behind a certain decision, but less on sustaining the collaborative process that architects employ when making decisions. In this paper, we propose an innovative architectural decision making process that sustains the collaboration of architects, the timely notification of involved stakeholders, the inclusion of feedback cycles to improve the overall quality of the architecting process and a tag-based traceability system that leverages informal learning. The analysis of the current state of the practice in the industry has been conducted within various workshops and interviews with our industry cooperation partner - the software provider of one of the biggest insurance trusts worldwide. Based on these results, we have identified various improvement potentials that are still not addressed by existing research in the field.

Software architecture stands at the backbone of any software system. An up-to-date description of the architecture greatly contributes to its understanding, evaluation and evolution. Despite its importance, the architecture is typically described only in the preliminary development phases and later becomes subject of continuous degradation. Therefore, methods and corresponding tool support for reconstructing the current views of a system's architecture have been developed and proposed. Current state of the art addresses the reconstruction of static and dynamic views separately. The reconstruction is typically conducted post-mortem using heavy weight infrastructures. We have conceptually defined and built a light-weight run-time monitoring infrastructure that produces meaningful real-time visualizations of object-level interactions. We consider that the possibility to observe the behavior of a system in real-time positively impacts the documentation of the software architecture, its understandability, communication and trace ability to usage scenarios. We have evaluated the monitoring infrastructure on a software project in different development stages. The evaluation has shown very promising results.

Metrics and especially metric-based monitoring dashboards provide valuable information and insights for managers in software development organizations. However, implementing and launching a company wide metrics program is very hard and time consuming. This paper describes the history and our experience with the development of a metrics program at Generali Deutschland Informatik Services, a CMMI level 3 certified company. We also provide important lessons learned alongside a list of consolidated best practices for the implementation and maintenance of a large metrics and dashboard program. We believe that both are useful for every practitioner and researcher in this field and help to build better and more sustainable metrics and dashboard development processes and infrastructures.

Large scale measurement systems are hard to build and to maintain. In this paper we propose an architecture blueprint for a federalist Enterprise Measurement Infrastructure (EMI) which helps to address these typical weaknesses of centralistic measurement systems. The EMI is based on the ideas of Service Oriented Measurements. We combined these with modern ideas from the area of Enterprise Application Integration and extended the ISO 15939 data flow to allow a more flexible and elegant solution. The current prototypes of EMI implementations and field studies prove the benefits of the architecture blueprint over existing solutions. We strongly belief that the EMI can help to build better, extendible, and maintainable measurement systems which are integrated and aligned with modern business needs.

The architecture of software systems should be well documented and up to date. Knowledge about the software architecture of a software system enables reasoning regarding the software's qualities such as modifiability, extensibility, security, etc. However, very often the architecture is only described during the initial phases of a software project and then undergoes progressive degradation. A degenerated architecture description cannot be used for reasoning regarding the qualities of the software, even if it possibly conveys the required functionality. This paper proposes an approach for a continuous model-based monitoring and semi-automatic evaluation of software architectures, meant to support the architecture-based evolution of software systems at various abstraction levels.

A variety of reference models such as CMMI, COBIT or ITIL supports IT organizations to improve their processes. Although these process improvement reference models (IRM) cover different domains they also share some similarities. There are organizations that address multiple domains and want to take the guidance of different IRMs. As IRMs overlap in some processes, we present an approach to compare parts of IRMs (the IRMs' procedures) that is based on a common IRM integration model and on similarity metrics. Our approach enables organizations to efficiently adopt and assess multiple IRMs by automatically identifying similarities and specific details of the different IRMs.

The lifecycle of defects reports and enhancement requests collected in the Bugzilla database of the GNOME project provides valuable information on the evolution of the change request process and for the assessment of process quality in the GNOME sub projects. We present a quality model for the analysis of quality characteristics that is based on evaluating metrics on the Bugzilla database, and illustrate it with a comparative evaluation for 25 of the largest products within GNOME.

While engineering of embedded & real-time systems has moved much into the focus of the research community, being strongly promoted by those prominent application areas as the automotive, aerospace & defense, or telecommunications industry, small embedded & real-time systems, as they can be found in somehow marginal application areas as the industrial automation, are still treated a bit stepmotherly. In particular, profound methodical support for the software development of such small devices is almost unavailable. With MeDUSA we especially target the domain of such small embedded & real-time systems, and explicitly address the very special technological, economical, and organizational constraints that have to be faced in such marginal application areas.

Model driven development has evolved to a mature methodology and technology usable for some industrial settings. Within the automation domain it is an upcoming approach. This paper addresses challenges present in the automation domain when it comes to the usage of model driven development. Quality, life cycle, legacy systems, mental approach and safety challenges are briefly discussed.

Prototyping, a method and technique frequently used in many engineering disciplines, has been adopted as a technique in software engineering to improve the calculation of new projects involving risks. However, there has so far been a lack of documented experience with the use of prototyping in industrial software production. The paper tries to close this gap. First, we introduce central prototyping concepts and terminology. We also present five industrial software projects in which explicit use was made of prototyping. Based on our analysis of these projects we present the resulting conclusions: prototyping means more than rapidly developing user interfaces; prototyping is a central part of a development strategy; prototyping means end user involvement; finding the right mixture of prototypes improves the development process.>