ABSTRACT Objectives Cookiecutter is a popular, mature open‐source Python library for automating the creation of customized projects from templates. This kind of scaffolding is useful for enabling reuse, encapsulating expertise, achieve uniformity, and facilitating project creation for a range of languages and domains, including microservices, web applications, and data science. Despite their success, there is a lack of descriptions of general‐purpose project templating tools such as Cookiecutter in the literature. The objective of the paper is to provide a description of Cookiecutter that is useful for researchers and practitioners. Methods Our work is informed by our own use of Cookiecutter in the context of an industrial project. We describe Cookiecutter with the help of four different research questions. The first two relate to how Cookiecutter works (RQ1) and how it is used (RQ2). The latter two are concerned with providing guidance to users of Cookiecutter (RQ3) and identifying challenges that they may face (RQ4). Result Our answer to question RQ1 provides a succinct, high‐level description of the structure of Cookiecutter templates and Cookiecutter's execution semantics. For question RQ2, we provide an analysis of the 100 most popular Cookiecutter templates on GitHub and descriptions of three applications of Cookiecutter in different domains. For question RQ3, we identify quality attributes for Cookiecutter projects together with best practice recommendations. For question RQ4, our discussion of challenges is structured around different lifecycle activities related to the overall management of Cookiecutter templates. The potential for research results in related areas such as software product lines, feature and variability modeling, and model‐driven engineering to help address these challenges is highlighted. Conclusion The paper provides a comprehensive discussion of Cookiecutter, a successful general‐purpose project templating tool with demonstrated industrial use. The discussion covers fundamental and practical aspects of Cookiecutter and thus targets practitioners as well as researchers interested in general‐purpose templating and Cookiecutter in particular.

Documentation comments are essential for maintainability, yet they are often missing or outdated. This is true not only for programs in general-purpose languages, but also for artifacts in other languages often found in software projects such as scripts or configuration files. To address this problem, we present an approach that uses Large Language Models (LLMs) to generate header comments (aka, ‘block comments’ or ‘doc-strings’) for elements of different languages in different documentation formats. Given a file in some language and a description of elements in the file to be documented and the documentation format to be used, the approach generates header comments for all undocumented elements in the file that is guaranteed to conform to the documentation format. We describe a prototype implementation and its integration into an industrial development pipeline. Feedback from our industrial partner, an LLM-as-judge evaluation, and the participants of a user study involving a broad range of languages indicates that the approach is viable, able to produce sufficiently high-quality documentation in general, and holds potential for improving industrial documentation practices across different programming languages and teams.

The iterative and incremental nature of software development using models typically makes a model of a system incomplete (i.e., partial) until a more advanced and complete stage of development is reached. Existing model execution approaches (interpretation of models or code generation) do not support the execution of partial models. Supporting the execution of partial models at early stages of software development allows early detection of defects, which can be fixed more easily and at lower cost. This paper proposes a conceptual framework for the execution of partial models, which consists of three steps:static analysis,automatic refinement, andinput-driven execution. First, a static analysis that respects the execution semantics of models is applied to detect problematic elements of models that cause problems for the execution. Second, using model transformation techniques, the models are refined automatically, mainly by adding decision points where missing information can be supplied. Third, refined models are executed, and when the execution reaches the decision points, it uses inputs obtained either interactively or by a script that captures how to deal with partial elements. We created an execution engine calledPMExecfor the execution of partial models of UML-RT (i.e., a modeling language for the development of soft real-time systems) that embodies our proposed framework. We evaluatedPMExecbased on several use-cases that show that the static analysis, refinement, and application of user input can be carried out with reasonable performance, and that the overhead of approach, which is mostly due to the refinement and the increase in model complexity it causes, is manageable. We also discuss the properties of the refinement formally, and show how the refinement preserves the original behaviors of the model.

As software evolves, regression testing techniques are typically used to ensure the new changes are not adversely affecting the existing features. Despite recent advances, regression testing for distributed systems remains challenging and extremely costly. Existing techniques often require running a failing system several time before detecting a regression. As a result, conventional approaches that use re-execution without considering the inherent non-determinism of distributed systems, and providing no (or low) control over execution are inadequate in many ways. In this paper, we present MRegTest, a replay-based regression testing framework in the context of model-driven development to facilitate deterministic replay of traces for detecting regressions while offering sufficient control for the purpose of testing over the execution of the changed system. The experimental results show that compared to the traditional approaches that annotate traces with timestamps and variable values MRegTest detects almost all regressions while reducing the size of the trace significantly and incurring similar runtime overhead.

Ordering and replaying of execution traces of distributed systems is a challenging problem. State-of-the-art approaches annotate the traces with logical or physical timestamps. However, both kinds of timestamps have their drawbacks, including increased trace size. We examine the problem of determining consistent orderings of execution traces in the context of model-driven development of reactive distributed systems, that is, systems whose code has been generated from communicating state machine models. By leveraging key concepts of state machines and existing model analysis and transformation techniques, we propose an approach to collecting and reordering execution traces that does not rely on timestamps. We describe a prototype implementation of our approach and an evaluation. The experimental results show that compared to reordering based on logical timestamps using vector time (clocks), our approach reduces the size of the trace information collected by more than half while incurring similar runtime overhead.

The heterogeneity and complexity of Internet of Things (IoT) applications present new challenges to the software development process. Model-Driven Software Development (MDSD) is increasingly being recognized as a key paradigm in tackling many of these challenges, as evident by the emergence of a significant number of MDSD frameworks targeting IoT in the past couple of years. At the heart of IoT applications are embedded and realtime systems, a domain where model-driven development is well-established and many existing tools have a proven track record. Unfortunately, only a handful of these tools support out-of-the-box integration with the IoT. In this work, we discuss the different design and implementation decisions for enabling existing actor-oriented MDSD tools for the IoT. Moreover, we propose an integration approach based on the use of proxy actors and system interfaces. The approach offers seamless and flexible integration of external IoT devices into the user's model. We implement and evaluate our approach using the MDSD tool Papyrus for Realtime as a testbed.

Model Driven Engineering (MDE) techniques raise the level of abstraction at which developers construct software. However, modern cyber-physical systems are becoming more prevalent and complex and hence software models that represent the structure and behavior of such systems still tend to be large and complex. These models may have numerous if not infinite possible behaviors, with complex communications between their components. Appropriate software testing techniques to generate test cases with high coverage rate to put these systems to test at the model-level (without the need to understand the underlying code generator or refer to the generated code) are therefore important. Concolic testing, a hybrid testing technique that benefits from both concrete and symbolic execution, gains a high execution coverage and is used extensively in the industry for program testing but not for software models. In this paper, we present a novel technique and its tool mCUTE1, an open source 2 model-level concolic testing engine. We describe the implementation of our tool in the context of Papyrus-RT, an open source Model Driven Engineering (MDE) tool based on UML-RT, and report the results of validating our tool using a set of benchmark models.

This paper presents PMExec, a tool that supports the execution of partial UML-RT models. To this end, the tool implements the following steps: static analysis, automatic refinement, and input-driven execution. The static analysis that respects the execution semantics of UML-RT models is used to detect problematic model elements, i.e., elements that cause problems during execution due to the partiality. Then, the models are refined automatically using model transformation techniques, which mostly add decision points where missing information can be supplied. Third, the refined models are executed, and when the execution reaches the decision points, input required to continue the execution is obtained either interactively or from a script that captures how to deal with partial elements. We have evaluated PMExec using several use-cases that show that the static analysis, refinement, and application of user input can be carried out with reasonable performance, and that the overhead of approach is manageable. https://youtu.be/BRKsselcMnc Note: Interested readers can refer to [1] for a thorough discussion and evaluation of this work.

Data science pipelines are a sequence of data processing steps that aim to derive knowledge and insights from raw data. Data science pipeline tools simplify the creation and automation of data science pipelines by providing reusable building blocks that users can drag and drop into their pipelines. Such a graphical, model-driven approach enables users with limited data science expertise to create complex pipelines. However, recent studies show that there exist several data science pitfalls that can yield spurious results and, consequently, misleading insights. Yet, none of the popular pipeline tools have built-in quality control measures to detect these pitfalls. Therefore, in this paper, we propose an approach called Pitfalls Analyzer to detect common pitfalls in data science pipelines. As a proof-of-concept, we implemented a prototype of the Pitfalls Analyzer for KNIME, which is one of the most popular data science pipeline tools. Our prototype is model-driven, since the detection of pitfalls is accomplished using pipelines that were created with KNIME building blocks. To showcase the effectiveness of our approach, we run our prototype on 11 pipelines that were created by KNIME experts for 3 Internet-of-Things (IoT) projects. The results indicate that our prototype flags all and only those instances of the pitfalls that we were able to flag while manually inspecting the pipelines.

Testing models of modern cyber-physical systems is not straightforward due to timing constraints, numerous if not infinite possible behaviors, and complex communications between components. Software testing tools and approaches that can generate test cases to test these systems are therefore important. Many of the existing automatic approaches support testing at the implementation level only. The existing model-level testing tools either treat the model as a black box (e.g., random testing approaches) or have limitations when it comes to generating complex test sequences (e.g., symbolic execution). This paper presents a novel approach and tool support for automatic unit testing of models of real-time embedded systems by conducting concolic testing, a hybrid testing technique based on concrete and symbolic execution. Our technique conducts automatic concolic testing in two phases. In the first phase, model is isolated from its environment, is transformed to a testable model and is integrated with a test harness. In the second phase, the harness tests the model concolically and reports the test execution results. We describe an implementation of our approach in the context of Papyrus-RT, an open source Model Driven Engineering (MDE) tool based on the modeling language UML-RT, and report the results of applying our concolic testing approach to a set of standard benchmark models to validate our approach.

The Linux kernel provides its services to the application layer using so-called system calls. All system calls combined form the Application Programming Interface (API) of the kernel. Hence, system calls provide us with a window into the development process and design decisions that are made for the Linux kernel. Our paper [1] presents the result of an empirical study of the changes (8,770) that were made to the system calls during the last decade (i.e., from April 2005 to December 2014). The main contributions and most important findings of our study are:

Models of Real-time Embedded (RTE) systems may encompass many components with often many different kinds of dependencies describing, e.g., structural relationships or the flow of data, control, or messages. Understanding and properly accounting for them during development, testing and debugging can be challenging. This paper presents an approach for slicing models of RTE systems to facilitate model understanding and other model-level activities. A key novelty of our approach is the support for models with composite components (with multiple hierarchical levels) and capturing the dependencies that involve structural and behavioural model elements, including a combination of the two. Moreover, we describe an implementation of our approach in the context of Papyrus-RT, an open source Model Driven Engineering (MDE) tool based on the modeling language UML-RT. We conclude the paper with the results of applying our slicer to a set of UML-RT models to validate our approach and to demonstrate the applications of our approach for facilitating model-level analysis tasks, such as testing and debugging.

This paper is an evaluation of UML for Real-Time (UML-RT) for modelling Self-Adaptive Software (SAS) systems. Using a systematic review of the different features of UML-RT (optional capsules, SAP/SPP communication, hierarchical state machines, etc.), we analyse the suitability of the language for modelling structural and behavioural adaptations at design-and run-time. We evaluate these features in the context of their current state of support in Papyrus-RT, an Eclipse-based MDE tool for UML-RT recently developed by the Eclipse PolarSys Working Group. The use of UML-RT and Eclipse Papyrus for Real-Time (Papyrus-RT) for different kinds of adaptation is demonstrated using two real-time system case studies.

Studies of code repositories have long been used to understand the use of programming languages and to provide insight into how they should evolve. Such studies can highlight features that are rarely used and can safely be removed to simplify the language. Conversely, combinations of features that are frequently used together can be identified and possibly replaced with new features to improve the user experience. Unfortunately, this kind of research has not been as popular in Model Driven Development (MDD). More specifically, using repositories of model transformations (in any language) to understand how the features of these languages are used has not been investigated much, despite its potential benefits. In this paper, we study the use of the ATL model transformation language in an ATL transformation repository. We identify three research questions aimed at providing insight into how ATL's features are actually used. Using the TXL source transformation language, we implement a parser-based analyzer to extract information from the ATL Zoo. We use this information to answer these research questions and provide additional observations based on manual inspection of ATL artifacts.

Providing proper support for debugging models at model-level is one of the main barriers to a broader adoption of Model Driven Development (MDD). In this paper, we focus on the use of MDD for the development of real-time embedded systems (RTE). We introduce a new platform-independent approach to implement model-level debuggers. We describe how to realize support for model-level debugging entirely in terms of the modeling language and show how to implement this support in terms of a model-to-model transformation. Key advantages of the approach over existing work are that (1) it does not require a program debugger for the code generated from the model, and that (2) any changes to, e.g., the code generator, the target language, or the hardware platform leave the debugger completely unaffected. We also describe an implementation of the approach in the context of Papyrus-RT, an open source MDD tool based on the modeling language UML-RT. We summarize the results of the use of our model-based debugger on several use cases to determine its overhead in terms of size and performance. Despite being a prototype, the performance overhead is in the order of microseconds, while the size overhead is comparable with that of GDB, the GNU Debugger.

An ontology is typically defined in terms of some vocabulary that is defined using other ontologies. When the vocabulary changes, it is important for a dependent ontology to evolve in a consistent manner. Automating the migration of ontologies based on changes to their vocabulary is requisite to proper adoption of ontologies for varying fields of use. Oital is a transformation language capable of automatically migrating dependent ontologies. It helps make ontologies easier to maintain as they evolve.

The relative ease of test case generation associated with model-based testing can lead to an increased number of test cases being identified for any given system; this is problematic as it is becoming near impossible to run (or even generate) all of the possible tests in available time frames. Test case prioritization is a method of ranking the tests in order of importance, or priority based on criteria specific to a domain or implementation, and selecting some subset of tests to generate and run. Some approaches require the generation of all tests, and simply prioritize the ones to be run, however we propose an approach that would prevent unnecessary generation of tests through the use of symbolic execution trees to determine which tests provide the most benefit to coverage of execution. Our approach makes use of fuzzy logic, specifically fuzzy control systems, to prioritize test cases generated from these execution; the prioritization is based on natural language rules about testing priority. Within this paper we present our motivation, some background research, our methodology and implementation, results, and conclusions.

Software development guidelines are a set of rules which can help improve the quality of software. These rules are defined on the basis of experience gained by the software development community over time. Software antipatterns are a powerful and effective form of guidelines used for the identification of bad design choices and development practices that often lead to poor-quality software. This paper introduces a set of seven state machine antipatterns for the model-based development of real time embedded software systems. Each of these antipatterns is described with a pair of examples: one for the antipattern itself and a second one for improved, refactored solution.

This paper introduces two complementary techniques, memoization-based and dependency-based incremental symbolic execution, that aim to optimize the analysis of state machine models that undergo change. We implement the two proposed techniques on IBM Rhapsody Statecharts and present some evaluation results.

To facilitate modular development, the use of state machines has been proposed to specify the protocol (i.e., the sequence of messages) that each port of a component can engage in. The protocol conformance checking problem consists of determining whether the actual behavior of a component conforms to the protocol specifications on its ports. In this paper, we consider this problem in the context of the model-driven development (MDD) of embedded systems based on UML 2, in which UML 2 state machines are used to specify component behavior. We provide a definition of conformance which slightly extends those found in the literature and reduce the conformance check to a state space exploration. We describe a tool implementing the approach using the Java PathFinder software model checker and the MDD tool IBM Rational RoseRT, discuss its application to three case studies, and show how the tool repeatedly allowed us to find unexpected conformance errors with encouraging performance. We conclude that the approach is promising for supporting the modular development of embedded components in the context of industrial applications of MDD.

In Model Driven Engineering (MDE), models are first-class citizens, and model transformation is MDE's "heart and soul". Since model transformations are executed for a family of conforming models, their validity becomes a crucial issue. This paper proposes to explore the question of the formal verification of model transformation properties through a tri-dimensional approach: the transformation involved, the properties of interest addressed, and the formal verification techniques used to establish the properties. This work allows a better understanding of the expected properties for a particular transformation, and facilitates the identification of the suitable tools and techniques for enabling their verification.

Model driven development (MDD) is on the rise in software engineering and no more so than in the realm of realtime and embedded systems. Being able to leverage the code generation and validation techniques made available through MDD is worth exploring, and is a large area of focus in academic and industrial research. However given the iterative nature of MDD, the evolution of models causes test case generation to occur multiple times throughout a software modeling project. Currently, the existing process of regenerating test cases for a modified model of a system can be costly, inefficient, and even redundant. Thus, it is our goal to achieve an improved understanding of the impact of typical state machine evolution steps on test cases, and how this impact can be mitigated by reusing previously generated test cases. We are also aiming to implement this in a software prototype to automate and evaluate our work.

A Mobile Agent System (MAS) is a special kind of distributed system in which the agent software can move from one physical host to another. This paper describes a new approach, together with its implementation and evaluation, for checking the conformance of a MAS with respect to an executable model. In order to check the effectiveness of our conformance check, we have built a mutation-based evaluation framework. Part of the framework is a set of 29 new mutation operators for mobile agent systems. Our conformance checking approach is used to compare the mutated agents with the executable model and determine nonconformance. Our experimental results suggest that our approach holds promise for the generation and detection of non-equivalent mutants.

Model Driven Development (MDD) is an approach to software development built around the notion of models. One of its implementation is the IBM RSA RTE, which uses the UML-RT modeling language. In this paper we introduce the tool SAUML (Symbolic Analysis of UML-RT Models) that enhances the current practice of MDD with the analyses of UML-RT models. The implemented technique is based on symbolic execution, features modularity and supports the reuse of analysis results. The paper gives an overview of this technique and its implementation in the IBM RSA RTE tool.

Kiltera is a language for modelling, analysis and simulation of time-sensitive, event-driven systems with support for (channel) mobility, introduced in. In this paper we present an updated version of the language to support distributed computation. We present the language from an informal perspective and discuss its implementation based on event-scheduling and time-warp for distributed simulation. We also present a nontrivial application to modelling load-balancing in server farms.

Model checking has for years been advertised as a way of ensuring the correctness of complex software systems. However, there exist surprisingly few critical studies of the application of model checking to industrial-scale software systems by people other than the model checker's own authors. In this paper we report our experience in applying the Spin model checker to the validation of the failover protocols of a commercial telecommunications system. While we conclude that model checking is not yet ready for such applications, we find that current research in the model checking community is working to address the difficulties we encountered.

The development of controls for the execution of concurrent code is non-trivial. We show how existing discrete-event system (DES) theory can be successfully applied to this problem. From code without concurrency controls and a specification of desired behaviours, concurrency control code is generated. By applying rigorously proven DES theory, we guarantee that the control scheme is nonblocking (and thus free of both deadlock and livelock) and minimally restrictive. Some conflicts between specifications and source can be automatically resolved without introducing new specifications. Moreover, the approach is independent of specific programming or specification languages. Two examples using Java are presented to illustrate the approach. Additional applicable DES results are discussed as future work.

Today, many formal analysis tools are not only used to provide certainty but are also used to debug software systems - a role that has traditional been reserved for testing tools. We are interested in exploring the complementary relationship as well as tradeoffs between testing and formal analysis with respect to debugging and more specifically bug detection. In this paper we present an approach to the assessment of testing and formal analysis tools using metrics to measure the quantity and efficiency of each technique at finding bugs. We also present an assessment framework that has been constructed to allow for symmetrical comparison and evaluation of tests versus properties. We are currently beginning to conduct experiments and this paper presents a discussion of possible outcomes of our proposed empirical study.

In run-time safety analysis the executions of a concurrent program are monitored and analyzed with respect to safety properties. Similar to testing, run-time analysis is quite efficient, but it also tends to be incomplete. The results pertain only to the observed executions which may constitute just a small subset of all possible executions.

We show how the state space exploration tool VeriSoft can be used to analyze parallel C/C++ programs compositionally. VeriSoft is used to check assume/guarantee specifications of parallel processes automatically. The analysis is meant to complement standard assume/guarantee reasoning which is usually carried out solely with "pencil and paper". While a successful analysis does not always imply the general correctness of the specification, it increases the confidence in the verification effort. An unsuccessful analysis always produces a counterexample which can be used to correct the specification or the program. VeriSoft's optimization and visualization techniques make the analysis relatively efficient and effective.

Model checking and other finite-state analysis techniques have been very successful when used with hardware systems and less successful with software systems. It is especially difficult to analyze software systems developed with the implicit invocation architectural style because the loose coupling of their components increases the size of the finite state model. In this paper we provide insight into the larger problem of how to make model checking a better analysis and verification tool for software systems. Specifically, we will extend an existing approach to model checking implicit invocation to allow for the modeling of larger and more realistic systems. Our focus will be on improving the representation of events, event delivery policies and event-method bindings. We also evaluate our technique on two non-trivial examples. In one of our examples, we will show how with iterative analysis a system parameter can be chosen to meet the appropriate system requirements.

Abstract. Parallel computers have not yet had the expected impact on mainstream computing. Parallelism adds a level of complexity to the programming task that makes it very error-prone. Moreover, a large variety of very different parallel architectures exists. Porting an implementation from one machine to another may require substantial changes. This paper addresses some of these problems by developing a formal basis for the design of parallel programs in the form of a refinement calculus. The calculus allows the stepwise formal derivation of an abstract, low-level implementation from a trusted, high-level specification. The calculus thus helps structuring and documenting the development process. Portability is increased, because the introduction of a machine-dependent feature can be located in the refinement tree. Development efforts above this point in the tree are independent of that feature and are thus reusable. Moreover, the discovery of new, possibly more efficient solutions is facilitated. Last but not least, programs are correct by construction, which obviates the need for difficult debugging. Our programming/specification notation supports fair parallelism, shared-variable and message-passing concurrency, local variables and channels. The calculus rests on a compositional trace semantics that treats shared-variable and message-passing concurrency uniformly. The refinement relation combines a context-sensitive notion of trace inclusion and assumption-commitment reasoning to achieve compositionality. The calculus straddles both concurrency paradigms, that is, a shared-variable program can be refined into a distributed, message-passing program and vice versa.

Abstract. Implicit invocation [SuN92, GaN91] has become an important architectural style for large-scale system design and evolution. This paper addresses the lack of specification and verification formalisms for such systems. A formal computational model for implicit invocation is presented. We develop a verification framework for implicit invocation that is based on Jones' rely/guarantee reasoning for concurrent systems [Jon83, Stø91]. The application of the framework is illustrated with several examples. The merits and limitations of the rely/guarantee paradigm in the context of implicit invocation systems are also discussed.