Abstract The Eclipse Supervisory Control Engineering Toolkit (ESCET™) is an open-source project to provide a model-based approach and toolkit for developing supervisory controllers, targeting their entire engineering process. It supports synthesis-based engineering of supervisory controllers for discrete-event systems, combining model-based engineering with computer-aided design to automatically generate correct-by-construction controllers. At its heart is supervisory controller synthesis, a formal technique for the automatic derivation of supervisory controllers from the unrestricted system behavior and system requirements. Vital for the future development of these techniques and tools is the ESCET project’s open environment, allowing industry and academia to collaborate on creating an industrial-strength toolkit. We report on some crucial developments of the toolkit in the context of research projects with Rijkswaterstaat and ASML that have considerably improved its capability to deal with the complexity of real-life systems as well as its usability.

Item does not contain fulltext

Complex embedded systems, such as robotics, automotive and high-tech manufacturing, are hard to maintain due to their complex nature. To advance our understanding of the software engineering practice for complex embedded systems, we conducted a series of empirical studies at ASML, a leading manufacturer of lithography machines for semi-conductor industry. We started with an interview study exploring how developers use execution logs, essential artifacts that capture the runtime behavior of software systems. The empirical insights obtained from this study led us to explore subtopics about model inference from logs, modeling practice and log comparison. Motivated by the observation that developers often manually sketch behavioral models based on logs, we propose a model inference technique that can extract models by combining log analysis, and analysis of a running system under stimuli. As observed in this model inference study, the transition from code to models requires developers to work with a hybrid system which consists of handwritten code and models. We then study modeling practices and the roles of model in such hybrid systems. Particularly, we study why developers violate modeling guidelines, providing implications for researchers and tool builders to support developers in modeling complex embedded systems. Another interesting observation from the interview study is that developers face challenges in comparing multiple logs generated from such systems. We therefore conduct a literature study to provide an overview of the existing techniques and identify the limitations of the existing techniques. In this project, we study logs and models in complex embedded systems, providing tool builders, researchers and practitioners with implications to facilitate log analysis, model inference, modeling practice and log comparison.

Abstract Context Models, as the main artifact in model-driven engineering, have been extensively used in the area of embedded systems for code generation and verification. One of the most popular behavioral modeling techniques is the state machine. Many state machine modeling guidelines recommend that a state machine should have more than one state in order to be meaningful. However, single-state state machines (SSSMs) violating this recommendation have been used in modeling cases reported in the literature. Objective We aim for understanding the phenomenon of using SSSMs in practice as understanding why developers violate the modeling guidelines is the first step towards improvement of modeling tools and practice. Method To study the phenomenon, we conducted an exploratory study which consists of two complementary studies. The first study investigated the prevalence and role of SSSMs in the domain of embedded systems, as well as the reasons why developers use them and their perceived advantages and disadvantages. We employed the sequential explanatory strategy, including repository mining and interview, to study 1500 state machines from 26 components at ASML, a leading company in manufacturing lithography machines from the semiconductor industry. In the second study, we investigated the evolutionary aspects of SSSMs, exploring when SSSMs are introduced to the systems and how developers modify them by mining the largest state-machine-based component from the company. Results We observe that 25 out of 26 components contain SSSMs. Our interviews suggest that SSSMs are used to interface with the existing code, to deal with tool limitations, to facilitate maintenance and to ease verification. Our study on the evolutionary aspects of SSSMs reveals that the need for SSSMs to deal with tool limitations grew continuously over the years. Moreover, only a minority of SSSMs have been changed between SSSM and multiple-state state machine (MSSM) during their evolution. The most frequent modifications developers made to SSSMs is inserting events with constraints on the execution of the events. Conclusions Based on our results, we provide implications for developers and tool builders. Furthermore, we formulate hypotheses about the effectiveness of SSSMs, the impacts of SSSMs on development, maintenance and verification as well as the evolution of SSSMs.

This article presents a modular automaton-based framework to specify flexible manufacturing systems and to optimize the makespan of product batches. The Batch Makespan Optimization (BMO) problem is NP-Hard and optimization can therefore take prohibitively long, depending on the size of the state-space induced by the specification. To tame the state-space explosion problem, we develop an algebra based on automata equivalence and inclusion relations that consider both behavior and structure. The algebra allows us to systematically relate the languages induced by the automata, their state-space sizes, and their solutions to the BMO problem. Further, we introduce a novel constraint-based approach to systematically prune the state-space based on the the notions of nonpermutation-repulsiveness and permutation-attractiveness. We prove that constraining a nonpermutation-repulsing automaton with a permutation-attracting constraint always reduces the state-space. This approach allows us to (i) compute optimal solutions of the BMO problem when the (additional) constraints are taken into account and (ii) compute bounds for the (original) BMO problem (without using the constraints). We demonstrate the effectiveness of our approach by optimizing an industrial wafer handling controller.

Models, as the main artifact in model-driven engineering, have been extensively used in the area of embedded systems for code generation and verification. One of the most popular behavioral modeling techniques is state machine. Many state machine modeling guidelines recommend that a state machine should have more than one state in order to be meaningful. However, single-state state machines (SSSMs) violating this recommendation have been used in modeling cases reported in the literature.

Abstract High-tech companies are struggling today with the maintenance of legacy software. Legacy software is vital to many organizations as it contains the important business logic. To facilitate maintenance of legacy software, a comprehensive understanding of the software’s behavior is essential. In terms of component-based software engineering, it is necessary to completely understand the behavior of components in relation to their interfaces, i.e., their interface protocols, and to preserve this behavior during the maintenance activities of the components. For this purpose, we present an approach to infer the interface protocols of software components from the behavioral models of those components, learned by a blackbox technique called active (automata) learning. To validate the learned results, we applied our approach to the software components developed with model-based engineering so that equivalence can be checked between the learned models and the reference models, ensuring the behavioral relations are preserved. Experimenting with components having reference models and performing equivalence checking builds confidence that applying active learning technique to reverse engineer legacy software components, for which no reference models are available, will also yield correct results. To apply our approach in practice, we present an automated framework for conducting active learning on a large set of components and deriving their interface protocols. Using the framework, we validated our methodology by applying active learning on 202 industrial software components, out of which, interface protocols could be successfully derived for 156 components within our given time bound of 1 h for each component.

This paper covers the Rigorous Examination of Reactive Systems (RERS) Challenge 2019. For the first time in the history of RERS, the challenge features industrial tracks where benchmark programs that participants need to analyze are synthesized from real-world models. These new tracks comprise LTL, CTL, and Reachability properties. In addition, we have further improved our benchmark generation infrastructure for parallel programs towards a full automation. RERS 2019 is part of TOOLympics, an event that hosts several popular challenges and competitions. In this paper, we highlight the newly added industrial tracks and our changes in response to the discussions at and results of the last RERS Challenge in Cyprus.

Inferring behavioral models (e.g., state machines) of software systems is an important element of re-engineering activities. Model inference techniques can be categorized as active or passive learning, constructing models by (dynamically) interacting with systems or (statically) analyzing traces, respectively. Application of those techniques in the industry is, however, hindered by the trade-off between learning time and completeness achieved (active learning) or by incomplete input logs (passive learning). We investigate the learning time/completeness achieved trade-off of active learning with a pilot study at ASML, provider of lithography systems for the semiconductor industry. To resolve the trade-off we advocate extending active learning with execution logs and passive learning results. We apply the extended approach to eighteen components used in ASML TWINSCAN lithography machines. Compared to traditional active learning, our approach significantly reduces the active learning time. Moreover, it is capable of learning the behavior missed by the traditional active learning approach.

We develop a model-based approach to predict timing of service-based software applications on Linux-based multi-core platforms for alternative mappings (affinity and priority settings). Service-based applications consist of communicating sequential (Linux) processes. These processes execute functions (also called services), but can only execute them one at a time. Models are inferred automatically from execution traces to enable timing optimization of existing (legacy) systems. Our approach relies on a linear progress approximation of functions. We compute the expected share of each function based on the mapping (affinity and priority) parameters and the functions that are currently active. We validate our models by carrying out a controlled lab experiment consisting of a multi-process pipelined application mapped in different ways on a quadcore Intel i7 processor. A broad class of affinity and priority settings is fundamentally unpredictable due to Linux binding policies. We show that predictability can be achieved if the platform is partitioned in disjoint clusters of cores such that i) each process is bound to such a cluster, ii) processes with non real-time priorities are bound to singleton clusters, and iii) all processes bound to a non-singleton cluster have different real-time priorities. For mappings using singleton clusters with niceness priorities only, our model predicts execution latencies (for each pipeline iteration) with errors less than 5% relative to the measured execution times. For mappings using a non-singleton cluster (with different real-time priorities) relative errors of less than 2% are obtained. When real-time and niceness priorities are mixed, we predict with errors of 7%.

Over the last decades, the complexity of high-tech systems, and the software systems controlling them, has increased considerably. In practice, it is hard to keep knowledge and documentation of these ever-evolving software systems up-to-date with their actual realization; we are dealing with legacy software. Clearly, this lack of knowledge, insight, and understanding is more and more becoming a critical issue. Process mining provides an interesting opportunity to improve understanding and analyze software behavior based on observations from the system on the run. However, a concrete software process analysis methodology was lacking. This paper 1) discusses a software process analysis case study at ASML, a large high-tech company, and, based on the lessons learned, 2) presents a concrete methodology for analyzing software processes. The presented methodology actively includes the system under analysis and is based on practical experiences in applying process mining on industrial-scale legacy software.

A recurring theme in discussions about the adoption of Model-Based Engineering (MBE) is its effectiveness. This is because there is a lack of empirical assessment of the processes and (tool-)use of MBE in practice. We conducted a multiple-case study by observing 2 two-month MBE projects from which software for a Mars rover were developed. We focused on assessing the distribution of the total software development effort over different development activities. Moreover, we observed and collected challenges reported by the developers during the execution of projects. We found that the majority of the effort is spent on the collaboration and communication activities. Furthermore, our inquiry into challenges showed that tool-related challenges are the most encountered.

Model-driven engineering is used in the design of systems to (a.o.) enable analysis early in the design process. For instance, by using domain-specific languages, enabling engineers to model systems in terms of their domain, rather then encoding them into general purpose modeling languages. Domain-specific languages, like classical software, evolve over time. When domain languages evolve, they may trigger co-evolution of models, model-to-model transformations, editors (both graphical and textual), and other artifacts that depend on the domain-specific language. This co-evolution can be tedious and very costly. In literature, various approaches are proposed towards automated co-evolution. However, these approaches do not reach full automation. Several other studies have shown that there are theoretical limitations to the level of automation that can be achieved in certain scenarios. For several scenarios full automation can never be achieved. We wish to gain insight to which extent practically occurring scenarios can be automated. To gain this insight, in this paper, we investigate on a large-scale industrial repository, which (co-)evolutionary scenarios occur in practice, and compare them with the various scenarios and their theoretical automatability. We then assess whether practically occurring scenarios can be fully automated.

Developing complex (sub)systems is a multi-disciplinary activity resulting in several, complementary models, possibly on different abstraction levels. The relations between all these models are usually loosely defined in terms of informal documents. It is not uncommon that only till the moment of integration at implementation level, shortcomings or misunderstanding between the different disciplines is revealed. In order to keep models consistent and to reason about multiple models, the relations between models have to be formalized. MultiDisciplinary System Engineering (MDSE) ecosystems provide a means for this. These ecosystems formalize the domain of interest using Domain Specific Languages (DSLs), and formalize the relations between models by means of automated model transformations. This enables consistency checking between domain and aspect models and facilitates multi-disciplinary analysis of the single (sub)system at hand. MDSE ecosystems provide the means to analyze a single (sub)system model. A set of models of different (sub)systems can be analyzed to derive best modeling practices and modeling patterns, and to measure whether a MDSE ecosystem fulfills its needs. The MDSE ecosystem itself can be instrumented to analyze how the MDSE ecosystem is used in practice. The evolution of models, DSLs and complete MDSE ecosystems is studied to identify and develop means that support evolution at minimal costs while maintaining high quality. In this paper, we present the anatomy of MDSE ecosystems with industrial examples, the ongoing work to enable the various types of analysis, each with their dedicated purpose. We conclude with a number of future research directions.

In this paper we address the makespan optimization of industrial-sized manufacturing systems. We introduce a framework which specifies functional system requirements in a compositional way and automatically computes makespan optimal solutions respecting these requirements. We show the optimization problem to be NP-Hard. To scale towards systems of industrial complexity, we propose a novel approach based on a subclass of compositional requirements which we call constraints. We prove that these constraints always prune the worst-case optimization-space thereby increasing the odds of finding an optimal solution (with respect to the additional constraints). We demonstrate the applicability of the framework on an industrial-sized manufacturing system.

System design is a difficult process with many design-choices for which the impact may be difficult to foresee. Manufacturing system design is no exception to this. Increased use of flexible manufacturing systems which are able to perform different operations/use-cases further raises the design complexity. One important criterion to consider is the overall makespan and associated critical path for the different use-cases of the system. Stochastic critical path analysis plays a fundamental role in providing useful feedback for system designers to evaluate alternative specifications, which traditional fixed-time analysis cannot. In this paper, we extend our formal model-based framework, for the specification and design of manufacturing systems, with stochastic analysis abilities by associating a criticality index to each action performed by the system. This index can then be visualized and used within the framework such that a system designer can make better informed decisions. We propose a Monte-Carlo method as an estimation algorithm and we explicitly define and use confidence intervals to achieve an acceptable estimation error. We further demonstrate the use of the extended framework and stochastic analysis with an example manufacturing system.

Over the past years, there has been an increase in the application of model driven engineering in industry. Similar to traditional software engineering, understanding how technologies are actually used in practice is essential for developing good tooling, and decision making processes. Unfortunately, obtaining and analyzing empirical data in a model-driven context is still tedious and time consuming, introducing large lead-times. In this paper we present a framework for the automated extraction, analysis, and visualization of data and metrics on model-driven artifacts. We subsequently present various examples of how the framework was successfully applied in a large industrial setting to answer a plethora of different questions with respect to decision making and tool development.

This paper introduces a formal modeling approach for compositional specification of both functionality and timing of manufacturing systems. Functionality aspects can be considered orthogonally to timing aspects. The functional aspects are specified using two abstraction levels; high-level activities and lower level actions. Design of a functionally correct controller is possible by looking only at the activity level, abstracting from the different execution orders of actions and their timing. As a result, controller design can be performed on a much smaller state space compared to an explicit model where timing and actions are present. The performance of the controller can be analyzed and optimized by taking into account the timing characteristics. Since formal semantics are given in terms of a (max, +) state space, various existing performance analysis techniques can be used. We illustrate the approach, including performance analysis, on an example manufacturing system.

Domain specific languages (DSLs) ease the adoption of formal specification in industry. They allow developers to describe their specification models in concepts of their domain. However, DSLs evolve over time, causing specification models to have to co-evolve to reflect the evolution in the DSL. The maintenance overhead introduced by these, often manual, changes to specification models threatens to overshadow the advantages of DSL usage in industry. To this extent, many approaches have been proposed in the literature to facilitate DSL maintenance by automating model co-changes. In this paper, we evaluate the ability of a tool, Edapt, to support the change and co-change in twenty-two industrial DSLs and corresponding specification models over a maintenance period of four years. We observe that the tool is only able to automatically co-change specification models for 72% of the DSL changes. To address the remaining 28% of the changes, we extend Edapt. The resulting extension allows automatically co-changing specification models for 98% of the DSL changes.

Domain-specific languages (DSLs) allow users to model systems using concepts from a specific domain. Evolution of DSLs triggers co-evolution of models developed in these languages. Manual co-evolution of the thousands of models is unfeasible, calling for an automated support. A prerequisite to automating model co-evolution with respect to DSL evolution is the ability to formally specify DSL evolution, e.g., using predefined evolution operators. Success or failure of the practical application of the operator-based approach therefore depends heavily on the operators offered by the operator library at hand. In this paper we evaluate the completeness of the state-of-the-art operator library claimed to be "practically complete" (which we denote as H) by using it to specify evolution of an ecosystem of 22 commercial DSLs over the period of four years. We observe that 11% of the changes cannot be specified. However, there is no guarantee that extending the library with the identified deficiencies will be sufficient to specify evolution of other DSLs. To mitigate this, we design a theoretically complete library of operators, R. We observe that 77% of the operators from R are absent from H. Of the deficiencies in H, 72% could not be revealed by means of studying the extensive industrial ecosystem above. Our study suggests that the existing operator libraries are not extensive enough to specify evolution of large model-driven software ecosystems. Since extending operator libraries on a per-case study basis does not yield satisfactory results so far, we advocate an alternative, i.e. a theoretically complete library of operators R.

We investigate quality improvement in QVT operational mappings (QVTo) model transformations, one of the languages defined in the OMG standard on model-to-model transformations. Two research questions are addressed. First, how can we assess quality of QVTo model transformations? Second, how can we develop higher-quality QVTo transformations? To address the first question, we utilize a bottom–up approach, starting with a broad exploratory study including QVTo expert interviews, a review of existing material, and introspection. We then formalize QVTo transformation quality into a QVTo quality model. The quality model is validated through a survey of a broader group of QVTo developers. We find that although many quality properties recognized as important for QVTo do have counterparts in general purpose languages, a number of them are specific to QVTo or model transformation languages. To address the second research question, we leverage the quality model to identify developer support tooling for QVTo. We then implemented and evaluated one of the tools, namely a code test coverage tool. In designing the tool, code coverage criteria for QVTo model transformations are also identified. The primary contributions of this paper are a QVTo quality model relevant to QVTo practitioners and an open-source code coverage tool already usable by QVTo transformation developers. Secondary contributions are a bottom–up approach to building a quality model, a validation approach leveraging developer perceptions to evaluate quality properties, code test coverage criteria for QVTo, and numerous directions for future research and tooling related to QVTo quality.

This paper proposes a modeling approach to capture the mapping of an application on a platform. The approach is based on Scenario-Aware Dataflow (SADF) models. In contrast to the related work, we express the complete design-space in a single formal SADF model. This allows us to have a compact and explorable state-space linked with an executable model capable of symbolically analyzing different mappings for their timing behavior. We can model different bindings for application tasks, different static-orders schedules for tasks bound in shared resources, as well as naturally capturing resource claiming/unclaiming using SADF semantics. Moreover, by using the inherent properties of dataflow graphs and the dynamic behavior of a Finite-State Machine, we can model different levels of pipelining, such as full application pipelining and interleaved pipelining of consecutive executions of the application. The size of the model is independent of the number of executions of the application. Since we are able to capture all this behavior in a single SADF model we can use available dataflow analysis, such as worst-case and best-case throughput and deadlock-freedom checking. Furthermore, since the model captures the design-space independently of the analysis technique, one can use different exploration approaches to analyze different sets of requirements.

Development of high-level supervisory controllers is an important challenge in the design of high-tech systems. It has become a significant issue due to increased complexity, combined with demands for verified quality, time to market, ease of development, and integration of new functionality. To deal with these challenges, model-based engineering approaches are suggested as a cost-effective way to support easy adaptation, validation, synthesis, and verification of controllers. This paper presents an industrial case study on modular design of a supervisory controller for wafer logistics in lithography machines. The uncontrolled system and control requirements are modeled independently in a modular way, using small, loosely coupled and minimally restrictive extended finite automata. The multiparty synchronization mechanism that is part of the specification formalism provides clear advantages in terms of modularity, traceability, and adaptability of the model. We show that being able to refer to variables and states of automata in guard expressions and state-based requirements, enabled by the use of extended finite automata, provides concise models. Additionally, we show how modular synthesis allows construction of local supervisors that ensure safety of parts of the system, since monolithic synthesis is not feasible for our industrial case.

Analyzing worst-case application timing for systems with shared resources is difficult, especially when non-monotonic arbitration policies like First-Come-First-Served (FCFS) scheduling are used in combination with varying task execution times. Analysis methods that conservatively analyze these systems are often based on state-space exploration, which is not scalable due to its inherent susceptibility to combinatorial explosion. We propose a scalable timing analysis method on periodically restarted Directed Acyclic Task Graphs, that can provide conservative bounds on task timing properties when shared resources with FCFS scheduling are used. By expressing task enabling and completion times in intervals, denoting best-case and worst-case timing properties, contention on the shared resources can be estimated using conservative approximations. With an industrial case study we show that our approach can easily analyze models with thousands of tasks in less than 10 seconds, and the worst-case bounds obtained show an average improvement of 46% compared to bounds obtained by static worst-case analysis.

In this paper, we present a case-study on application of Ramadge-Wonham supervisory control theory (abbreviated by SCT in the sequel) to a patient support system of a magnetic resonance imaging (MRI) scanner. We discuss the whole developmental cycle, starting from the mathematical models of the uncontrolled system and of the control requirements, and ending with the implementation of the obtained controller on the actual hardware. The obtained controller was tested on the physical system. In this case study, we attempted to build the models in a modular way, in order to decrease the computational complexity of the controller synthesis and to improve the adaptability of the models. An important advantage of SCT is that it allows automatic generation of the controller, and that it can thus improve adaptability of the control software. We also briefly discuss our experience on the adaptability of the control software, obtained in the course of this case study.

Latest trends in embedded platform architectures show a steady shift from high frequency single core platforms to lower-frequency but highly-parallel execution platforms. Scheduling applications with stringent latency requirements on such multiprocessor platforms is challenging. Our work is motivated by the scheduling challenges faced by ASML, the world's leading provider of wafer scanners. A wafer scanner is a complex cyber-physical system that manipulates silicon wafers with extreme accuracy at high throughput. Typical control applications of the wafer scanner consist of thousands of precedence-constrained tasks with latency requirements. Machines are customized so that precise characteristics of the control applications to be scheduled and the execution platform are only known during machine start-up. This results in large-scale scheduling problems that need to be solved during start-up of the machine under a strict timing constraint on the schedule delivery time. This paper introduces a fast and scalable static-order scheduling approach for applications with stringent latency requirements and a fixed binding on multiprocessor platforms. It uses a heuristic that makes scheduling decisions based on a new metric to find feasible schedules that meet timing requirements as quickly as possible and it is shown to be scalable to very large task graphs. The computation of this metric exploits the binding information of the application. The approach will be incorporated into the ASML's latest generation of wafer scanners.