Automation engineering in process industries involves mapping IO list specifications for sensor and actuator signals to control function blocks from vendor libraries. Due to the non-standardized format and content of IO lists, it is challenging to automate this mapping as it involves interpreting unstructured data (e.g., signal descriptions). Previous approaches proposed mapping of IO list contents to common data schemas, such as ontologies or UML models, but did not provide means to automate the initial data import from heterogeneous input formats. We propose IO-AutoMapper, an LLM-supported method to automatically process spreadsheet IO lists and map their entries to control library function blocks. Experiments with five IO lists with 300 signals each showed more than 98% correct mappings and low LLM processing costs. While LLM hallucinations could not yet be fully eliminated, the method has the potential to reduce human labor for mapping IO list entries by more than 90%.

Testing PLC and DCS control logic in industrial automation is laborious and challenging since appropriate test cases are often complex and difficult to formulate. Researchers have previously proposed several automated test case generation approaches for PLC software applying symbolic execution and search-based techniques. Often requiring formal specifications and performing a mechanical analysis of programs, these approaches may uncover specific programming errors but some-times suffer from state space explosion and cannot process rather informal specifications. We proposed a novel approach for the automatic generation of PLC test cases that queries a Large Language Model (LLM) to synthesize test cases for code provided in a prompt. Experiments with ten open-source function blocks from the OSCAT automation library showed that the approach is fast, easy to use, and can yield test cases with high statement coverage for low-to-medium complex programs. However, we also found that LLM-generated test cases suffer from erroneous assertions in many cases, which still require manual adaption.

The IEEE International Conference on Software Architecture (ICSA) is the premier gathering of practitioners and researchers interested in software architecture, component-based software engineering, and quality aspects of complex software systems. The 21st IEEE International Conference on Software Architecture (ICSA 2024) continued the tradition of a working conference, where attendees met and where software architects were able to explain the challenges they face and try to influence the future of the field. Interactive working sessions were the place where researchers met practitioners to identify opportunities to shape the future of our field.

Large language models (LLMs) providing generative AI have become popular to support software engineers in creating, summarizing, optimizing, and documenting source code. It is still unknown how LLMs can support control engineers using typical control programming languages in programming tasks. Researchers have explored GitHub CoPilot or DeepMind AlphaCode for source code generation but did not yet tackle control logic programming. A key contribution of this paper is an exploratory study, for which we created 100 LLM prompts in 10 representative categories to analyze control logic generation for of PLCs and DCS from natural language. We tested the prompts by generating answers with ChatGPT using the GPT-4 LLM. It generated syntactically correct IEC 61131-3 Structured Text code in many cases and demonstrated useful reasoning skills that could boost control engineer productivity. Our prompt collection is the basis for a more formal LLM benchmark to test and compare such models for control logic generation.

With containers becoming a prevalent method of software deployment, there is an increasing interest to use container orchestration frameworks not only in data centers, but also on resource-constrained hardware, such as Internet-of-Things devices, Edge gateways, or developer workstations. Consequently, software vendors have released several lightweight Kubernetes (K8s) distributions for container orchestration in the last few years, but it remains difficult for software developers to select an appropriate solution. Existing studies on lightweight K8s distribution performance tested only small workloads, showed inconclusive results, and did not cover recently released distributions. The contribution of this paper is a comparison of MicroK8s, k3s, k0s, and MicroShift, investigating their minimal resource usage as well as control plane and data plane performance in stress scenarios. While k3s and k0s showed by a small amount the highest control plane throughput and MicroShift showed the highest data plane throughput, usability, security, and maintainability are additional factors that drive the decision for an appropriate distribution.

Process plant operators use graphical user interfaces to supervise complex processes with tens of thousands of instruments. Such systems may require more than 1000 operator screens, but an operator can only view 3-4 screens simultaneously ("keyhole-effect"), thus prolonging root cause analysis and emergency reactions. This effect contributes to the billions of USD lost in the process industry every year due to unscheduled downtimes. Researchers have proposed alternative, zoomable 2D user interfaces (“HawkEye HMIs”) similar to Google Maps, to address the keyhole-effect, but manual efforts to create these HMIs proved to be cost-prohibitive. This paper proposes a novel method for HawkEye-HMI-Generation based on semi-automated page layouting and automated level-of-detail zooming, which can significantly lower graphics engineering costs. We tested a prototypical implementation on two cases from large production plants with up to 90 process graphics and more than 5000 graphical nodes, demonstrating high scalability, good usability, and a more than 70 percent cost savings potential.

Process engineers design industrial process plants using piping and instrumentation diagrams (P&IDs). Today, data analysts who diagnose plant disturbances based on historical signal trends usually analyze these complex diagrams manually on paper, which is time-consuming and error-prone. In the last 15 years, researchers have thus proposed several approaches and tools to turn these diagrams into machine-readable models that can be processed by software tools. Yet, these tools lack sophisticated query interfaces and intuitive visualizations. We propose the method TOPNAV to navigate plant topology models and aid data analytics. The method supports systematic searching for elements and paths in topology models and feeding the results into analytical tools to facilitate statistical analyses. In a user study, an up to 90% time reduction was observed compared to manual P&ID analysis, while reducing errors significantly.

Many Internet-of-Things (IoT) applications for smart homes, connected factories, or car-to-car communication utilize broker-based publish/subscribe communication protocols, such as the MQTT protocol. Commercial IoT applications have high reliability requirements for messaging, as lost messages due to unstable Internet connections or node failures can harm devices or even human beings. MQTT brokers implement numerous architectural availability tactics, but former analyses of MQTT communication have mainly focused on performance measurements under stable conditions. We have created the MAYHEM resilience testing tool for MQTT brokers and applied it in various resilience experiments on different MQTT brokers (VerneMQ, Mosquitto, HiveMQ, EMQ X). We found that MQTT QoS level 0 is already robust against minor packet loss, that selected broker message persistency solutions can lead to lost messages, and that most clustered MQTT brokers favor availability and performance over communication integrity. The results can support IoT practitioners in architectural decisions and researchers as well as broker vendors in optimizing designs and implementations.

Summary Distributed control systems are currently evolving towards industrial Internet of Things (IoT) systems communicating fully using Internet protocols. This creates opportunities for streamlining costly commissioning processes, which today require substantial manual work for installing, configuring, and integrating thousands of actuators and sensors. The vision of “plug‐and‐produce” control systems has been pursued for more than 15 years, but existing approaches fell short regarding configuration tasks and vendor neutrality. This paper introduces the standards‐based IoT reference architecture OpenPnP, which allows largely automating the configuration and integration tasks of industrial commissioning processes. The architecture includes a number of design and technology decisions and the required implementation can be scaled down to resource‐constrained industrial devices. This paper demonstrates how OpenPnP can reduce configuration and integration efforts up to 90% in typical settings, while potentially scaling well up to tens of thousands of communicated signals. Practitioners can orient their implementations towards OpenPnP, therefore potentially enabling “plug‐and‐produce” in many thousands of control systems.

The concepts of decision forces and the decision forces viewpoint were proposed to help software architects to make architectural decisions more transparent and the documentation of their rationales more explicit. However, practical experience reports and guidelines on how to use the viewpoint in typical industrial project setups are not available. Existing works mainly focus on basic tool support for the documentation of the viewpoint or show how forces can be used as part of focused architecture review sessions. With this paper, we share experiences and lessons learned from applying the decision forces viewpoint in a distributed industrial project setup, which involves consultants supporting architects during the re-design process of an existing large software system. Alongside our findings, we describe new forces that can serve as template for similar projects, discuss challenges applying them in a distributed consultancy project, and share ideas for potential extensions.

The OPC UA communication architecture is currently becoming an integral part of industrial automation systems, which control complex production processes, such as electric power generation or paper production. With a recently released extension for pub/sub communication, OPC UA can now also support fast cyclic control applications, but the bottlenecks of OPC UA implementations and their scalability on resource-constrained industrial devices are not yet well understood. Former OPC UA performance evaluations mainly concerned client/server round-trip times or focused on jitter, but did not explore resource bottlenecks or create predictive performance models. We have carried out extensive performance measurements with OPC UA client/server and pub/sub communication and created a CPU utilization prediction model based on linear regression that can be used to size hardware environments. We found that the server CPU is the main bottleneck for OPC UA pub/sub communication, but allows a throughput of up to 40,000 signals per second on a Raspberry Pi Zero. We also found that the client/server session management overhead can severely impact performance, if more than 20 clients access a single server.

Distributed control systems are currently evolving towards Industrial Internet-of-Things (IIoT) systems. However, they still suffer from complex commissioning processes that incur high costs. Researchers have proposed several so-called "Plug and Produce" (PnP) approaches, where commissioning shall be largely automated, but they have suffered from semantic ambiguities and usually rely on proprietary information models. We propose a novel reference architecture for PnP in IIoT systems, which is based on OPC UA and PLCopen standards and can reduce industrial device commissioning times across vendor products to a few seconds. Our proof-of-concept implementation can handle more than 500 signals per millisecond during runtime, sufficient for most application scenarios.

Industrie 4.0 (I4.0) aims at a manufacturer-independent, vertical- and horizontal-oriented communication and cooperation within smart factories. This is only manageable using international standards. The so-called reference architecture model for Industrie 4.0 (RAMI4.0) and the requirement specification of an I4.0 component are already available. RAMI4.0 could be the basis for the interoperability of the interactions. The working group “Semantic and interaction model for I4.0 components” (GMA 7.20) made progress in this direction. The language used for the interaction of I4.0 components needs model definitions for the interaction consisting of the structure of the components, syntax for the description of the messages and the means to assign the meaning to the language elements. This paper discusses the results for a broader audience.

Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of "what" versus "how" enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

Software engineers and architects of complex software systems need to balance hard quality attribute requirements while at the same time manage risks and make decisions with a system-wide and long-lasting impact. To achieve these tasks efficiently, they need quantitative information about design-time and run-time system aspects through usable and quick tools. While there is body of work focusing on code quality and metrics, their applicability at the design and architecture level and at scale are inconsistent and not proven. We are interested in exploring whether architecture can assist with better contextualizing existing system and code quality and metrics approaches. Furthermore, we ask whether we need additional architecture-level metrics to make progress and whether something as complex and subtle as software architecture can be quantified. The goal of this workshop is to discuss progress, gather empirical evidence, and identify priorities for a research agenda on architecture and metrics in the software engineering field.

Architectural Knowledge Management (AKM) has been a major topic in software architecture research since 2004. Open AKM problems include an effective, seamless transition from reusable knowledge found in patterns books and technology blogs to project-specific decision guidance and an efficient, practical approach to knowledge application and maintenance. We extended our previous work with concepts for problem space modeling, focusing on reusable knowledge, as well as solution space management, focusing on project-level decisions. We implemented these concepts in ADMentor, an extension of Sparx Enterprise Architect. AD Mentor features rapid problem space modeling, UML model linkage, question-option-criteria diagram support, meta-information for model tailoring, as well as decision backlog management. We validated ADMentor by modeling and applying 85 cloud application design decisions and 75 workflow management decisions, creating one problem and three sample solution spaces covering control system architectures, and obtaining user feedback on tool and model content.

During the last decade, researchers have proposed a number of model transformations enabling performance predictions. These transformations map performance-annotated software architecture models into stochastic models solved by analytical means or by simulation. However, so far, a detailed quantitative evaluation of the accuracy and efficiency of different transformations is missing, making it hard to select an adequate transformation for a given context. This paper provides an in-depth comparison and quantitative evaluation of representative model transformations to, e.g., queueing petri nets and layered queueing networks. The semantic gaps between typical source model abstractions and the different analysis techniques are revealed. The accuracy and efficiency of each transformation are evaluated by considering four case studies representing systems of different size and complexity. The presented results and insights gained from the evaluation help software architects and performance engineers to select the appropriate transformation for a given context, thus significantly improving the usability of model transformations for performance prediction.

Today's industrial control systems store large amounts of monitored sensor data in order to optimize industrial processes. In the last decades, architects have designed such systems mainly under the assumption that they operate in closed, plant-side IT infrastructures without horizontal scalability. Cloud technologies could be used in this context to save local IT costs and enable higher scalability, but their maturity for industrial applications with high requirements for responsiveness and robustness is not yet well understood. We propose a conceptual architecture as a basis to designing cloud-native monitoring systems. As a first step we benchmarked three open source time-series databases (OpenTSDB, KairosDB and Databus) on cloud infrastructures with up to 36 nodes with workloads from realistic industrial applications. We found that at least KairosDB fulfills our initial hypotheses concerning scalability and reliability.

In companies with a large portfolio of software or software-intensive products, functional overlaps are often perceived between independent products. In such situations it is advisable to systematically analyze the potential of systematic reuse and Software Product Lines. To this end, several domain analysis approaches, e.g., SEI Technical Probe, have been proposed to decide whether a set of products with a perceived functional overlap should be integrated into a single product line. Based on the principles of those approaches we devised our own approach. One important property is the inherent flexibility of the method to be able to apply it to four different application cases in industrial software products at ABB. In this paper we present our refined approach for domain analysis. The results and lessons learned are meant to support industrial researchers and practitioners alike. Moreover, the lessons learned highlight real-world findings concerning software reuse.

Architecture decisions are often not explicitly documented in practice but reside in the architect's mind as tacit knowledge, even though explicit capturing and documentation of architecture decisions has been associated with a multitude of benefits. As part of a research collaboration with ABB, we developed a tool to document architecture decisions. This tool is an add-in for Enterprise Architect and is an implementation of a viewpoint-based decision documentation framework. To validate the add-in, we conducted an exploratory case study with ABB architects. In the study, we assessed the status quo of architecture decision documentation, identified architects' expectations of the ideal decision documentation tool, and evaluated the new add-in. We found that although awareness of decision documentation is increasing at ABB, several barriers exist that limit the use of decisions in practice. Regarding their ideal tool, architects want a descriptive and efficient approach. Supplemental features like reporting or decision sharing are requested. The new add-in, was well-perceived by the architects. As a result of the study, we propose a clearer separation of problem, outcomes, and alternatives for the decision documentation framework.

The 2nd International Workshop on Software Engineering Challenges for the Smart Grid focuses on understanding and identifying the unique challenges and opportunities for SE to contribute to and enhance the design and development of the smart grid. In smart grids, the geographical scale, requirements on real-time performance and reliability, and diversity of application functionality all combine to produce a unique, highly demanding problem domain for SE to address. The objective of this workshop is to bring together members of the SE community and the power engineering community to understand these requirements and determine the most appropriate SE tools, methods and techniques.

Agreements for sharing of software between entities in a corporation have to be tailored to fit the situation. Such agreements are not legal documents and must address different issues than traditional software licenses. We found that these agreements should cover what is granted, payment, support, ownership and liability. In a case study we learned that an agreement should list its assumptions on the structure and processes of the software organization. The presented work enables others to create guidelines for software sharing agreements tailored to their organization and shares lessons about the differences between software product lines and corporate software sharing and reuse.

In a large corporate organization there are sometimes similar software products in certain subdomains with a perceived functional overlap. This promises to be an opportunity for systematic reuse to reduce software development and maintenance costs. In such situations companies have used different domain analysis approaches (e.g., SEI Technical Probe) that helped to assess technical and organizational potential for a software product line approach. We applied existing domain analysis approaches for software product line engineering and tailored them to include a feature analysis as well as architecture evaluation. In this paper, we report our experiences from applying the approach in two subdomains of industrial automation.

Complex information flows in the domain of industrial software systems complicate the creation of performance models to validate the challenging performance requirements. Performance models using annotated UML diagrams or mathematical notations are difficult to discuss with stakeholders from the industrial automation domain, who often have a limited software engineering background. We introduce a novel model transformation from Use Case Maps (UCM) to the Palladio Component Model (PCM), which enables performance modeling based on an intuitive notation for complex information flows. The resulting models can be solved using existing simulators or analytical solvers. We validated the correctness of the transformation with three case study models, and performed a user study. The results showed a performance prediction deviation of less than 10 percent compared to a reference model in most cases.

Economically sustainable software systems must be able to cost-effectively evolve in response to changes in their environment, their usage profile, and business demands. However, in many software development projects, sustainability is treated as an afterthought, as developers are driven by time-to-market pressure and are often not educated to apply sustainability-improving techniques. While software engineering research and practice has suggested a large amount of such techniques, a holistic overview is missing and the effectiveness of individual techniques is often not sufficiently validated. On this behalf we created a catalog of “software sustainability guidelines” to support project managers, software architects, and developers during system design, development, operation, and maintenance. This paper describes how we derived these guidelines and how we applied selected techniques from them in two industrial case studies. We report several lessons learned about sustainable software development.

Determining the trade-off between performance and costs of a distributed software system is important as it enables fulfilling performance requirements in a cost-efficient way. The large amount of design alternatives for such systems often leads software architects to select a suboptimal solution, which may either waste resources or cannot cope with future workloads. Recently, several approaches have appeared to assist software architects with this design task. In this paper, we present a case study applying one of these approaches, i.e. PerOpteryx, to explore the design space of an existing industrial distributed software system from ABB. To facilitate the design exploration, we created a highly detailed performance and cost model, which was instrumental in determining a cost-efficient architecture solution using an evolutionary algorithm. The case study demonstrates the capabilities of various modern performance modeling tools and a design space exploration tool in an industrial setting,provides lessons learned, and helps other software architects in solving similar problems.

With the increasing importance of reliability in business and industrial software systems, new techniques of architecture-based reliability engineering are becoming an integral part of the development process. These techniques can assist system architects in evaluating the reliability impact of their design decisions. Architecture-based reliability engineering is only effective if the involved reliability models reflect the interaction and usage of software components and their deployment to potentially unreliable hardware. However, existing approaches either neglect individual impact factors on reliability or hard-code them into formal models, which limits their applicability in component-based development processes. This paper introduces a reliability modeling and prediction technique that considers the relevant architectural factors of software systems by explicitly modeling the system usage profile and execution environment and automatically deriving component usage profiles. The technique offers a UML-like modeling notation whose models are automatically transformed into a formal analytical model. Our work builds upon the Palladio Component Model (PCM), employing novel techniques of information propagation and reliability assessment. We validate our technique with sensitivity analyses and simulation in two case studies. The case studies demonstrate effective support of usage profile analysis and architectural configuration ranking, together with the employment of reliability-improving architecture tactics.

Systematic decision support for architectural design decisions is a major concern for software architects of evolving service-oriented systems. In practice, architects often analyse the expected performance and reliability of design alternatives based on prototypes or former experience. Model-driven prediction methods claim to uncover the tradeoffs between different alternatives quantitatively while being more cost-effective and less error-prone. However, they often suffer from weak tool support and focus on single quality attributes. Furthermore, there is limited evidence on their effectiveness based on documented industrial case studies. Thus, we have applied a novel, model-driven prediction method called Q-ImPrESS on a large-scale process control system consisting of several million lines of code from the automation domain to evaluate its evolution scenarios. This paper reports our experiences with the method and lessons learned. Benefits of Q-ImPrESS are the good architectural decision support and comprehensive tool framework, while one drawback is the time-consuming data collection.

Performance predictions early in the software development process can help to detect problems before resources have been spent on implementation. The Palladio Component Model (PCM) is an example of a mature domain-specific modeling language for component-based systems enabling performance predictions at design time. PCM provides several alternative model solution methods based on analytical and simulation techniques. However, existing solution methods suffer from scalability issues and provide limited flexibility in trading-off between results accuracy and analysis overhead. Queueing Petri Nets (QPNs) are a general-purpose modeling formalism, at a lower level of abstraction, for which efficient and mature simulation-based solution techniques are available. This paper contributes a formal mapping from PCM to QPN models, implemented by means of an automated model-to-model transformation as part of a new PCM solution method based on simulation of QPNs. The limitations of the mapping and the accuracy and overhead of the new solution method compared to existing methods are evaluated in detail in the context of five case studies of different size and complexity. The new solution method proved to provide good accuracy with solution overhead up to 20 times lower compared to PCM's reference solver.

A multi-tenant software application is a special type of highly scalable, hosted software, in which the application and its infrastructure are shared among multiple tenants to save development and maintenance costs. The limited understanding of the underlying architectural concepts still prevents many software architects from designing such a system. Existing documentation on multi-tenant software architectures is either technology-specific or database-centric. A more technology-independent perspective is required to enable wide-spread adoption of multi-tenant architectures. We propose the SPOSAD architectural style, which describes the components, connectors, and data elements of a multi-tenant architecture as well as constraints imposed on these elements. This paper describes the benefits of a such an architecture and the trade-offs for the related design decisions. To evaluate our proposal, we illustrate how concepts of the style help to make current Platform-as-a-Service (PaaS) environments, such as Force.com, Windows Azure, and Google App Engine scalable and customizable.

Architecture-based software reliability analysis methods shall help software architects to identify critical software components and to quantify their influence on the system reliability. Although researchers have proposed more than 20 methods in this area, empirical case studies applying these methods on large-scale industrial systems are rare. The costs and benefits of these methods remain unknown. On this behalf, we have applied the Cheung method on the software architecture of an industrial control system from ABB consisting of more than 100 components organized in nine subsystems with more than three million lines of code. We used the Littlewood/Verrall model to estimate subsystems failure rates and logging data to derive subsystem transition probabilities. We constructed a discrete time Markov chain as an architectural model and conducted a sensitivity analysis. This paper summarizes our experiences and lessons learned. We found that architecture-based software reliability analysis is still difficult to apply and that more effective data collection techniques are required.