Thorough simulation testing is crucial for validating the correct behavior of small Uncrewed Aerial Systems (sUAS) across multiple scenarios, including adverse weather conditions (such as wind, and fog), diverse settings (hilly terrain, or urban areas), and varying mission profiles (surveillance, tracking). While various sUAS simulation tools exist to support developers, the entire process of creating, executing, and analyzing simulation tests remains a largely manual and cumbersome task. Developers must identify test scenarios, set up the simulation environment, integrate the System under Test (SuT) with simulation tools, formulate mission plans, and collect and analyze results. These labor-intensive tasks limit the ability of developers to conduct exhaustive testing across a wide range of scenarios. To alleviate this problem, in this paper, we propose Autosimtest, a Large Language Model (LLM)-driven framework, where multiple LLM agents collaborate to support the sUAS simulation testing process. This includes: (1) creating test scenarios that subject the SuT to unique environmental contexts; (2) preparing the simulation environment as per the test scenario; (3) generating diverse sUAS missions for the SuT to execute; and (4) analyzing simulation results and providing an interactive analytics interface. Further, the design of the framework is flexible for creating and testing scenarios for a variety of sUAS use cases, simulation tools, and SuT input requirements. We evaluated our approach by (a) conducting simulation testing of PX4 and ArduPilot flight-controller-based SuTs, (b) analyzing the performance of each agent, and (c) gathering feedback from sUAS developers. Our findings indicate that Autosimtest significantly improves the efficiency and scope of the sUAS testing process, allowing for more comprehensive and varied scenario evaluations while reducing the manual effort.

Abstract In the wake of a rapidly growing global focus on sustainable practices, the year 2023 marked a significant regulatory milestone, with the enactment of the Corporate Sustainability Reporting Directive. This directive significantly expands the scope of sustainability reporting, encompassing a broader array of enterprises, including large-scale, and small- and medium-sized enterprises within the EU. This regulatory development has profound implications for the software industry, as these companies need to provide comprehensive sustainability reporting for their software products. However, the industry still lacks models, tools, and methodologies for quantitatively assessing sustainability indicators during an agile software development process. In this paper, we introduce SustainScrum, a customized Scrum process model that incorporates sustainability evaluations into the development lifecycle of software products. More precisely, SustainScrum integrates the assessment of sustainability aspects into the backlog, user story management, and development stages of the Scrum process. Its primary objective is to ensure that sustainability considerations are systematically captured, evaluated, and addressed throughout the development process. It integrates the computation of quantitative sustainability indicators thereby advancing the ability to address sustainability challenges within software engineering practices. We performed an initial validation, investigating the applicability of SustainScrum on an open-source, publicly available requirements data set for agile development.

Small Unmanned Aerial Systems (sUAS) must meet rigorous safety standards when deployed in high-stress emergency response scenarios; however many reported accidents have involved humans in the loop. In this paper, we, therefore, present the HiFuzz testing framework, which uses fuzz testing to identify system vulnerabilities associated with human interactions. HiFuzz includes three distinct levels that progress from a low-cost, limited-fidelity, large-scale, no-hazard environment, using fully simulated Proxy Human Agents, via an intermediate level, where proxy humans are replaced with real humans, to a high-stakes, high-cost, real-world environment. Through applying HiFuzz to an autonomous multi-sUAS system-under-test, we show that each test level serves a unique purpose in revealing vulnerabilities and making the system more robust with respect to human mistakes. While HiFuzz is designed for testing sUAS systems, we further discuss its potential for use in other Cyber-Physical Systems.

The increased emphasis on competency management and learning objectives in higher education has led to a rise in Learning Analytics (LA) applications. These tools play a vital role in measuring and optimizing learning outcomes by analyzing and interpreting student-related data. \nLA tools furthermore provide course instructors with insights on how to refine teaching methods and material and address diversity in student performance to tailor instruction to individual needs. \nThis tool demonstration paper introduces our Learning Analytics Dashboard, designed for an introductory Python programming course. With a focus on gender diversity, the dashboard analyzes graded Jupyter Notebooks, to provide insights into student performance across assignments and exams. An initial assessment of the dashboard, applying it to our Python programming course in the previous year, has provided us with interesting insights and information on how to further improve our class and teaching materials. We present the dashboard's design, features, and outcomes while outlining our plans for its future development and enhancement.

Assurance of Learning and Competency-Based Education are increasingly important in higher education, not only for accreditation or transfer of credit points. Learning Analytics is crucial for making educational goals measurable and actionable, which is beneficial for program managers, course instructors, and students. \nWhile universities typically have an established tool landscape where relevant data is managed, information is typically scattered across various systems with different responsibilities and often only limited capabilities for sharing data. This diversity, however, significantly hampers the ability to analyze data, both on the course and curriculum level.\nTo address these shortcomings and to provide program managers, course instructions, and students with valuable insights, we devised an initial concept for a Multi-Level Shared Learning Analytics Framework to provide consistent definition and measurement of learning objectives, as well as tailored information, visualization, and analysis for different stakeholders.\nIn this paper, we present the results of initial interviews with stakeholders, devising core features. In addition, we assess potential risks and concerns that may arise from the implementation of such a framework and data analytics system. As a result, we identified six essential features and six main risks to guide further requirements elicitation and development of our proposed framework.

Artificial Intelligence (AI) approaches have been incorporated into modern learning environments and software engineering (SE) courses and curricula for several years. However, with the significant rise in popularity of large language models (LLMs) in general, and OpenAI's LLM-powered chatbot ChatGPT in particular in the last year, educators are faced with rapidly changing classroom environments and disrupted teaching principles. Examples range from programming assignment solutions that are fully generated via ChatGPT, to various forms of cheating during exams. However, despite these negative aspects and emerging challenges, AI tools in general, and LLM applications in particular, can also provide significant opportunities in a wide variety of SE courses, supporting both students and educators in meaningful ways. In this early research paper, we present preliminary results of a systematic analysis of current trends in the area of AI, and how they can be integrated into university-level SE curricula, guidelines, and approaches to support both instructors and learners. We collected both teaching and research papers and analyzed their potential usage in SE education, using the ACM Computer Science Curriculum Guidelines CS2023. As an initial outcome, we discuss a series of opportunities for AI applications and further research areas.

Testing and validating Cyber-Physical Systems (CPSs) in the aerospace domain, such as field testing of drone rescue missions, poses challenges due to volatile mission environments, such as weather conditions. While testing processes and methodologies are well established, structured guidance and execution support for field tests are still weak. This paper identifies requirements for field testing of drone missions and introduces the Field Testing Scenario Management (FiTS) approach for adaptive field testing guidance. FiTS aims to provide sufficient guidance for field testers as a foundation for efficient data collection to facilitate quality assurance and iterative improvement of field tests and CPSs. FiTS shall leverage concepts from scenario-based requirements engineering and Behavior-Driven Development to define structured and reusable test scenarios, with dedicated tasks and responsibilities for role-specific guidance. We evaluate FiTS by (i) applying it to three use cases for a search-and-rescue drone application to demonstrate feasibility and (ii) interviews with three experienced drone developers to assess its usefulness and collect further requirements. The study results indicate FiTS to be feasible and useful to facilitate drone field testing and data analysis.

Failures in safety-critical Cyber-Physical Systems (CPS), both software and hardware-related, can lead to severe incidents impacting physical infrastructure or even harming humans. As a result, extensive simulations and field tests need to be conducted, as part of the verification and validation of system requirements, to ensure system safety. However, current simulation and field testing practices, particularly in the domain of small Unmanned Aerial Systems (sUAS), are ad-hoc and lack a thorough, structured testing process. Furthermore, there is a dearth of standard processes and methodologies to inform the design of comprehensive simulation and field tests. This gap in the testing process leads to the deployment of sUAS applications that are: (a) tested in simulation environments which do not adequately capture the real-world complexity, such as environmental factors, due to a lack of tool support; (b) not subjected to a comprehensive range of scenarios during simulation testing to validate the system requirements, due to the absence of a process defining the relationship between requirements and simulation tests; and (c) not analyzed through standard safety analysis processes, because of missing traceability between simulation testing artifacts and safety analysis artifacts. To address these issues, we have developed an initial framework for validating CPS, specifically focusing on sUAS and robotic applications. We demonstrate the suitability of our framework by applying it to an example from the sUAS domain. Our preliminary results confirm the applicability of our framework. We conclude with a research roadmap to outline our next research goals along with our current proposal.

The Human Machine Teaming (HMT) paradigm focuses on supporting partnerships between humans and autonomous machines. HMT describes requirements for transparency, augmented cognition, and coordination that enable far richer partnerships than those found in typical human-on-the-loop and human-in-the-loop systems. Autonomous, self-adaptive systems in domains such as autonomous driving, robotics, and Cyber-Physical Systems, are often implemented using the MAPE-K feedback loop as the primary reference model. However, while MAPE-K enables fully autonomous behavior, it does not explicitly address the interactions that occur between humans and autonomous machines as intended by HMT. In this article, we, therefore, present the MAPE-K HMT framework, which utilizes runtime models to augment the monitoring, analysis, planning, and execution phases of the MAPE-K loop to support HMT despite the different operational cadences of humans and machines. We draw on examples from our own emergency response system of interactive, autonomous, small unmanned aerial systems to illustrate the application of MAPE-K HMT in both a simulated and physical environment, and we discuss how the various HMT models are connected and can be integrated into a MAPE-K solution.

Flight-time failures of small Uncrewed Aerial Systems (sUAS) can have a severe impact on people or the environment. Therefore, sUAS applications must be thoroughly evaluated and tested to ensure their adherence to specified requirements, and safe behavior under real-world conditions, such as poor weather, wireless interference, and satellite failure. However, current simulation environments for autonomous vehicles, including sUAS, provide limited support for validating their behavior in diverse environmental contexts and moreover, lack a test harness to facilitate structured testing based on system-level requirements. We address these shortcomings by eliciting and specifying requirements for an sUAS testing and simulation platform, and developing and deploying it. The constructed platform, DroneReq Validator (DRV), allows sUAS developers to define the operating context, configure multi-sUAS mission requirements, specify safety properties, and deploy their own custom sUAS applications in a high-fidelity 3D environment. The DRV Monitoring system collects runtime data from sUAS and the environment, analyzes compliance with safety properties, and captures violations. We report on two case studies in which we used our platform prior to real-world sUAS deployments, in order to evaluate sUAS mission behavior in various environmental contexts. Furthermore, we conducted a study with developers and found that DRV simplifies the process of specifying requirements-driven test scenarios and analyzing acceptance test results.

Internet of Things (IoT) devices have become ubiquitous in our everyday life, with security becoming an ever-growing issue as more and more cyber-attack incidents being reported, primarily due to deficiencies in existing security mechanisms. However, while, for example, cloud-based applications, or industrial automation systems of systems possess significant resources for monitoring health, and determining their status and correct behavior at runtime, IoT devices operate with limited hardware capabilities and under tight resource constraints, making monitoring, analysis, and response activities a challenging endeavor. Following the NIST Cybersecurity Framework, IoT devices need to identify, protect, detect, respond, and recover from cyber-attacks, unauthorized access, and other security threats. A common way to provide self-adaptation to changing conditions is the MAPE-K loop with four pivotal phases: Monitor, Analyze, Plan, and Execute. This paper presents DSec4IoT, a “Distributed MAPE-K Framework for Self-Protective IoT Devices”. Our framework leverages the idea of distributed MAPE-K patterns and establishes a model for managing and controlling Self-Protective IoT Devices. We evaluate our approach by simulating port scans and performing adaptation activities. Results have confirmed that DSec4IoT can be easily applied to detect and mitigate them.

When dealing with safety–critical systems, various regulations, standards, and guidelines stipulate stringent requirements for certification and traceability of artifacts, but typically lack details with regards to the corresponding software engineering process. Given the industrial practice of only using semi-formal notations for describing engineering processes – with the lack of proper tool mapping – engineers and developers need to invest a significant amount of time and effort to ensure that all steps mandated by quality assurance are followed. The sheer size and complexity of systems and regulations make manual, timely feedback from Quality Assurance (QA) engineers infeasible. In order to address these issues, in this paper, we propose a novel framework for tracking, and “passively” executing processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to three case studies: a safety–critical open-source community system, a safety–critical system in the air-traffic control domain, and a non-safety–critical, web-based system. Results from our analysis confirm that trace links are often corrected or completed after the work step has been considered finished, and the engineer has already moved on to another step. Thus, support for timely and automated constraint checking has significant potential to reduce rework as the engineer receives continuous feedback already during their work step.

Software vulnerabilities can affect critical systems within an organization impacting processes, workflows, privacy, and safety. When a software vulnerability becomes known, affected systems are at risk until appropriate updates become available and eventually deployed. This period can last from a few days to several months, during which attackers can develop exploits and take advantage of the vulnerability. It is tedious and time-consuming to keep track of vulnerabilities manually and perform necessary actions to shut down, update, or modify systems. Vulnerabilities affect system components, such as a web server, but sometimes only target specific versions or component combinations. In this paper, we propose a novel approach for automated mode switching of software systems to support system administrators in dealing with vulnerabilities and reducing the risk of exposure. We rely on model-driven techniques and use a multi-modal architecture to react to discovered vulnerabilities and provide automated contingency support. We have developed a dedicated domain-specific language to describe potential mitigation as mode switches. We have evaluated our approach with a web server case study, analyzing historical vulnerability data. Based on the vulnerabilities scores sum, we demonstrated that switching to less vulnerable modes reduced the attack surface in 98.9% of the analyzed time. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Runtime monitoring is critical for ensuring safe operation and for enabling self-adaptive behavior of Cyber-Physical Systems (CPS). Monitors are established by identifying runtime properties of interest, creating probes to instrument the system, and defining constraints to be checked at runtime. For many systems, implementing and setting up a monitoring platform can be tedious and time-consuming, as generic monitoring platforms do not adequately cover domain-specific monitoring requirements. This situation is exacerbated when the System under Monitoring (SuM) evolves, requiring changes in the monitoring platform. Most existing approaches lack support for the automated generation and setup of monitors for diverse technologies and do not provide adequate support for dealing with system evolution. In this paper, we present GRuM (Generating CPS Runtime Monitors), a framework that combines model-driven techniques and runtime monitoring, to automatically generate a customized monitoring platform for a given SuM. Relevant properties are captured in a Domain Model Fragment, and changes to the SuM can be easily accommodated by automatically regenerating the platform code. To demonstrate the feasibility and performance we evaluated GRuM against two different systems using TurtleBot robots and Unmanned Aerial Vehicles. Results show that GRuM facilitates the creation and evolution of a runtime monitoring platform with little effort and that the platform can handle a substantial amount of events and data.

Cyber–Physical Systems (CPS) are increasingly used in safety–critical scenarios where ensuring their correct behavior at runtime becomes a crucial task. Therefore, the behavior of the CPS needs to be monitored at runtime so that violations of requirements can be detected. With the inception of edge devices that facilitate runtime analysis at the edge and the increasingly diverse environments that CPS operate in, flexible monitoring approaches are needed that consider the data that needs to be monitored and the analyses performed on that data. In this paper, we propose AMon, a flexible adaptive monitoring framework that supports the specification and validation of monitoring adaptation rules, using a domain-specific language. Based on these rules, AMon automatically generates code for direct deployment onto devices. We evaluated AMon by applying it to TurtleBot Robots and a fleet of Unmanned Aerial Vehicles. Furthermore, we conducted a user study assessing the understandability and ease of use of our language. Results show that creating multiple adaptation rules with our DSL is feasible with minimal effort, and that adaptive monitoring can reduce the amount of runtime data transmitted from the edge device according to the current state of the system and its monitoring needs.

The MAPE-K feedback loop has been established as the primary reference model for self-adaptive and autonomous systems in domains such as autonomous driving, robotics, and Cyber-Physical Systems. At the same time, the Human Machine Teaming (HMT) paradigm is designed to promote partnerships between humans and autonomous machines. It goes far beyond the degree of collaboration expected in human-on-the-loop and human-in-the-loop systems and emphasizes interactions, partnership, and teamwork between humans and machines. However, while MAPE-K enables fully autonomous behavior, it does not explicitly address the interactions between humans and machines as intended by HMT. In this paper, we present the MAPE-KHMT framework which augments the traditional MAPE-K loop with support for HMT. We identify critical human-machine teaming factors and describe the infrastructure needed across the various phases of the MAPE-K loop in order to effectively support HMT. This includes runtime models that are constructed and populated dynamically across monitoring, analysis, planning, and execution phases to support human-machine partnerships. We illustrate MAPE-KHMT using examples from an autonomous multi-UAV emergency response system, and present guidelines for integrating HMT into MAPE-K.

Self-adaptive systems typically operate in heterogeneous environments and need to optimize their behavior based on a variety of quality attributes to meet stakeholders' needs. During adaptation planning, these quality attributes are considered in the form of constraints, describing requirements that must be fulfilled, and utility functions, which are used to select an optimal plan among several alternatives. Up until now, most automated planning approaches are not designed to adapt quality attributes, their priorities, and their trade-offs at run time. Instead, both utility functions and constraints are commonly defined at design time. There exists a clear lack of run-time mechanisms that support their adaptation in response to changes in the environment or in stakeholders' preferences. In this paper, we present initial work that combines automated planning and adaptation of quality attributes to address this gap. The approach helps to semi-automatically adjust utility functions and constraints based on changes at run time. We present a preliminary experimental evaluation that indicates that our approach can provide plans with higher utility values while fulfilling changed or added constraints. We conclude this paper with our envisioned research outlook and plans for future empirical studies.

As technologies such as the Internet of Things (IoT) and Cyber-Physical Systems (CPS) are becoming ubiquitous, systems adopting these technologies are getting increasingly complex. Digital Twins (DTs) provide comprehensive views on such systems, the data they generate during runtime, as well as their usage and evolution over time. Setting up the required infrastructure to run a Digital Twin is still an ambitious task that involves significant upfront efforts from domain experts, although existing knowledge about the systems, such as engineering models, may be already available for reuse. To address this issue, we present AML4DT, a model-driven framework supporting the development and maintenance of Digital Twin infrastructures by employing AutomationML (AML) models. We automatically establish a connection between systems and their DTs based on dedicated DT models. These DT models are automatically derived from existing AutomationML models, which are produced in the engineering phases of a system. Additionally, to alleviate the maintenance of the DTs, AML4DT facilitates the synchronization of the AutomationML models with the DT infrastructure for several evolution cases. A case study shows the benefits of developing and maintaining DTs based on AutomationML models using the proposed AML4DT framework. For this particular study, the effort of performing the required tasks could be reduced by about 50%.

Regulations, standards, and guidelines for safety-critical systems stipulate stringent traceability but do not prescribe the corresponding, detailed software engineering process. Given the industrial practice of using only semi-formal notations to describe engineering processes, processes are rarely "executable" and developers have to spend significant manual effort in ensuring that they follow the steps mandated by quality assurance. The size and complexity of systems and regulations makes manual, timely feedback from Quality Assurance (QA) engineers infeasible. In this paper we propose a novel framework for tracking processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to two different case studies; one open source community system and a safety-critical system in the air-traffic control domain. Results from the analysis show that trace links are often corrected or completed after the fact and thus timely and automated constraint checking support has significant potential on reducing rework.

With the rise of new AI technologies, autonomous systems are moving towards a paradigm in which increasing levels of responsibility are shifted from the human to the system, creating a transition from human-in-the-loop systems to human-on-the-loop (HoTL) systems. This has a significant impact on the safety analysis of such systems, as new types of errors occurring at the boundaries of human-machine interactions need to be taken into consideration. Traditional safety analysis typically focuses on system-level hazards with little focus on user-related or user-induced hazards that can cause critical system failures. To address this issue, we construct domain-level safety analysis assets for sUAS (small unmanned aerial systems) applications and describe the process we followed to explicitly, and systematically identify Human Interaction Points (HiPs), Hazard Factors and Mitigations from system hazards. We evaluate our approach by first investigating the extent to which recent sUAS incidents are covered by our hazard trees, and second by performing a study with six domain experts using our hazard trees to identify and document hazards for sUAS usage scenarios. Our study showed that our hazard trees provided effective coverage for a wide variety of sUAS application scenarios and were useful for stimulating safety thinking and helping users to identify and potentially mitigate human-interaction hazards.

Abstract Many software systems today provide support for adaptation and reconfiguration at runtime, in response to changes in their environment. Such adaptive systems are designed to run continuously and may not be shut down for reconfiguration or maintenance tasks. The variability of such systems has to be explicitly managed, together with mechanisms that control their runtime adaptation and reconfiguration. Dynamic software product lines (DSPLs) can help to achieve this. However, dealing with evolution is particularly challenging in a DSPL, as changes made at runtime can easily lead to inconsistencies. This paper describes the challenges of evolving DSPLs using an example cyber‐physical system for home automation. We discuss the shortcomings of existing work and present a reference architecture to support DSPL evolution. To demonstrate its feasibility and flexibility, we implemented the proposed reference architecture for two different DSPLs: the aforementioned cyber‐physical system, which uses feature models to describe its variability, and a runtime monitoring infrastructure, which is based on decision models. To assess the industrial applicability of our approach, we also implemented the reference architecture for a real‐world DSPL, an automation software system for injection molding machines. Our results provide evidence on the flexibility, performance, and industrial applicability of our approach.

The growing adoption of unmanned aerial vehicles (UAVs) for tasks such as eCommerce, aerial surveillance, and environmental monitoring introduces the need for new safety mechanisms in an increasingly cluttered airspace. In our work we thus emphasize safety issues that emerge at the intersection of infrastructures responsible for controlling the airspace, and the diverse UAVs operating in their space. We build on safety assurance cases (SAC) - a state-of-the-art solution for reasoning about safety - and propose a novel approach based on interlocking SACs. The infrastructure safety case (ISAC) specifies assumptions upon UAV behavior, while each UAV demonstrates compliance to the ISAC by presenting its own (pluggable) safety case (pSAC) which connects to the ISAC through a set of interlock points. To collect information on each UAV we enforce a “trust but monitor” policy, supported by runtime monitoring and an underlying reputation model. We evaluate our approach in three ways: first by developing ISACs for two UAV infrastructures, second by running simulations to evaluate end-to-end effectiveness, and finally via an outdoor field-study with physical UAVs. The results show that interlocking SACs can be effective for identifying, specifying, and monitoring safety-related constraints upon UAVs flying in a controlled airspace.

The use of semi-autonomous Unmanned Aerial Vehicles (UAV) to support emergency response scenarios, such as fire surveillance and search and rescue, offers the potential for huge societal benefits. However, designing an effective solution in this complex domain represents a "wicked design" problem, requiring a careful balance between trade-offs associated with drone autonomy versus human control, mission functionality versus safety, and the diverse needs of different stakeholders. This paper focuses on designing for situational awareness (SA) using a scenario-driven, participatory design process. We developed SA cards describing six common design-problems, known as SA demons, and three new demons of importance to our domain. We then used these SA cards to equip domain experts with SA knowledge so that they could more fully engage in the design process. We designed a potentially reusable solution for achieving SA in multi-stakeholder, multi-UAV, emergency response applications.

Safety Assurance Cases (SACs) are increasingly used to guide and evaluate the safety of software-intensive systems. They are used to construct a hierarchically organized set of claims, arguments, and evidence in order to provide a structured argument that a system is safe for use. However, as the system evolves and grows in size, a SAC can be difficult to maintain. In this paper we utilize design science to develop a novel solution for identifying areas of a SAC that are affected by changes to the system. Moreover, we generate actionable recommendations for updating the SAC, including its underlying artifacts and trace links, in order to evolve an existing safety case for use in a new version of the system. Our approach, Safety Artifact Forest Analysis (SAFA), leverages traceability to automatically compare software artifacts from a previously approved or certified version with a new version of the system. We identify, visualize, and explain changes in a Delta Tree. We evaluate our approach using the Dronology system for monitoring and coordinating the actions of cooperating, small Unmanned Aerial Vehicles. Results from a user study show that SAFA helped users to identify changes that potentially impacted system safety and provided information that could be used to help maintain and evolve a SAC.

Complex software systems evolve frequently, e.g., when introducing new features or fixing bugs during maintenance. However, understanding the impact of such changes on system behavior is often difficult. Many approaches have thus been proposed that analyze systems before and after changes, e.g., by comparing source code, model-based representations, or system execution logs. In this paper, we propose an approach for comparing run-time constraints, synthesized by a constraint mining algorithm, based on execution logs recorded before and after changes. Specifically, automatically mined constraints define the expected timing and order of recurring events and the values of data elements attached to events. Our approach presents the differences of the mined constraints to users, thereby providing a higher-level view on software evolution and supporting the analysis of the impact of changes on system behavior. We present a motivating example and a preliminary evaluation based on a cyber-physical system controlling unmanned aerial vehicles. The results of our preliminary evaluation show that our approach can help to analyze changed behavior and thus contributes to understanding software evolution.

Unmanned aerial vehicles (UAVs) are becoming increasingly pervasive in everyday life, supporting diverse use cases such as aerial photography, delivery of goods, or disaster reconnaissance and management. UAVs are cyber-physical systems (CPS): they integrate computation (embedded software and control systems) with physical components (the UAVs flying in the physical world). UAVs in particular and CPS in general require monitoring capabilities to detect and possibly mitigate erroneous and safety-critical behavior at runtime. Existing monitoring approaches mostly do not adequately address UAV CPS characteristics such as the high number of dynamically instantiated components, the tight int elements, and the massive amounts of data that need to be processed. In this paper we report results of a case study on monitoring in UAVs. We discuss CPS-specific monitoring challenges and present a prototype we implemented by extending \reminds, a framework for software monitoring so far mainly used in the domain of metallurgical plants. Additionally, we demonstrate the applicability and scalability of our approach by monitoring a real control and management system for UAVs in simulations with up to 30 drones flying in an urban area.

Traditionally, safety-critical projects have been developed using the waterfall process. However, this makes it costly and challenging to incrementally introduce new features and to certify the modified product for use. As a result, there has been increasing interest in adopting agile development paradigms within the safety-critical domain. This in turn introduces numerous challenges. In this paper we address the specific problems of discovering, analyzing, specifying, and managing safety requirements within the agile Scrum process. We propose SafetyScrum, a methodology that augments the Scrum lifecycle with incrementally applied safety-related activities and introduces the notion of "safety debt" for incrementally tracking the current safety status of a project. We demonstrate the viability of SafetyScrum for managing safety stories in an agile development environment by applying it to a project in which our existing Unmanned Aerial Vehicle system is enhanced to support a River-Rescue scenario.

Industrial software systems are often systems of systems (SoS) whose full behavior only emerges during operation. They therefore require monitoring techniques to observe systems and detect deviations from their requirements. The focus of existing monitoring approaches, however, is mainly on detecting violations of expected behavior, while support for diagnosing violations is typically limited or even neglected. Diagnosis is particularly challenging in SoS due to their technological heterogeneity and the diversity of development tools in use. Uncovering the root cause of a violation typically requires developers to trace violations to artifacts such as source code or requirements documents, which is difficult without detailed domain knowledge. In this paper we describe our experiences of developing a tool-supported approach facilitating the diagnosis of requirements violations in SoS. We describe how we complemented a requirements monitoring model with a system artifact model relating SoS artifacts needed for diagnosis with monitored events. We customized our approach to an industrial SoS and conducted a scenario-based walkthrough with engineers developing the SoS and engineers and researchers unfamiliar with it. The results of our evaluation have shown that our approach can significantly ease diagnosing violations in a real-world SoS.

This extended abstract summarizes our paper entitled "A Comparison Framework for Runtime Monitoring Approaches" published in the Journal on Systems and Software in vol. 125 in 2017 (https://doi.org/10.1016/jjss.2016.12.034). This paper provides the following contributions: (i) a framework that supports analyzing and comparing runtime monitoring approaches using different dimensions and elements; (ii) an application of the framework to analyze and compare 32 existing monitoring approaches; and (iii) a discussion of perspectives and potential future applications of our framework, e.g., to support the selection of an approach for a particular monitoring problem or application context.

Domain-specific languages (DSLs) are frequently used in software engineering. In contrast to general-purpose languages, DSLs are designed for a special purpose in a particular domain. Due to volatile user requirements and new technologies DSLs, similar to the software systems they describe or produce, are subject to continuous evolution. This work explores existing research on DSL evolution to summarize, structure and analyze this area of research, and to identify trends and open issues. We conducted a systematic mapping study and identified 98 papers as potentially relevant for our study. By applying inclusion and exclusion criteria we selected a set of 34 papers relevant for DSL evolution. We classified and analyzed these papers to create a map of the research field. We conclude that DSL evolution is a topic of increasing relevancy. However, research on language evolution so far did not focus much on the characteristics DSLs exhibit. Also, there are not many cross-references between our primary studies meaning researchers are often not aware of potentially useful work. Our study results help researchers and practitioners working on DSL-based approaches to get an overview of existing research on DSL evolution and open challenges.

Industrial software systems are often systems of systems (SoS) whose full behavior only emerges at runtime. The systems and their interactions thus need to be continuously monitored and checked during operation to determine compliance with requirements. Many requirements monitoring approaches have been proposed. However, only few of these come with tools that present and visualize monitoring results and details on requirements violations to end users such as industrial engineers. In this tool demo paper we present visualization capabilities we have been developing motivated by industrial scenarios. Our tool complements ReMinds, an existing requirements monitoring framework, which supports collecting, aggregating, and analyzing events and event data in architecturally heterogeneous SoS. Our visualizations support a `drill-down' scenario for monitoring and diagnosis: starting from a graphical status overview of the monitored systems and their relations, engineers can view trends and statistics about performed analyses and diagnose the root cause of problems by inspecting the events and event data that led to a specific violation. Initial industry feedback we received confirms the usefulness of our tool support. Demo video: https://youtu.be/iv7kWzeNkdk..

Requirements Engineering (RE) is comprised of various tasks related to discovering, documenting, and maintaining different kinds of requirements. To accomplish these tasks, a Requirements Engineer or Business Analyst needs to retrieve and combine information from multiple sources such as use case models, interview scripts, and business rules. However, collecting and analyzing all the required data can be tedious and the resulting data is often incomplete with inadequate trace links. Analyzing real-world queries can shed light on the questions requirements professionals would like to ask and the artifacts needed to support such questions. We therefore conducted an online survey with requirements professionals in the IT industry. Our analysis included 29 survey responses and a total of 159 natural language queries. Using open coding and grounded theory, we analyzed and grouped these queries into 9 different query purposes and 54 sub-purposes, and also identified frequently used artifacts. The results from the survey could help project-level planners identify important questions, proactively instrument their environments with supporting tools, and strategically collect data that is needed to answer the queries of interest to their project.

Software project artifacts such as source code, requirements, and change logs represent a gold-mine of actionable information. As a result, software analytic solutions have been developed to mine repositories and answer questions such as "who is the expert?," "which classes are fault prone?," or even "who are the domain experts for these fault-prone classes?" Analytics often require training and configuring in order to maximize performance within the context of each project. A cold-start problem exists when a function is applied within a project context without first configuring the analytic functions on project-specific data. This scenario exists because of the non-trivial effort necessary to instrument a project environment with candidate tools and algorithms and to empirically evaluate alternate configurations. We address the cold-start problem by comparatively evaluating 'best-of-breed' and 'profile-driven' solutions, both of which reuse known configurations in new project contexts. We describe and evaluate our approach against 20 project datasets for the three analytic areas of artifact connectivity, fault-prediction, and finding the expert, and show that the best-of-breed approach outperformed the profile-driven approach in all three areas; however, while it delivered acceptable results for artifact connectivity and find the expert, both techniques underperformed for cold-start fault prediction.

Complex software-intensive systems are often described as systems of systems (SoS) comprising heterogeneous architectural elements. As SoS behavior fully emerges during operation only, runtime monitoring is needed to detect deviations from requirements. Today, diverse approaches exist to define and check runtime behavior and performance characteristics. However, existing approaches often focus on specific types of systems and address certain kinds of checks, thus impeding their use in industrial SoS. Furthermore, as many SoS need to run continuously for long periods, the dynamic definition and deployment of constraints needs to be supported. In this paper we describe experiences of developing and applying a DSL-based approach for monitoring an SoS in the domain of industrial automation software. We evaluate both the expressiveness of our DSL as well as the scalability of the constraint checker. We also describe lessons learned.

The behavior of systems of systems (SoS) emerges only fully during operation and is hard to predict. SoS thus need to be monitored at runtime to detect deviations from important requirements. However, existing approaches for checking runtime behavior and performance characteristics are limited with respect to the kinds of checks and the types of technologies supported, which impedes their use in industrial SoS. In this tool demonstration paper we describe the ReMinds tool suite for runtime monitoring of SoS developed in response to industrial monitoring scenarios. ReMinds provides comprehensive tool support for instrumenting systems, extracting events and data at runtime, defining constraints to check expected behavior and properties, and visualizing constraint violations to facilitate diagnosis.

Many software systems today can be characterized as systems of systems (SoS) comprising interrelated and heterogeneous systems developed by diverse teams over many years. Due to their scale, complexity, and heterogeneity engineers face significant challenges when determining the compliance of SoS with their requirements. Requirements monitoring approaches are a viable solution for checking system properties at runtime. However, existing approaches do not adequately consider the characteristics of SoS: different types of requirements exist at different levels and across different systems; requirements are maintained by different stakeholders; and systems are implemented using diverse technologies. This paper describes a three-dimensional requirements monitoring model (RMM) for SoS providing the following contributions: (i) our approach allows modeling the monitoring scopes of requirements with respect to the SoS architecture; (ii) it employs event models to abstract from different technologies and systems to be monitored; and (iii) it supports instantiating the RMM at runtime depending on the actual SoS configuration. To evaluate the feasibility of our approach we created a RMM for a real-world SoS from the automation software domain. We evaluated the model by instantiating it using an existing monitoring framework and a simulator running parts of this SoS. The results indicate that the model is sufficiently expressive to support monitoring SoS requirements of a directed SoS. It further facilitates diagnosis by discovering violations of requirements across different levels and systems in realistic monitoring scenarios.

In many domains systems need to run continuously and cannot be shut down for reconfiguration or maintenance tasks. Cyber-physical or cloud-based systems, for instance, thus often provide means to support their adaptation at runtime. The required flexibility and adaptability of systems suggests the application of Software Product Line (spl) principles to manage their variability and to support their reconfiguration. Specifically, Dynamic Software Product Lines (dspl) have been proposed to support the management and binding of variability at runtime. While spl evolution has been widely studied, it has so far not been investigated in detail in a dspl context. Variability models that are used in a dspl have to co-evolve and be kept consistent with the systems they represent to support reconfiguration even after changes to the systems at runtime. In this short paper we present a classification of the required operations for jointly evolving problem and solution space in a dspl. We analyze the impact of such operations on the consistency of a dspl and propose an approach to deal with the described issues. We describe a runtime monitoring system used in the domain of industrial automation software as an example of a dspl evolving at runtime to motivate and explain our work.

An increasing number of software systems today are systems of systems (SoS) that have been developed by diverse teams over many years. Such systems emerge gradually and it is hard to analyze or predict their behavior due to their scale, complexity, and heterogeneity. In particular, certain behavior only emerges at runtime due to complex interactions between the involved systems and their environment. Requirements monitoring has been proposed as a solution for checking at runtime whether systems adhere to their requirements. However, existing requirements monitoring approaches have been designed for single systems and therefore do not adequately consider the characteristics of SoS. More specifically, requirements in SoS exist at different levels, across different systems, and are owned by diverse stakeholders. Furthermore, requirements monitoring for SoS has to be flexible with respect to technologies and architectural patterns. This thesis will identify the capabilities required for requirements monitoring of SoS. It will further provide a flexible and tailorable infrastructure to support engineers and maintenance staff in observing and analyzing the behavior of a SoS at runtime. We plan to evaluate our work by assessing its usefulness in the context of an industrial SoS.

Many software systems today have system-of systems (SoS) architectures comprising interrelated and heterogeneous systems, which are developed by multiple teams and companies. Such systems emerge gradually and it is hard to analyze or predict their behavior due to their scale and complexity. In particular, certain behavior only emerges at runtime due to complex interactions between the involved systems and their environment. Monitoring the behavior of SoS at runtime is thus essential during development and evolution. However, existing monitoring approaches are often limited to particular architectural styles or technologies and are thus hard to apply in SoS architectures. In this paper we first analyze the challenges for monitoring SoS based on an industrial SoS for the automation of metallurgical plants. We then propose a flexible framework for monitoring heterogeneous systems within a SoS. We demonstrate its feasibility by applying it to two systems of an industrial SoS. We also report results of an evaluation assessing the framework's performance and scalability.

Large-scale software-intensive systems are often considered as systems of systems (SoS) comprising multiple heterogeneous but interrelated systems. The engineering of SoS often involves the derivation of system variants from multiple interrelated product lines to meet the overall requirements. If multiple teams and experts are involved in the configuration of these individual systems, their individual configuration choices may conflict with each other or violate constraints. This paper illustrates industrial challenges based on a previously conducted case study on distributed configuration in multi product lines. We then present CoDiM, a tool-supported approach for defining and checking constraints in distributed configuration of an SoS. Our approach is integrated in the product line tool suite DOPLER developed in cooperation with industry partners. An application scenario from a real-world multi product line demonstrates how our approach allows detecting violations of constraints during distributed configuration of an SoS. The approach provides immediate feedback to configurers during product derivation and enables the dynamic definition of constraints even during configuration time to accommodate changes. CoDiM further supports constraint templates which can be parameterized to allow their reuse in different multi product line configurations.

Business calculations like break-even, return on investment, or cost are essential in many domains to support decision making while configuring products. For instance, customers and sales people need to estimate and compare the business value of different product variants. Some product line approaches provide initial support, e.g., by defining quality attributes in relation to features. However, an approach that allows domain engineers to easily define business calculations together with variability models is still lacking. In product configuration, calculation results need to be instantly presented to end users after making configuration choices. Further, due to the often high number of calculations, the presentation of calculation results to end users can be challenging. These challenges cannot be addressed by integrating off-the-shelf applications performing the calculations with product line tools. We thus present an approach based on dedicated calculation models that are related to variability models. Our approach seamlessly integrates business calculations with product configuration and provides support for formatting calculations and calculation results. We use the DOPLER tool suite to deploy calculations together with variability models to end users in product configuration. We evaluate the expressiveness and practical relevance of the approach by investigating the development of business calculations for 15 product lines from the domain of industrial automation.

Industrial experiences show that support for sharing and deploying product line models and tools is essential when institutionalizing product line engineering. This paper presents key workflows together with an infrastructure providing support for this purpose. Our approach supports distributed users sharing work products during variability modeling, product derivation, and product line evolution. The approach is based on product line bundles (PLiBs) for packaging models and tool support for specific product lines. Using three industrial scenarios and an industrial product line example we demonstrate how our infrastructure supports the deployment of models and tools in practical settings.

The complexity of product line variability models makes it hard to maintain their consistency over time regardless of the modeling approach used. Engineers thus need support for detecting and resolving inconsistencies. We describe experiences of applying a tool-supported approach for incremental consistency checking on variability models. Our approach significantly improves the overall performance and scalability compared to batch-oriented techniques and allows providing immediate feedback to modelers. It is extensible as new consistency constraints can easily be added. Furthermore, the approach is flexible as it is not limited to variability models and it also checks the consistency of the models with the underlying code base of the product line. We report the results of a thorough evaluation based on real-world product line models and discuss lessons learned.