Cyber-physical production systems increasingly involve collaborative robotic missions, which come with a higher demand for robustness and safety. Practitioners rely on risk assessments to identify potential failures and implement measures to mitigate their risks. Ensuring that mitigation strategies derived from risk assessments are adequately considered in the software implementation can be challenging, especially when stakeholders involved in the assessment process lack a programming background. This leads to a disconnection between the outputs of risk assessments and the actual implementation of robotic missions. To address this issue, there is a need to integrate software engineering practices into the risk assessment process to ensure consistency and traceability between the outputs of risk assessments and their corresponding software implementation.This paper presents a design science study that conceived a model-based approach for early risk assessment in a development-centric way. Our approach supports risk assessment activities by using behavior-tree models. We evaluated the approach together with five practitioners from four companies. This approach is the first attempt to use behavior-tree models to support risk assessment. Our findings highlight the potential of behavior-tree models in supporting early identification, visualization, and bridging the gap between code implementation and the outputs of risk assessments. Our findings suggest research directions for further development of the approach to increase its applicability and usefulness in practice.

TRUST in human-robot interactions (HRI) is essential for effective collaboration and user acceptance of robotic systems. However, trust can be challenged by conflicting goals between the user and the system, such as in the context of proxemics when a robot invades a human’s personal space. As robotic systems increasingly adapt their actions autonomously, intelligent conflict resolution is necessary to not undermine humans’ trust in the robot’s decisions due to a decreased sense of control and understanding. Therefore, this study investigates how humans can be involved and supported during automated conflict resolution to maintain their trust by exploring different degrees of explainability and automation. We applied a within-subjects experimental design, where 20 participants experienced four conditions varying in levels of automation and explainability during a simulated conflict between their need to preserve their personal space and the robot’s requirement to approach to perform a task. We measured the participants’ trust through questionnaires and interviews. Our findings suggest that trust is positively influenced by the level of control users have during conflict situations. However, the robot’s explanations did not significantly impact trust. Our insights highlight the need for conflict management strategies in HRI that balance automation with user involvement to foster greater levels of trust when collaborating with robots.

Machine-learning-enabled systems are becoming increasingly common in different industries. Due to the impact of uncertainty and the pronounced role of data, ensuring the quality of such systems requires consideration of several unique characteristics in addition to traditional ones. This range of quality attributes can be achieved by the implementation of specific architectural tactics. Such architectural decisions affect the further functioning of the system and its compliance with business goals. Architectural decisions have to be made with attention to possible quality trade-offs to prevent the cost of mitigating unintended side effects. A related work analysis revealed the need for a thorough study of existing architectural decisions and their impact on various quality attributes in the context of machine-learning-enabled systems. In this paper, to address this goal, we present comprehensive research on the quality of such systems, architectural tactics, and their possible quality consequences. Based on a systematic literature review of 206 primary sources, we identified 11 common quality attributes, and 16 relevant architectural tactics together along with 85 potential quality trade-offs. Our results systematize existing research in building architectures of ML-enabled systems. They can be used by software architects and researchers at the system design stage to estimate the possible consequences of decisions made. • A Common Quality Model for ML-enabled systems. • A List of Architectural Tactics to Achieve Identified Quality Attributes. • A Broad Analysis of Quality Trade-offs.

Abstract Large-scale systems development commonly faces the challenge of managing relevant knowledge between different organizational groups, particularly in increasingly agile contexts. Here, there is a conflict between coordination and group autonomy, and it is challenging to determine what necessary coordination information must be shared by what teams or groups, and what can be left to local team management. We introduce a way to manage this complexity using a modeling framework based on two core concepts: methodological islands (i.e., groups using different development methods than the surrounding organization) and boundary objects (i.e., artifacts that create a common understanding across team borders). However, we found that companies often lack a systematic way of assessing coordination issues and the use of boundary objects between methodological islands. As part of an iterative design science study, we have addressed this gap by producing a modeling framework (BOMI: Boundary Objects and Methodological Islands) to better capture and analyze coordination and knowledge management in practice. This framework includes a metamodel, as well as a list of bad smells over this metamodel that can be leveraged to detect inter-team coordination issues. The framework also includes a methodology to suggest concrete modeling steps and broader guidelines to help apply the approach successfully in practice. We have developed Eclipse-based tool support for the BOMI method, allowing for both graphical and textual model creation, and including an implementation of views over BOMI instance models in order to manage model complexity. We have evaluated these artifacts iteratively together with five large-scale companies developing complex systems. In this work, we describe the BOMI framework and its iterative evaluation in several real cases, reporting on lessons learned and identifying future work. We have produced a matured and stable modeling framework which facilitates understanding and reflection over complex organizational configurations, communication, governance, and coordination of knowledge artifacts in large-scale agile system development.

Security assurance cases (SACs) consist of arguments that are supported by evidence to justify that a system is acceptably secure. However, they are a relatively static representation of the system’s security and therefore currently not effective at runtime which make them difficult to maintain and unable to support users during threats. The aim of this paper is to investigate how SACs can be adapted to become more effective at runtime and increase confidence in the system’s security. We extend an example SAC with game theory, which models the interaction between the system and attacker and identifies their optimal strategies based on their payoffs and likelihoods. The extension was added as a security control in the assurance case, where a security claim indicates what strategy should be taken at runtime. This claim changes dynamically with the recommended strategy output by the game-theoretic model at runtime. Based on the results of the evaluation, the extension was considered to be potentially effective, however this would further depend on how it is implemented in practice.

Requirement conflicts commonly o ccur i n soft-ware development, especially for complex systems that involve many requirements. Resolving these conflicts c an b every time-consuming and costly. Moreover, due to the contextual nature of requirements conflicts, resolutionstrategies a re n eeded that can be tailored to a specific conflict an d it s co ntext. Currently, there is a lack of research on what categories of conflicts exist and how practitioners manage these conflicts. Toe nable more adaptive resolution strategies, the aim of this research is to map what types of requirement conflicts a re e ncountered a nd how they are managed in practice. Through an interview study with eleven participants from eight companies in six domains, we identified three levels of conflict ty pes th at co nnect to four types of causes. These types and causes revealed two main dimensions that impact conflict management. One dimension is related to the nature of the conflict, either technical or social, a nd the other dimension is related to the nature of the requirement scope, which is either too constraining or too undefined resulting in stakeholders making assumptions that create conflicts. We found that these two dimensions impact what conflict resolution strategies are most commonly used for different types of conflicts.

Despite the recognized importance of software architecture, it is common that the implementation diverges from the intended architecture over time. This phenomenon is referred to as architectural drift. In the past decades, mainly technical solutions and tools have been developed to detect and address architectural inconsistencies and drift. There is still a lack of evidence from the perspective of developers and a lack of best practices to manage drift. This mixed-methods study relies on interviews with 11 developers and a survey answered by 63 developers from different companies and domains. We analyzed the data by dividing developers into senior and junior to see the different perspectives based on work experience. We found that juniors tend to rely more on documentation, while seniors have a more experience-related approach. We identified practices that developers use to mitigate drift, including defining clear responsibilities, setting best practices, and maintaining reliable documentation. Finally, we designed and evaluated guidelines to help developers to face architectural drift.

Software engineering artifacts are central to DevOps, enabling the collaboration of teams involved with integrating the development and operations domains. However, collaboration around DevOps artifacts has yet to receive detailed research attention. We apply the sociological concept of Boundary Objects to describe and evaluate the specific software engineering artifacts that enable a cross-disciplinary understanding. Using this focus, we investigate how different DevOps stakeholders can collaborate efficiently using common artifacts. We performed a multiple case study and conducted twelve semi-structured interviews with DevOps practitioners in nine companies. We elicited participants’ collaboration practices, focusing on the coordination of stakeholders and the use of engineering artifacts as a means of translation. This paper presents a consolidated overview of four categories of DevOps Boundary Objects and eleven stakeholder groups relevant to DevOps. To help practitioners assess cross-disciplinary knowledge management strategies, we detail how DevOps Boundary Objects contribute to four areas of DevOps knowledge and propose derived dimensions to evaluate their use.

In software design, guaranteeing the correctness of run-time system behavior while achieving an acceptable balance among multiple quality attributes remains a challenging problem. Moreover, providing guarantees about the satisfaction of those requirements when systems are subject to uncertain environments is even more challenging. While recent developments in architectural analysis techniques can assist architects in exploring the satisfaction of quantitative guarantees across the design space, existing approaches are still limited because they do not explicitly link design decisions to satisfaction of quality requirements. Furthermore, the amount of information they yield can be overwhelming to a human designer, making it difficult to see the forest for the trees. In this paper we present ExTrA (Explaining Tradeoffs of software Architecture design spaces), an approach to analyzing architectural design spaces that addresses these limitations and provides a basis for explaining design tradeoffs. Our approach employs dimensionality reduction techniques employed in machine learning pipelines like Principal Component Analysis (PCA) and Decision Tree Learning (DTL) to enable architects to understand how design decisions contribute to the satisfaction of extra-functional properties across the design space. Our results show feasibility of the approach in two case studies and evidence that combining complementary techniques like PCA and DTL is a viable approach to facilitate comprehension of tradeoffs in poorly-understood design spaces.

Cyber–Physical Systems (CPS) are increasingly used in safety–critical scenarios where ensuring their correct behavior at runtime becomes a crucial task. Therefore, the behavior of the CPS needs to be monitored at runtime so that violations of requirements can be detected. With the inception of edge devices that facilitate runtime analysis at the edge and the increasingly diverse environments that CPS operate in, flexible monitoring approaches are needed that consider the data that needs to be monitored and the analyses performed on that data. In this paper, we propose AMon, a flexible adaptive monitoring framework that supports the specification and validation of monitoring adaptation rules, using a domain-specific language. Based on these rules, AMon automatically generates code for direct deployment onto devices. We evaluated AMon by applying it to TurtleBot Robots and a fleet of Unmanned Aerial Vehicles. Furthermore, we conducted a user study assessing the understandability and ease of use of our language. Results show that creating multiple adaptation rules with our DSL is feasible with minimal effort, and that adaptive monitoring can reduce the amount of runtime data transmitted from the edge device according to the current state of the system and its monitoring needs.

Self-adaptive systems commonly operate in heterogeneous contexts and need to consider multiple quality attributes. Human stakeholders often express their quality preferences by defining utility functions, which are used by self-adaptive systems to automatically generate adaptation plans. However, the adaptation space of realistic systems is large and it is obscure how utility functions impact the generated adaptation behavior, as well as structural, behavioral, and quality constraints. Moreover, human stakeholders are often not aware of the underlying tradeoffs between quality attributes. To address this issue, we present an approach that uses machine learning techniques (dimensionality reduction, clustering, and decision tree learning) to explain the reasoning behind automated planning. Our approach focuses on the tradeoffs between quality attributes and how the choice of weights in utility functions results in different plans being generated. We help humans understand quality attribute tradeoffs, identify key decisions in adaptation behavior, and explore how differences in utility functions result in different adaptation alternatives. We present two systems to demonstrate the approach’s applicability and consider its potential application to 24 exemplar self-adaptive systems. Moreover, we describe our assessment of the tradeoff between the information reduction and the amount of explained variance retained by the results obtained with our approach.

Self-adaptive systems typically operate in heterogeneous environments and need to optimize their behavior based on a variety of quality attributes to meet stakeholders' needs. During adaptation planning, these quality attributes are considered in the form of constraints, describing requirements that must be fulfilled, and utility functions, which are used to select an optimal plan among several alternatives. Up until now, most automated planning approaches are not designed to adapt quality attributes, their priorities, and their trade-offs at run time. Instead, both utility functions and constraints are commonly defined at design time. There exists a clear lack of run-time mechanisms that support their adaptation in response to changes in the environment or in stakeholders' preferences. In this paper, we present initial work that combines automated planning and adaptation of quality attributes to address this gap. The approach helps to semi-automatically adjust utility functions and constraints based on changes at run time. We present a preliminary experimental evaluation that indicates that our approach can provide plans with higher utility values while fulfilling changed or added constraints. We conclude this paper with our envisioned research outlook and plans for future empirical studies.

When designing complex automotive systems in practice, employed technologies and architectural decisions need to reflect business goals. While the software architecture community has acknowledged the need to align business goals with architectural decisions, there is a lack of practical approaches to achieve this alignment. In this paper, we intend to close this gap by providing a systematic approach for architecture-business alignment. The approach describes how to align architecture with business concerns by eliciting goals, identifying quality attributes, and deriving architectural tactics. We iteratively developed and evaluated the approach together with an international automotive manufacturer. We show the application of the proposed approach within our participating company leveraging a use case related to software-over-the-air technologies. The proposed approach is perceived as beneficial by our participants, since it provides a structured mechanism to align architecture and business goals by determining key architectural concerns as quality attributes and tactics.

Large-scale system development companies are increasingly adopting agile methods. While this adoption may improve lead-times, such companies need to balance two trade-offs: (i) the need to have a uniform, consistent development method on system level with the need for specialised methods for teams in different disciplines (e.g., hardware, software, mechanics, sales, support); (ii) the need for comprehensive documentation on system level with the need to have lightweight documentation enabling iterative and agile work. With specialised methods for teams, isolated teams work within larger ecosystems of plan-driven culture, i.e., teams become agile "islands". At the boundaries, these teams share knowledge which needs to be managed well for a correct system to be developed. While it is useful to support diverse and specialised methods, it is important to understand which islands are repeatedly encountered, the reasons or factors triggering their existence, and how best to handle coordination between them. Based on a multiple case study, this work presents a catalogue of islands and the boundary objects between them. We believe this work will be beneficial to practitioners aiming to understand their ecosystems and researchers addressing communication and coordination challenges in large-scale development.

Traceability is crucial for many activities in software and systems engineering including monitoring the development progress, and proving compliance with standards. In practice, the use and maintenance of trace links are challenging as artifacts undergo constant change, and development takes place in distributed scenarios with multiple collaborating stakeholders. Although traceability management in general has been addressed in previous studies, there is a need for empirical insights into the collaborative aspects of traceability management and how it is situated in existing development contexts. The study reported in this paper aims to close this gap by investigating the relation of collaboration and traceability management, based on an understanding of characteristics of the development effort. In our multiple exploratory case study, we conducted semi-structured interviews with 24 individuals from 15 industrial projects. We explored which challenges arise, how traceability management can support collaboration, how collaboration relates to traceability management approaches, and what characteristics of the development effort influence traceability management and collaboration. We found that practitioners struggle with the following challenges: (1) collaboration across team and tool boundaries, (2) conveying the benefits of traceability, and (3) traceability maintenance. If these challenges are addressed, we found that traceability can facilitate communication and knowledge management in distributed contexts. Moreover, there exist multiple approaches to traceability management with diverse collaboration approaches, i.e., requirements-centered, developer-driven, and mixed approaches. While traceability can be leveraged in software development with both agile and plan-driven paradigms, a certain level of rigor is needed to realize its benefits and overcome challenges. To support practitioners, we provide principles of collaborative traceability management. The main contribution of this paper is empirical evidence of how culture, processes, and organization impact traceability management and collaboration, and principles to support practitioners with collaborative traceability management. We show that collaboration and traceability management have the potential to be mutually beneficial—when investing in one, also the other one is positively affected.

The need to support software architecture evolution has been well recognized, even more since the rise of agile methods. However, assuring the conformance between architecture descriptions and the implementation remains challenging. Inconsistencies emanate among multiple architecture descriptions, and between architecture descriptions and code. As a consequence, architecture descriptions are not always trusted and used to the extent that their authors wish for. In this paper, we present two surveys with 93 and 72 participants to examine architectural inconsistencies, with a focus on how they evolve over time and can be mitigated using practical guidelines. We identified the importance of capturing emerging elements to keep the architecture description consistent with the implementation, and consider the current-state and future-state architecture separately. Consequences of inconsistencies typically arise at later stages, especially if an architecture description concerns multiple teams. Our guidelines suggest to limit the upfront architecture to stable decisions, while paying attention to concerns that matter across team borders. In the ideal case, companies should aim to integrate architects into the teams to capture emerging aspects with time.

Practitioners struggle with creating and evolving an architecture when developing complex and safety-critical systems in large-scale agile contexts. A key issue is the trade-off between upfront planning and flexibility to embrace change. In particular, the coordination of interfaces is an important challenge, as interfaces determine and regulate the exchange of information between components, subsystems, and systems, which are often developed by multiple teams. In a fast-changing environment, boundary objects between teams can provide the sufficient stability to align software or systems, while maintaining a sufficient degree of autonomy. However, a better understanding of interfaces as boundary objects is needed to give practical guidance. This paper presents an exploratory case study with an automotive OEM to identify characteristics of different interfaces, from non-critical interfaces that can be changed frequently and quickly, to those that are critical and require more stability and a rigorous change process. We identify what dimensions impact how interfaces are changed, what categories of interfaces exist along these dimensions, and how categories of interfaces change over time. We conclude with suggestions for practices to manage the different categories of interfaces in large-scale agile development.

Summary Agile methods are increasingly introduced in automotive companies in the attempt to become more efficient and flexible in the system development. The adoption of agile practices influences communication between stakeholders and makes companies rethink the management of artifacts and documentation like requirements, safety compliance documents, and architecture models. Practitioners aim to reduce irrelevant documentation but face a lack of guidance to determine what artifacts are needed and how they should be managed. This paper presents artifacts, challenges, guidelines, and practices for the continuous management of systems engineering artifacts in automotive based on a theoretical and empirical understanding of the topic. In collaboration with 53 practitioners from six automotive companies, we conducted a design‐science study involving interviews, a questionnaire, focus groups, and practical data analysis of a systems engineering tool. The guidelines suggest the distinction between artifacts that are shared among different actors in a company (boundary objects) and those that are used within a team (locally relevant artifacts). We propose an analysis approach to identify boundary objects and three practices to manage systems engineering artifacts in industry.

Robotic applications are typically realized using ad hoc and domain-specific solutions, which challenges the engineering and cross-project reuse of such applications. Especially in complex scenarios, where self-adaptive robots collaborate among themselves or with humans, the effective and systematic engineering of such applications is becoming increasingly important. Such scenarios require decentralized software architectures that foster fault-tolerant ways of managing large teams of (possibly) heterogeneous robots. To the best of our knowledge, no existing architecture for robot applications supports decentralized and self-adaptive collaboration. To address this gap, we conducted a design science study with 21 practitioners and experts in the field of robotics to develop an architecture fulfilling these requirements through several iterations. We present SERA, an architecture for robot applications that supports human-robot collaboration, as well as adaptation and coordination of single- and multi-robot systems in a decentralized fashion. SERA is based on layers that contain components that manage the adaptation at different levels of abstraction and communicate through well-defined interfaces. We successfully validated SERA by considering a set of real scenarios, by both using simulators and real robots, by involving robotic experts, and by benchmarking it with state-of-the-art solutions.

Automotive companies increasingly include proven agile methods in their plan-driven system development. The adoption of agile methods impacts not only the way individuals collaborate, but also the management of artifacts like requirements, test cases, safety documentation, and models. While practitioners aim to reduce unnecessary documentation, there is a lack of guidance for automotive companies with respect to what artifacts are needed and how to manage them. To close this knowledge gap and create practical guidelines, we conducted a design-science study together with 53 practitioners from six automotive companies. Using interviews, surveys, and focus groups, we analyzed categories of artifacts and practical challenges to create applicable guidelines to collaboratively manage artifacts in agile automotive contexts. Our findings indicate that different practices are required to manage artifacts that are shared among different teams within the company (boundary objects) and those that are relevant within a specific team (locally relevant artifacts).

T-Reqs is a text-based requirements management solution based on the git version control system. It combines useful conventions, templates and helper scripts with powerful existing solutions from the git ecosystem and provides a working solution to address some known requirements engineering challenges in large-scale agile system development. Specifically, it allows agile cross-functional teams to be aware of requirements at system level and enables them to efficiently propose updates to those requirements. Based on our experience with T-Reqs, we i) relate known requirements challenges of large-scale agile system development to tool support; ii) list key requirements for tooling in such a context; and iii) propose concrete solutions for challenges.

Traceability is an important concern for numerous software engineering activities. Establishing traceability links is a challenging and cost-intensive task, which is uneconomical without suitable strategies for maintaining high link quality. Current approaches to Traceability Management (TM), however, often make important assumptions and choices without ensuring that the consequences and implications for traceability maintenance are feasible and desirable in practice.

Traceability and trace link management are important for various reasons, including managing knowledge about a complex software system, monitoring the progress of its development, and proving that it is developed in accordance to regulations. However, it is difficult to maintain and use trace links in real-world projects where artifacts undergo constant change and multiple stakeholders are involved. In this paper, we extend the current body of knowledge on traceability management by regarding its collaborative aspects in an industrial setting. Based on 15 industrial cases and semi-structured interviews with 24 practitioners, we identify challenges involved in collaborative traceability management, and how traceability management can be used to enable collaboration. Our findings show that main challenges are boundaries between organizations and tools, a lack of common goals and responsibilities, and the difficulty of collaboratively maintaining trace links. We also identify traceability as an important facilitator for communication and knowledge management across these boundaries.

To meet end-user performance expectations, precise performance requirements are needed during development and testing, e.g., to conduct detailed performance and load tests. However, in practice, several factors complicate performance requirements elicitation: lacking skills in performance requirements engineering, outdated or unavailable functional specifications and architecture models, the specification of the system's context, lack of experience to collect good performance requirements in an industrial setting with very limited time, etc. From the small set of available non-functional requirements engineering methods, no method exists that alone leads to precise and complete performance requirements with feasible effort and which has been reported to work in an industrial setting. In this paper, we present our experiences in combining existing requirements engineering methods into a performance requirements method called PROPRE. It has been designed to require no up-to-date system documentation and to be applicable with limited time and effort. We have successfully applied PROPRE in an industrial case study from the process automation domain. Our lessons learned show that the stakeholders gathered good performance requirements which now improve performance testing.