Security Chaos Engineering (SCE) is a proactive approach to identify vulnerabilities and enhance security of systems. It embraces continuous security experimentation to build confidence in the capability of systems to withstand malicious conditions. Different SCE techniques are proposed for enhancing the resilience of software systems. The diversity of SCE techniques indicates the need for their collective analysis to uncover valuable practices and potential research opportunities. To fulfill this need, we consolidate and unify the knowledge on SCE practices through a systematic literature review. The results show that there has been limited and unsystematic investigation of SCE by the community, highlighting the importance of creating and promoting guidelines for SCE practices. Therefore, we create SCENE, a comprehensive set of guidelines for systematically reporting SCE. The goal is to support the clarity, consistency, and reproducibility of SCE practices. SCENE guidelines are evaluated by cybersecurity practitioners and active researchers in the field, and is mapped to established methodological guidelines. The results indicates that SCENE is perceived positive in terms of usefulness, understandability, practicality, and completeness. SCENE is also found to complement established experimental reporting guidelines and bridge the gap between academic studies and industrial use.

In recent years, interest in the potential influence of architectural smells on software maintainability has grown. Yet, empirical evidence directly linking maintainability quality attributes — such as modularity and testability — with architectural smells is scarce. This study analyzes seven architectural smells across 378 versions of eight open-source projects. We developed a tool to gather data on architectural smells, associated projects, and package-level quality attributes. The empirical findings reinforce that certain architectural smells indeed correlate with specific quality attributes - a notion previously backed merely by argument. Most architectural smells negatively correlated with project-level testability but not project-level modularity. While there is not a consistent negative correlation with testability at the package level, many smells display a pronounced negative association with the maintainability quality attributes. • The size of projects has a significant impact on both modularity and testability. • Architectural smells show a negative relationship with testability. • Addressing Dense Structure smell can improve modularity and testability. • Addressing God Component smell can improve modularity and testability. • Addressing Scattered Functionality smell can improve modularity and testability.

Software engineering is becoming more and more distributed. Developers and other stakeholders are often located in different locations, departments, and countries and operating within different time zones. Most online software design and modeling tools are not adequate for distributed collaboration since they do not support awareness and lack features for effective communication. The aim of our research is to support distributed software design activities in Virtual Reality (VR). Using design science research methodology, we design and evaluate a tool for collaborative design in VR. We evaluate the collaboration efficiency and recall of design information when using the VR software design environment compared to a non-VR software design environment. Moreover, we collect the perceptions and preferences of users to explore the opportunities and challenges that were incurred by using the VR software design environment. We find that there is no significant difference in the efficiency and recall of design information when using the VR compared to the non-VR environment. Furthermore, we find that developers are more satisfied with collaboration in VR. The results of our research and similar studies show that working in VR is not yet faster or more efficient than working on standard desktops. It is very important to improve the interface in VR (gestures with haptics, keyboard and voice input), as confirmed by the difference in results between the first and second evaluation.

Since 1998, the ACM/IEEE 25th International Conference on Model Driven Engineering Languages and Systems (MODELS) has been studying all aspects surrounding modeling in software engineering, from languages and methods to tools and applications. In order to enable empirical studies, the MODELS community developed a need for having examples of models, especially of models used in real software development projects. Such models may be used for a range of purposes, but mostly related to domain analysis and software design (at various levels of abstraction). However, finding such models was very difficult. The most used ones had their origin in academic books or student projects, which addressed “artificial” applications, i.e., were not base on real-case scenarios. To address this issue, the authors of this reflection paper, members of the modeling and of the mining software repositories fields, came together with the aim of creating a dataset with an abundance of modeling projects by mining GitHub. As a scoping of our effort we targeted models represented using the UML notation because this is the lingua franca in practice for software modeling. As a result, almost 100k models from 22k projects were made publicly available, known as the Lindholmen dataset. In this paper, we analyse the impact of our research, and compare this to what we envisioned in 2016. We draw practical lessons gained from this effort, reflect on the perils and pitfalls of the dataset, and point out promising avenues of research. We base our reflection on the systematic analysis of recent research literature, and especially those papers citing our dataset and its associated publications. What we envisioned in the original research when making the dataset available has to a major extent not come true; however, fellow researchers have found alternative uses of the dataset. By understanding the possibilities and shortcomings of the current dataset, we aim to offer the research community i) future research avenues of how the data can be used; and ii) raise awareness of the limitations, not only to point out threats to validity of research, but also to encourage fellow researchers to find ideas to overcome them. Our reflections can also be helpful to researchers who want to perform similar mining efforts.

Abstract Designing software is an activity in which software developers think and make design decisions that shape the structure and behavior of software products. Designing software is one of the least understood software engineering activities. In a collaborative design setting, various types of distances can lead to challenges and effects that potentially affect how software is designed. To contribute to a better understanding of collaborative software design, we investigate how communication gaps caused by social and geographic distances affect its design thinking and the creativity of its discussions. To this end, we conducted a multiple‐case study exploring the design thinking and creativity of colocated and distributed software developers in a collaborative design setting. Compared with colocated developers, distributed developers spend less time on exploring the problem space, which could be related to different sociotechnical challenges, such as lack of awareness and common understanding. Distributed development does not seem to affect the creativity of their activities. Developers engaging in collaborative design need to be aware that problem space exploration is reduced in a distributed setting. Unless distributed teams take compensatory measures, this could adversely affect the development. Regarding the effect distance has on creativity, our results are inconclusive and further studies are needed.

Security Assurance Cases (SAC) are structured arguments and evidence bodies used to reason about the security of a certain system. SACs are gaining focus in the automotive industry, as the needs for security assurance are growing in this domain. However, the state-of-the-arts lack a mature approach able to suit the needs of the automotive industry. In this article, we present CASCADE, an asset-driven approach for creating SAC, which is inspired by the upcoming security standard ISO/SAE-21434 as well as the internal needs of automotive Original Equipment Manufacturers (OEMs). CASCADE also differentiates itself from the state-of-the-art by incorporating a way to reason about the quality of the constructed security assurance case. We created the approach by conducting an iterative design science research study. We illustrate the results using the example case of the road vehicle’s headlamp provided in the ISO standard. We also illustrate how our approach aligns well with the structure and content of the ISO/SAE-21434 standard, hence demonstrating the practical applicability of CASCADE in an industrial context.

The automotive domain has got its own share of advancements in information and communication technology, providing more services and leading to more connectivity. However, more connectivity and openness raise cyber security and safety concerns. Indeed, services that depend on online connectivity can serve as entry points for attacks on different assets of the vehicle. This study explores collaborative ways of selecting response techniques to counter real-time cyber attacks on automotive systems. The aim is to mitigate the attacks more quickly than a single vehicle would be able to do, and increase the survivability chances of the collaborating vehicles. To achieve that, the design science research methodology is employed. As a result, we present RIPOSTE, a framework for collaborative real-time evaluation and selection of suitable response techniques when an attack is in progress. We evaluate the framework from a safety perspective by conducting a qualitative study involving domain experts. The proposed framework is deemed slightly unsafe, and insights into how to improve the overall safety of the framework are provided.

Software is the main driver of added-value in many of the systems that surround us. While its complexity is increasing, so is the diversity of systems driven by software. To meet the challenges emerging from this combination, it is necessary to mobilize increasingly large and heterogeneous multidisciplinary teams, comprising software experts, as well as experts from various domains related to the systems driven by software. Hence, the quality of communication about software between stakeholders of different domains and with different personalities is becoming a key issue for successfully engineering software-intensive systems. The goal of this study, thus, is to investigate the effect of the representation of software design models on the communication of design decisions between stakeholders with diverse personality traits. As a result, this study finds that graphical representations of software design models are better than textual representations in enhancing the communication and increasing the productivity of stakeholders with diverse personalities.

UML is the de facto standard notation for graphically representing software. UML diagrams are used in the analysis, construction, and maintenance of software systems. Mostly, UML diagrams capture an abstract view of a (piece of a) software system. A key purpose of UML diagrams is to share knowledge about the system among developers. The quality of the layout of UML diagrams plays a crucial role in their comprehension. In this paper, we present an automated method for evaluating the layout quality of UML class diagrams. We use machine learning based on features extracted from the class diagram images using image processing. Such an automated evaluator has several uses: (1) From an industrial perspective, this tool could be used for automated quality assurance for class diagrams (e.g., as part of a quality monitor integrated into a DevOps toolchain). For example, automated feedback can be generated once a UML diagram is checked in the project repository. (2) In an educational setting, the evaluator can grade the layout aspect of student assignments in courses on software modeling, analysis, and design. (3) In the field of algorithm design for graph layouts, our evaluator can assess the layouts generated by such algorithms. In this way, this evaluator opens up the road for using machine learning to learn good layouting algorithms. We use machine learning techniques to build (linear) regression models based on features extracted from the class diagram images using image processing. As ground truth, we use a dataset of 600+ UML Class Diagrams for which experts manually label the quality of the layout. This paper makes the following contributions: (1) We show the feasibility of the automatic evaluation of the layout quality of UML class diagrams. (2) We analyze which features of UML class diagrams are most strongly related to the quality of their layout. (3) We evaluate the performance of our layout evaluator. (4) We offer a dataset of labeled UML class diagrams. In this dataset, we supply for every diagram the following information: (a) a manually established ground truth of the quality of the layout, (b) an automatically established value for the layout-quality of the diagram (produced by our classifier), and (c) the values of key features of the layout of the diagram (obtained by image processing). This dataset can be used for replication of our study and others to build on and improve on this work. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board..

Container-based virtualization is gaining popularity in different domains, as it supports continuous development and improves the efficiency and reliability of run-time environments. Different techniques are proposed for monitoring the security of containers. However, there are no guidelines supporting the selection of suitable techniques for the tasks at hand. We aim to support the selection and design of techniques for monitoring container-based virtualization environments. : First, we review the literature and identify techniques for monitoring containerized environments. Second, we classify these techniques according to a set of categories, such as technical characteristic, applicability, effectiveness, and evaluation. We further detail the pros and cons that are associated with each of the identified techniques. As a result, we present CONSERVE, a multi-dimensional decision support framework for an informed and optimal selection of a suitable set of container monitoring techniques to be implemented in different application domains. A mix of eighteen researchers and practitioners evaluated the ease of use, understandability, usefulness, efficiency, applicability, and completeness of the framework. The evaluation shows a high level of interest, and points out to potential benefits.

Vehicles have become complex computer systems with multiple communication interfaces. In the future, vehicles will have even more connections to e.g., infrastructure, pedestrian smartphones, cloud, road-side-units and the Internet. External and physical interfaces, as well as internal communication buses have shown to have potential to be exploited for attack purposes. As a consequence, there is an increase in regulations which demand compliance with vehicle cyber resilience requirements. However, there is currently no clear guidance on how to comply with these regulations from a technical perspective.To address this issue, we have performed a comprehensive threat and risk analysis based on published attacks against vehicles from the past 10 years, from which we further derive necessary security and resilience techniques. The work is done using the SPMT methodology where we identify vital vehicle assets, threat actors, their motivations and objectives, and develop a comprehensive threat model. Moreover, we develop a comprehensive attack model by analyzing the identified threats and attacks. These attacks are filtered and categorized based on attack type, probability, and consequence criteria. Additionally, we perform an exhaustive mapping between asset, attack, threat actor, threat category, and required mitigation mechanism for each attack, resulting in a presentation of a secure and resilient vehicle design. Ultimately, we present the Resilient Shield a novel and imperative framework to justify and ensure security and resilience within the automotive domain.

Context: Designing software is an activity in which software developers think and make design decisions that ultimately shape the structure and behavior of software products. Currently, designing software is one of the least understood activities in which software developers engage. In a collaborative design setting, distances such as geographic, cultural, or social distance can lead to socio-technical challenges that potentially affect the way software is designed.

Today, there is a wide range of techniques to support software comprehension. However, we do not fully understand yet what techniques really help novices, to comprehend a software system. In this paper, we present a master level project course on software evolution, which has a large focus on software comprehension. We collected data about student's experience with diverse comprehension techniques during focus group discussions over the course of two years. Our results indicate that systematic code reading can be supported by additional techniques to guiding reading efforts. Most techniques are considered valuable for gaining an overview and some techniques are judged to be helpful only in later stages of software comprehension efforts.

Motivation: Software visualization can be helpful in comprehending the architecture of large software systems. Traditionally, software visualisation focuses on representing the structural perspectives of systems. In this paper we enrich this perspective by adding the notion of role-stereotype. This rolestereotype carries information about the type of functionality that a class has in the system as well as the types of collaborations with other classes that it typically has.Objective: We propose an interactive visualization called RoleViz, that visualizes system architectures in which architectural elements are annotated with their role-stereotypes.Method: We conducted a user-study in which developers use RoleViz and Softagram (a commercial tool for software architecture comprehension) to solve two separate comprehension tasks on a large open source system. We compared RoleViz against Softagram in terms of participant's: (i) perceived cognitive load, (ii) perceived usability, and (iii) understanding of the system. Result: In total, 16 developers participated in our study. Six of the participants explicitly indicated that visualizing roles helped them complete the assigned tasks. Our observations indicate significant differences in terms of participant's perceived usability and understanding scores.Conclusion: The participants achieved better scores on completing software understanding tasks with RoleViz without any cognitive-load penalty.

Abstract Context Software engineering is a social and collaborative activity. Communicating and sharing knowledge between software developers requires much effort. Hence, the quality of communication plays an important role in influencing project success. To better understand the effect of communication on project success, more in-depth empirical studies investigating this phenomenon are needed. Objective We investigate the effect of using a graphical versus textual design description on co-located software design communication. Method Therefore, we conducted a family of experiments involving a mix of 240 software engineering students from four universities. We examined how different design representations (i.e., graphical vs. textual) affect the ability to Explain, Understand, Recall, and Actively Communicate knowledge. Results We found that the graphical design description is better than the textual in promoting Active Discussion between developers and improving the Recall of design details. Furthermore, compared to its unaltered version, a well-organized and motivated textual design description–that is used for the same amount of time–enhances the recall of design details and increases the amount of active discussions at the cost of reducing the perceived quality of explaining.

To fulfill the contractual requirement of the Sweden Compact agreement, the following funding note has to be added and placed in the Funding section of the original article: Open access funding provided by University of Gothenburg .

A recurring theme in discussions about the adoption of Model-Based Engineering (MBE) is its effectiveness. This is because there is a lack of empirical assessment of the processes and (tool-)use of MBE in practice. We conducted a multiple-case study by observing 2 two-month MBE projects from which software for a Mars rover were developed. We focused on assessing the distribution of the total software development effort over different development activities. Moreover, we observed and collected challenges reported by the developers during the execution of projects. We found that the majority of the effort is spent on the collaboration and communication activities. Furthermore, our inquiry into challenges showed that tool-related challenges are the most encountered.

The process of software understanding often requires developers to consult both high- and low-level software artifacts (i.e. models and code). The creation and persistence of such artifacts often take place in different environments, as well as seldom in one single environment. In both cases, software models and code fragments are viewable separately making the workspace overcrowded with many opened interfaces and tabs. In such a situation, developers might lose the big picture and spend unnecessary effort on navigation and locating the artifact of interest. To assist program comprehension and tackle the problem of software navigation, we present OctoBubbles, a multi-view interactive environment for concurrent visualization and synchronization of software models and code. A preliminary evaluation of OctoBubbles with 15 professional developers shows a high level of interest, and points out to potential benefits. Furthermore, we present a future plan to quantitatively investigate the effectiveness of the environment.

One argument in the discussion about the adoption of UML in industry is the supposedly large effort it takes to do modeling. Our study explores how the creation of UML models can be understood to consist of different cognitive activities: (i) designing: thinking about the design (ideation, key-design decision making), (ii) notation expression: expressing a design in a modeling notation and (iii) layouting: the spatial organization of model elements in a diagram. We explain that these different subactivities relate to different short-term and long-term benefits of modeling. In this study we present two controlled experiments with a total of 100 subjects creating models for a small system. In these experiments we focus on software models as represented through UML class diagram. Our results show that at least 56% of the effort spent on creating a class model is actually due to designing. Notation expression is around 41% of the model creation effort and layouting is in the order of 3%. This finding suggests that a significant part of creating models is devoted to design thinking about the problem.

Context: This research is positioned in the field of methods for creating software design and the teaching thereof. Goal: The goal of this research is to study the effects of using a collection of examples for creating a software design. Method: We ran a controlled experiment for evaluating the use of a broad collection of examples for creating software designs by software engineering students. In this study, we focus on software designs as represented through UML class diagrams. The treatment is the use of the collection of examples. These examples are offered via a searchable repository. The outcome variable we study is the quality of the design (as assessed by a group of experts). After this, all students were offered the opportunity to improve their design using the collection of examples. We ran a post-assignment questionnaire to collect qualitative data about the experience of the participants. Results: Considering six quality attributes measured by experts, our results show that: 1) the models of the students who used examples are 18% better than those of who did not use examples. 2) the models of the students who did not use examples for constructing became 19% better after updating their models using examples. We complement our statistical analysis with insights from the post assignment questionnaire. Also, we observed that students are more confident about their design when they use examples. Conclusion: Students deliver better software designs when they use a collection of example software designs.