Current software development practices are increasingly based on using both COTS and legacy components which make such systems prone to security vulnerabilities. The modern practice addressing ever changing conditions, DevOps, promotes frequent software deliveries, however, verification methods artifacts should be updated in a timely fashion to cope with the pace of the process. VeriDevOps, Horizon 2020 project, aims at providing a faster feedback loop for verifying the security requirements and other quality attributes of large scale cyber-physical systems. VeriDevOps focuses on optimizing the security verification activities, by automatically creating verifiable models directly from security requirements formulated in natural language, using these models to check security properties on design models and then generating artefacts such as, tests or monitors that can be used later in the DevOps process. The main drivers for these advances are: Natural Language Processing, a combined formal verification and model-based testing approach, and machine-learning-based security monitors. VeriDevOps is in its initial stage - the project started on 1.10.2020 and it will run for three years. In this paper we will present the major conceptual ideas behind the project approach as well as the organizational settings.

With the emergence of Cyber-Physical Systems (CPS), the increasing complexity in development and operation demands for an efficient engineering process. In the recent years DevOps promotes closer continuous integration of system development and its operational deployment perspectives. In this context, the use of Artificial Intelligence (AI) is beneficial to improve the system design and integration activities, however, it is still limited despite its high potential. AIDOaRT is a 3 years long H2020-ECSEL European project involving 32 organizations, grouped in clusters from 7 different countries, focusing on AI-augmented automation supporting modelling, coding, testing, monitoring and continuous development of Cyber-Physical Systems (CPS). The project proposes to apply Model-Driven Engineering (MDE) principles and techniques to provide a framework offering proper AI-enhanced methods and related tooling for building trustable CPSs. The framework is intended to work within the DevOps practices combining software development and information technology (IT) operations. In this regard, the project points at enabling AI for IT operations (AIOps) to auto-mate decision making process and complete system development tasks. This paper presents an overview of the project with the aim to discuss context, objectives and the proposed approach.

Background: Practitioners would like to take action based on software metrics, as long as they find them reliable. Existing literature explores how metrics can be made reliable, but remains unclear if there are other conditions necessary for a metric to be actionable. Context & Method: In the context of a European H2020 Project, we conducted a multiple case study to study metrics' use in four companies, and identified instances where these metrics influenced actions. We used an online questionnaire to enquire about the project participants' views on actionable metrics. Next, we invited one participant from each company to elaborate on the identified metrics' use for taking actions and the questionnaire responses (N=17). Result: We learned that a metric that is practical, contextual, and exhibits high data quality characteristics is actionable. Even a non-actionable metric can be useful, but an actionable metric mostly requires interpretation. However, the more these metrics are simple and reflect the software development context accurately, the less interpretation required to infer actionable information from the metric. Company size and project characteristics can also influence the type of metric that can be actionable. Conclusion: This exploration of industry's views on actionable metrics help characterize actionable metrics in practical terms. This awareness of what characteristics constitute an actionable metric can facilitate their definition and development right from the start of a software metrics program.

In this paper, we present the MegaM@Rt2 ECSEL project and discuss in details our approach for fostering collaboration in this project. We choose to use an internal hackathon approach that focuses on technical collaboration between case study owners and tool/method providers. The novelty of the approach is that we organize the technical workshop at our regular project progress meetings as a challenge-based contest involving all partners in the project. Case study partners submit their challenges related to the project goals and their use cases in advance. These challenges are concise enough to be experimented within approximately 4 hours. Teams are then formed to address those challenges. The teams include tool/method providers, case study owners and researchers/developers from other consortium members. On the hackathon day, partners work together to come with results addressing the challenges that are both interesting to encourage collaboration and convincing to continue further deeper investigations. Obtained results demonstrate that the hackathon approach stimulated knowledge exchanges among project partners and triggered new collaborations, notably between tool providers and use case owners.

Background: Despite the growth in the use of software analytics platforms in industry, little empirical evidence is available about the challenges that practitioners face and the value that these platforms provide. Aim: The goal of this research is to explore the benefits of using a software analytics platform for practitioners managing quality. Method: In a technology transfer project, a software analytics platform was incrementally developed between academic and industrial partners to address their software quality problems. This paper focuses on exploring the value provided by this software analytics platform in two pilot projects. Results: Practitioners emphasized major benefits including the improvement of product quality and process performance and an increased awareness of product readiness. They especially perceived the semi-automated functionality of generating quality requirements by the software analytics platform as the benefit with the highest impact and most novel value for them. Conclusions: Practitioners can benefit from modern software analytics platforms, especially if they have time to adopt such a platform carefully and integrate it into their quality assurance activities.

[Context] Automated test case design and execution at the GUI level of applications is not a fact in industrial practice. Tests are still mainly designed and executed manually. In previous work we have described TESTAR, a tool which allows to set-up fully automatic testing at the GUI level of applications to find severe faults such as crashes or non-responsiveness. [Method] This paper aims at the evaluation of TESTAR with an industrial case study. The case study was conducted at SOFTEAM, a French software company, while testing their Modelio SaaS system, a cloud-based system to manage virtual machines that run their popular graphical UML editor Modelio. [Goal] The goal of the study was to evaluate how the tool would perform within the context of SOFTEAM and on their software application. On the other hand, we were interested to see how easy or difficult it is to learn and implant our academic prototype within an industrial setting. [Results] The effectiveness and efficiency of the automated tests generated with TESTAR can definitely compete with that of the manual test suite. [Conclusions] The training materials as well as the user and installation manual of TESTAR need to be improved using the feedback received during the study. Finally, the need to program Java-code to create sophisticated oracles for testing created some initial problems and some resistance. However, it became clear that this could be solved by explaining the need for these oracles and compare them to the alternative of more expensive and complex human oracles. The need to raise consciousness that automated testing means programming solved most of the initial problems.

[Context] Numerous combinatorial testing techniques are available for generating test cases. However, many of them are never used in practice. [Objective] Considering that learn ability plays a vital role in initial adoption or rejection of a technology, in this paper we aim to investigate the learnability of a combinatorial testing tool in an industrial environment. [Method] A case study research method was designed and conducted, by including i) the definition of learnability measures for test cases models built using a combinatorial testing tool. ii) A training program was also implemented. iii) Qualitative and quantitative evaluation based on a three-level strategy was carried out (Reaction, Learning, and Performance). [Results] At the first level, the tool was perceived as easy to learn by the trainees (from a five-point ordinal scale). However, at the second level, during hands-on learning, it changed slightly: According to the working diaries, there were major difficulties. At third level, analyzing the learning curve of each trainee, we observe that semantic errors made per each subject were reduced slightly over the time.

Software maintenance and support services are key factors to the customer perception of software product quality. The overall goal of FastFix is to provide developers with a real-time maintenance environment that increases efficiency and reduces costs, improving accuracy in identification of failure causes and facilitating their resolution. To achieve this goal, FastFix observes application execution and user interaction at runtime. We give an overview of the functionality of FastFix and present one of its main application scenarios.

This work considers the case of system maintenance where systems are already deployed and for which some faults or security issues were not detected during the testing phase. We propose an approach based on control theory that allows for automatic generation of maintenance fixes. This approach disables faulty or vulnerable system functionalities and requires to instrument the system before deployment so that it can later be monitored and interact with a supervisor at runtime. This supervisor ensures some property designed after deployment in order to avoid future executions of faulty or vulnerable system functionalities. This property corresponds to a set of safe behaviors described as a Finite State Machine. The computation of supervisors can be performed automatically, relying on a sound Supervisory Control Theory. We first introduce some basic notions of Supervisory Control theory, then we present and illustrate our approach which also relies on automatic models extraction and instrumentation.

Security inspections, especially in the early development stage, are becoming increasingly important for bringing security-relevant aspects into software systems. Nowadays, such inspections often do not focus in detail on security. The well-known and approved benefits of inspections do not exploit their full potential regarding security. Thus, we have developed the Security Goal Indicator Tree (SGIT) for eliminating existing shortcomings. SGITs are a new approach for modeling and checking security-relevant aspects during the entire software development lifecycle. This article describes the modeling of such security-goal-based trees as part of requirements engineering. Initial experience was gathered from creating SGITs in an industrial environment. After the probands of our industry partner received training on existing security models, the necessary knowledge for creating security models was collected and applied. This resulted in three context-specific SGITs discussed in this article.