• Based on the results of a series of workshops with industrial practitioners, this study proposes the use of causal models as a supplement to natural language requirements for specifying software with ML components. • The paper provides a demonstration of a proposed causality-driven development concept on an industrial use case on anomaly detection in power systems. • The paper reports on initial results from laboratory experiments that indicate positive effects of the use of causal models during software development on the performance and robustness of a trained ML model for anomaly detection in an industrial prototyping setting. Unlike conventional software systems, where rules are explicitly defined to specify the desired behaviour, software components that incorporate machine learning (ML) infer such rules as associations from data. Requirements Engineering (RE) provides methods and tools for specifying the desired behaviour as structured natural language. However, the inherent ambiguity of natural language can make these specifications difficult to interpret. Moreover, it is challenging in RE to establish a clear link between the specified desired behaviour and data requirements necessary for training and validating ML models. In this paper, we explore the use of causal models to address this gap in RE. Through an exploratory case study, we found that causal models, represented as directed acyclic graphs (DAGs), support the collaborative discovery of an ML system’s operational context from a causal perspective. We also found that causal models can serve as part of the requirements specification for ML models because they encapsulate both data and model requirements needed to achieve the desired causal behaviour. We introduce a concept for causality-driven development , in which we show that data and model requirements, as well as a causal description of the operational context, can be discovered iteratively using graphical causal models. We demonstrate this approach using an industrial use case on anomaly detection with ML.

Data annotation, the systematic labeling of raw data (e.g., images, text) [1] , is foundational to the training of machine learning (ML) models, particularly in supervised learning. While data’s importance is clear, the specific processes and requirements for how this data should be annotated, appear inconsistently defined or informal within existing ML software system (MLS) development methodologies [2] . The effective specification of data annotation requirements, the challenges involved, and the traceability from system requirements to annotation activities represent critical considerations in the ML development lifecycle. Understanding these aspects is pertinent for AI/ML engineers and data scientists, requirements engineers, and organizations developing AI solutions.

The design of complex distributed systems typically follows a hierarchical process, supported by highly specialized views for decomposing the design task. Requirements and architec-ture often evolve simultaneously, requiring an architectural framework that supports integrated and collaborative design, including non-functional requirements and quality views. The framework must ensure the traceability of design decisions in order to build safety cases. Integrating requirements into software development is vital for aligning intended functionality with implemented code. However, extracting data from semi-formal requirements and maintaining alignment poses challenges due to its ambiguity and variability making extracting consistent information challenging. Aligning these requirements with other project artifacts can also be difficult due to interpretation differences, often requiring manual effort and leading to complexity and potential inconsistencies in development [1].

Abstract Driving automation systems, including autonomous driving and advanced driver assistance, are an important safety-critical domain. Such systems often incorporate perception systems that use machine learning to analyze the vehicle environment. We explore new or differing topics and challenges experienced by practitioners in this domain, which relate to requirements engineering (RE), quality, and systems and software engineering. We have conducted a semi-structured interview study with 19 participants across five companies and performed thematic analysis of the transcriptions. Practitioners have difficulty specifying upfront requirements and often rely on scenarios and operational design domains (ODDs) as RE artifacts. RE challenges relate to ODD detection and ODD exit detection, realistic scenarios, edge case specification, breaking down requirements, traceability, creating specifications for data and annotations, and quantifying quality requirements. Practitioners consider performance, reliability, robustness, user comfort, and—most importantly—safety as important quality attributes. Quality is assessed using statistical analysis of key metrics, and quality assurance is complicated by the addition of ML, simulation realism, and evolving standards. Systems are developed using a mix of methods, but these methods may not be sufficient for the needs of ML. Data quality methods must be a part of development methods. ML also requires a data-intensive verification and validation process, introducing data, analysis, and simulation challenges. Our findings contribute to understanding RE, safety engineering, and development methodologies for perception systems. This understanding and the collected challenges can drive future research for driving automation and other ML systems.

Abstract The development and operation of critical software that contains machine learning (ML) models requires diligence and established processes. Especially the training data used during the development of ML models have major influences on the later behaviour of the system. Runtime monitors are used to provide guarantees for that behaviour. Runtime monitors for example check that the data at runtime is compatible with the data used to train the model. In a first step towards identifying challenges when specifying requirements for training data and runtime monitors, we conducted and thematically analysed ten interviews with practitioners who develop ML models for critical applications in the automotive industry. We identified 17 themes describing the challenges and classified them in six challenge groups. In a second step, we found interconnection between the challenge themes through an additional semantic analysis of the interviews. We explored how the identified challenge themes and their interconnections can be mapped to different architecture views. This step involved identifying relevant architecture views such as data, context, hardware, AI model, and functional safety views that can address the identified challenges. The article presents a list of the identified underlying challenges, identified relations between the challenges and a mapping to architecture views. The intention of this work is to highlight once more that requirement specifications and system architecture are interlinked, even for AI-specific specification challenges such as specifying requirements for training data and runtime monitoring.

Abstract Good data quality is crucial for any data-driven system’s effective and safe operation. For critical safety systems, the significance of data quality is even higher since incorrect or low-quality data may cause fatal faults. However, there are challenges in identifying and managing data quality. In particular, there is no accepted process to define and continuously test data quality concerning what is necessary for operating the system. This lack is problematic because even safety-critical systems become increasingly dependent on data. Here, we propose a Candidate Framework for Data Quality Assessment and Maintenance (CaFDaQAM) to systematically manage data quality and related requirements based on design science research. The framework is constructed based on an advanced driver assistance system (ADAS) case study. The study is based on empirical data from a literature review, focus groups, and design workshops. The proposed framework consists of four components: a Data Quality Workflow, a List of Data Quality Challenges, a List of Data Quality Attributes, and Solution Candidates. Together, the components act as tools for data quality assessment and maintenance. The candidate framework and its components were validated in a focus group.

Software that contains machine learning algorithms is an integral part of automotive perception, for example, in driving automation systems. The development of such software, specifically the training and validation of the machine learning components, requires large annotated datasets. An industry of data and annotation services has emerged to serve the development of such data-intensive automotive software components. Wide-spread difficulties to specify data and annotation needs challenge collaborations between OEMs (Original Equipment Manufacturers) and their suppliers of software components, data, and annotations.This paper investigates the reasons for these difficulties for practitioners in the Swedish automotive industry to arrive at clear specifications for data and annotations. The results from an interview study show that a lack of effective metrics for data quality aspects, ambiguities in the way of working, unclear definitions of annotation quality, and deficits in the business ecosystems are causes for the difficulty in deriving the specifications. We provide a list of recommendations that can mitigate challenges when deriving specifications and we propose future research opportunities to overcome these challenges. Our work contributes towards the on-going research on accountability of machine learning as applied to complex software systems, especially for high-stake applications such as automated driving.

The VEDLIoT project aims to develop energy-efficient Deep Learning methodologies for distributed Artificial Intelligence of Things (AIoT) applications. During our project, we propose a holistic approach that focuses on optimizing algorithms while addressing safety and security challenges inherent to AIoT systems. The foundation of this approach lies in a modular and scalable cognitive IoT hardware platform, which leverages microserver technology to enable users to configure the hardware to meet the requirements of a diverse array of applications. Heterogeneous computing is used to boost performance and energy efficiency. In addition, the full spectrum of hardware accelerators is integrated, providing specialized ASICs as well as FPGAs for reconfigurable computing. The project's contributions span across trusted computing, remote attestation, and secure execution environments, with the ultimate goal of facilitating the design and deployment of robust and efficient AIoT systems. The overall architecture is validated on use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. Ten additional use cases are integrated via an open call, broadening the range of application areas.

Artificial intelligence (AI) in its various forms finds more and more its way into complex distributed systems. For instance, it is used locally, as part of a sensor system, on the edge for low-latency high-performance inference, or in the cloud, e.g. for data mining. Modern complex systems, such as connected vehicles, are often part of an Internet of Things (IoT). This poses additional architectural challenges. To manage complexity, architectures are described with architecture frameworks, which are composed of a number of architectural views connected through correspondence rules. Despite some attempts, the definition of a mathematical foundation for architecture frameworks that are suitable for the development of distributed AI systems still requires investigation and study. In this paper, we propose to extend the state of the art on architecture framework by providing a mathematical model for system architectures, which is scalable and supports co-evolution of different aspects for example of an AI system. Based on Design Science Research, this study starts by identifying the challenges with architectural frameworks in a use case of distributed AI systems. Then, we derive from the identified challenges four rules, and we formulate them by exploiting concepts from category theory. We show how compositional thinking can provide rules for the creation and management of architectural frameworks for complex systems, for example distributed systems with AI. The aim of the paper is not to provide viewpoints or architecture models specific to AI systems, but instead to provide guidelines based on a mathematical formulation on how a consistent framework can be built up with existing, or newly created, viewpoints. To put in practice and test the approach, the identified and formulated rules are applied to derive an architectural framework for the EU Horizon 2020 project “Very efficient deep learning in the IoT” (VEDLIoT) in the form of a case study.

Artificial Intelligence (AI), and especially machine learning can be used to find statistical patterns in datasets with thousands of variables with ease. But an understanding of causality is difficult to learn for a machine. For humans however, realising causal relations is often not a difficult process, as we can refer to experience or scientific knowledge. Here we propose the use of structural causal models, represented through direct acyclic graphs, to design, determine, and communicate causal relations hidden beyond the statistical models of an AI. The idea is to make human insight in causal relations explicit and use this knowledge during AI system development. In a joint-industry project we discovered that structural causal models can serve as living boundary objects that facilitate coordination of domain experts, data scientists, systems engineers, and AI experts in AI system development.

The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide range of applications. VEDLIoT offers a complete design flow for Next-Generation IoT devices required for collaboratively solving complex Deep Learning applications across distributed systems. The methods are tested on various use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. VEDLIoT is an H2020 EU project which started in November 2020. It is currently in an intermediate stage with the first results available.