Explainability has been acknowledged as a fundamental requirement for modern information systems. However, there are currently only few guidelines available to assist software professionals in dealing with this requirement and integrating it into systems. More precisely, there is a lack of frameworks and guidelines that help to define and operationalize explainability requirements. To address this need, we present a quality framework that aggregates external dependencies, characteristics of explanations, and evaluation methods to facilitate the analysis, operationalization, and evaluation of explainability requirements. We conducted a literature study to construct the framework and demonstrated its applicability by using it as a guideline for incorporating explanations into an existing navigation system. Finally, we evaluated the quality and effect of the explanations through an experiment within our case study. Our results show that the quality framework is applicable and beneficial in an industrial context and leads to the construction of explanations that increase usage frequency, system acceptance and user satisfaction.

Abstract The growing complexity of software systems and the influence of software-supported decisions in our society sparked the need for software that is transparent, accountable, and trustworthy. Explainability has been identified as a means to achieve these qualities. It is recognized as an emerging non-functional requirement (NFR) that has a significant impact on system quality. Accordingly, software engineers need means to assist them in incorporating this NFR into systems. This requires an early analysis of the benefits and possible design issues that arise from interrelationships between different quality aspects. However, explainability is currently under-researched in the domain of requirements engineering, and there is a lack of artifacts that support the requirements engineering process and system design. In this work, we remedy this deficit by proposing four artifacts: a definition of explainability, a conceptual model, a knowledge catalogue, and a reference model for explainable systems. These artifacts should support software and requirements engineers in understanding the definition of explainability and how it interacts with other quality aspects. Besides that, they may be considered a starting point to provide practical value in the refinement of explainability from high-level requirements to concrete design choices, as well as on the identification of methods and metrics for the evaluation of the implemented requirements.

The growing complexity of software systems and the influence of software-supported decisions in our society awoke the need for software that is transparent, accountable, and trust-worthy. Explainability has been identified as a means to achieve these qualities. It is recognized as an emerging non-functional requirement (NFR) that has a significant impact on system quality. However, in order to incorporate this NFR into systems, we need to understand what explainability means from a software engineering perspective and how it impacts other quality aspects in a system. This allows for an early analysis of the benefits and possible design issues that arise from interrelationships between different quality aspects. Nevertheless, explainability is currently under-researched in the domain of requirements engineering and there is a lack of conceptual models and knowledge catalogues that support the requirements engineering process and system design. In this work, we bridge this gap by proposing a definition, a model, and a catalogue for explainability. They illustrate how explainability interacts with other quality aspects and how it may impact various quality dimensions of a system. To this end, we conducted an interdisciplinary Systematic Literature Review and validated our findings with experts in workshops.

Abstract Software systems are becoming increasingly complex. Their ubiquitous presence makes users more dependent on their correctness in many aspects of daily life. As a result, there is a growing need to make software systems and their decisions more comprehensible, with more transparency in software-based decision making. Transparency is therefore becoming increasingly important as a non-functional requirement. However, the abstract quality aspect of transparency needs to be better understood and related to mechanisms that can foster it. The integration of explanations into software has often been discussed as a solution to mitigate system opacity. Yet, an important first step is to understand user requirements in terms of explainable software behavior: Are users really interested in software transparency and are explanations considered an appropriate way to achieve it? We conducted a survey with 107 end users to assess their opinion on the current level of transparency in software systems and what they consider to be the main advantages and disadvantages of embedded explanations. We assess the relationship between explanations and transparency and analyze its potential impact on software quality. As explainability has become an important issue, researchers and professionals have been discussing how to deal with it in practice. While there are differences of opinion on the need for built-in explanations, understanding this concept and its impact on software is a key step for requirements engineering. Based on our research results and on the study of existing literature, we offer recommendations for the elicitation and analysis of explainability and discuss strategies for the practice.

Software systems are getting more and more complex, with an increasing integration of machine-learning based decisions. The ubiquitous presence of these systems makes users more dependent on them and their correctness in many aspects of daily life. Thus, there is a rising need to make software systems and their decisions more comprehensible. This seems to call for more transparency in software-supported decisions. Therefore, transparency requirements have to be understood, elicited and translated to lower level requirements. However, there is a lack of understanding about the requirements engineering process for transparency and how the different roles, e.g., UX designers, data scientists, and other stakeholders, have to interact in this process. In order to fill this gap, the requirements engineering process for transparency requirements needs to be thoroughly investigated. For this purpose, I intend to conduct empirical studies with practitioners and other stakeholders to gain a deeper understanding of the process and its associated problems. Based on the results, additional research will be conducted to investigate and propose solutions with the purpose of supporting requirements engineering when transparency is required.

Software systems are getting more and more complex. Their ubiquitous presence makes users more dependent on them and their correctness in many aspects of daily life. Thus, there is a rising need to make software systems and their decisions more comprehensible. This seems to call for more transparency in software-supported decisions. Therefore, transparency is gaining importance as a non-functional requirement. However, the abstract quality aspect of transparency needs to be better understood and related to mechanisms that can foster it. Integrating explanations in software to leverage systems' opacity has been discussed often. Yet, an important first step is to understand user requirements with respect to explainable software behavior: Are users really interested in transparency, and are explanations considered an adequate mechanism to achieve it? We conducted a survey with 107 end-users to assess their opinion on the current status of transparency in software systems, and what they consider main advantages and disadvantages of explanations embedded in software. The overall attitude towards embedded explanations was positive. However, we also identified potential disadvantages. We assess the relation between explanations and transparency and analyze its possible impact on software quality.