The rapid development of artificial intelligence (AI) has led to increasing concerns about the capability of AI systems to make decisions and behave responsibly. Responsible AI (RAI) refers to the development and use of AI systems that benefit humans, society, and the environment while minimising the risk of negative consequences. To ensure responsible AI, the risks associated with AI systems' development and use must be identified, assessed and mitigated. Various AI risk assessment frameworks have been released recently by governments, organisations, and companies. However, it can be challenging for AI stakeholders to have a clear picture of the available frameworks and determine the most suitable ones for a specific context. Additionally, there is a need to identify areas that require further research or development of new frameworks, as well as updating and maintaining existing ones. To fill the gap, we present a mapping study of 16 existing AI risk assessment frameworks from the industry, governments, and non-government organizations (NGOs). We identify key characteristics of each framework and analyse them in terms of RAI principles, stakeholders, system lifecycle stages, geographical locations, targeted domains, and assessment methods. Our study provides a comprehensive analysis of the current state of the frameworks and highlights areas of convergence and divergence among them. We also identify the deficiencies in existing frameworks and outlines the essential characteristics of a concrete and connected framework AI risk assessment (C2AIRA) framework. Our findings and insights can help relevant stakeholders choose suitable AI risk assessment frameworks and guide the design of future frameworks towards concreteness and connectedness.

Abstract There is a growing awareness of the importance of human values in software systems. However, limited tools are available to support the integration of human values during software development. Most of these tools are focused on concepts related to specific, well-known human values (e.g., privacy, security) in software engineering. This paper aims to (partially) address this gap by developing a human values dashboard. We conducted a multi-stage study to design, implement and evaluate a human values dashboard. First, an exploratory study was conducted by interviewing 15 software practitioners to investigate the possibility of using a human values dashboard to help address human values in software development, its potential benefits, and required features. Second, we experimented with four Machine Learning approaches to detect the presence of human values in issue discussions. We used the best approach to develop a human values dashboard for software development. The dashboard displays whether any human values are present in each issue discussion. Finally, we interviewed ten different practitioners to investigate the usefulness of the dashboard in practice. This study found that the human values dashboard could help raise awareness, focus attention, and prioritise issues based on the presence of values. This study also identified two potential challenges to the adoption of the dashboard. First, the possible incorrect issues description that can mislead the automated values identification in the dashboard. Second, the lack of willingness of a company to adopt the dashboard.

The growing diffusion of software in society and its influence on people demands from its creators that their work carefully considers human values such as transparency, social responsibility, and equality. But how do software practitioners address human values in software engineering practice? We interviewed 31 software practitioners from two organizations, each having a strong values framework, with the aim to understand: (a) practitioners’ perceptions of human values and their role in software engineering; (b) practices that practitioners use to address human values in software; and (c) challenges they face during this process. We report our findings from two contrasting case organizations on how practitioners “engineer” values in their unique organizational settings. We found evidence that organizational culture significantly contributes to how values are addressed in software. We summarize recommendations from the practitioners to support proactive engineering of values-conscious software.

Agile methods are predominantly focused on delivering business values. But can Agile methods be adapted to effectively address and deliver human values such as social justice, privacy, and sustainability in the software they produce Human values are what an individual or a society considers important in life. Ignoring these human values in software can pose difficulties or risks for all stakeholders (e.g., user dissatisfaction, reputation damage, financial loss). To answer this question, we selected the Scaled AgileFramework (SAFe), one of the most commonly used Agile methods in the industry, and conducted a qualitative case study to identify possible intervention points within SAFe that are the most natural to address and integrate human values in software. We present five high-level empirically-justified sets of interventions in SAFe: artefacts, roles, ceremonies, practices, and culture. We elaborate how some currentAgile artefacts (e.g., user story), roles (e.g., product owner), ceremonies (e.g., stand-up meeting), and practices (e.g., business-facing testing) in SAFe can be modified to support the inclusion of human values in software. Further, our study suggests new and exclusive values-based artefacts (e.g., legislative requirement), ceremonies (e.g., values conversation), roles (e.g., values champion), and cultural practices (e.g., induction and hiring) to be introduced in SAFe for this purpose. Guided by our findings, we argue that existingAgile methods can account for human values in software delivery with some evolutionary adaptations.

Background: There is a growing awareness of the importance of human values (e.g., inclusiveness, privacy) in software systems. However, there are no practical tools to support the integration of human values during software development. We argue that a tool that can identify human values from software development artefacts and present them to varying software development roles can (partially) address this gap. We refer to such a tool as human values dashboard. Further to this, our understanding of such a tool is limited. Aims: This study aims to (1) investigate the possibility of using a human values dashboard to help address human values during software development, (2) identify possible benefits of using a human values dashboard, and (3) elicit practitioners' needs from a human values dashboard. Method: We conducted an exploratory study by interviewing 15 software practitioners. A dashboard prototype was developed to support the interview process. We applied thematic analysis to analyse the collected data. Results: Our study finds that a human values dashboard would be useful for the development team (e.g., project manager, developer, tester). Our participants acknowledge that development artefacts, especially requirements documents and issue discussions, are the most suitable source for identifying values for the dashboard. Our study also yields a set of high-level user requirements for a human values dashboard (e.g., it shall allow determining values priority of a project). Conclusions: Our study suggests that a values dashboard is potentially used to raise awareness of values and support values-based decision-making in software development. Future work will focus on addressing the requirements and using issue discussions as potential artefacts for the dashboard.

Failure to account for human values in software (e.g., equality and fairness) can result in user dissatisfaction and negative socio-economic impact. Engineering these values in software, however, requires technical and methodological support throughout the development life cycle. This paper investigates to what extent top Software Engineering (SE) conferences and journals have included research on human values in SE. We investigate the prevalence of human values in recent (2015 -- 2018) publications in these top venues. We classify these publications, based on their relevance to different values, against a widely used value structure adopted from the social sciences. Our results show that: (a) only a small proportion of the publications directly consider values, classified as directly relevant publications; (b) for the majority of the values, very few or no directly relevant publications were found; and (c) the prevalence of directly relevant publications was higher in SE conferences compared to SE journals. This paper shares these and other insights that may motivate future research on human values in software engineering.

Software failures that demonstrate violations of human values can result in financial losses, reputation damages and social implications. Therefore, integrating human values into software is vital to satisfy stakeholder needs. However, developing methodological approaches that allow systematic integration of human values throughout the software development life cycle is an open challenge. This paper proposes the Continual Value(s) Assessment (CVA) framework that uses extended goal and feature modeling techniques to support systematic integration, tracing and evaluation of human values in software systems. The CVA framework prescribes (i) brainstorming of value implications of system features based on conventional system artefacts and (ii) the expansion of the existing set of system features to better serve stakeholder values expectations. In a pilot study, we use an emergency alarm system for the elderly to demonstrate the feasibility of the framework. We further discuss the challenges we faced while applying the framework and present the lessons learned from the pilot study.

Software has become an integral part of human life and while Software Engineering (SE) is aware of the values of security or privacy that relate to software quality, to date it is unable to account for the majority of the other human values (e.g., social justice, trust, freedom). Engineering human values into software is now emerging as an area of study but requires theoretical and methodological support from SE research and practice. A part of the challenge of integrating human values into software is related to tracing human values throughout the Software Development Life Cycle (SDLC). However, the number of studies in the domain of Human Values and SE is very limited. Moreover, there has been no previous study or practice performed on Human Values Traceability in SE. Therefore, this research study examines existing techniques and their applicability to trace human values in software. First, we explore the existing techniques that have the potential to trace human values in software. This exploration is extended from SE to other related domains such as Requirement Engineering, Software Design, Values in Design and Software Traceability. Identified techniques are then evaluated to identify the extent to which they can be used to trace human values in different phases of the SDLC. Our preliminary results found goal modeling and goal-centric traceability as promising techniques to initiate human values traceability in software. Therefore, we apply goal modeling to a case study to explore the ways this technique can be used and potentially improved to trace human values. Further, the improved technique will be evaluated through another case study.

Software has become an integral part of human life. This gives rise to the need of developing software that respects human values such as transparency, fairness and privacy. Software that compromises on human values (e.g. privacy) can affect people's reputation and impinges on their ability to function in society with the usual freedom and autonomy. Integrating human values into software is, however, a challenging task due to its imprecise and subjective nature. Enforcing regulations is one way to make software development considerate of the desired standards and values. The European Union's General Data Protection Regulation (GDPR) on software is one such effort to protect EU citizens' data and personal information. GDPR prescribes data protection principles and data subject rights mainly to protect user privacy. Looking beyond privacy, we studied GDPR to identify the extent to which it covers human values. We mapped GDPR's data protection principles and data subject rights to a widely accepted human values structure adopted from social sciences. Our results show that GDPR addresses not only privacy but also several other human values including power, security and universalism. Moreover, fairness and transparency stand out as the most value-conscious principles prescribed in GDPR.

Software influences several aspects of people's lives and therefore needs to reflect their values. However, existing software engineering methods fail to account for human values, which may result in breaching those values in software and, therefore, dissatisfaction of users and loss of profit and reputation. To avoid such negative consequences, human values need to be integrated -- in a verifiable way -- into software. We refer to this as Operationalizing Human Values in Software. But this is not easy to achieve due to three main obstacles: first, human values are hard to define in a way that can be put into practice; second, existing software design decisions are mainly ignorant of values; finally, values are hard to determine and quantify in software. This paper aims to establish a research roadmap for overcoming these obstacles. The proposed roadmap focuses on (i) establishing practical definitions for human values, (ii) integrating values into software design, and (iii) measuring values in the software development life cycle.