Context: Interest in diversity in software development has significantly increased in recent years. Reporting on diversity in software projects can enhance user trust and assist regulators in evaluating adoption. Recent AI directives include clauses that mandate diversity information during development, highlighting the growing interest of public regulators. However, current documentation often neglects diversity in favor of technical features, partly due to a lack of tools for its description and annotation. Objectives: This work introduces the Software Diversity Card, a structured approach to document and share diversity-related aspects within software projects. It aims to profile the various teams involved in software development and governance, including user groups in testing and software adaptations for diverse social groups. Methods: We performed a literature review on diversity and inclusion in software development and an analysis of 1,000 top-starred Open Source Software (OSS) repositories in GitHub to identify diversity-related information. Moreover, we present a diversity modeling language, a toolkit for generating the cards using that language, and a study of its application in two real-world software projects. Results: Despite the growing awareness of diversity in the research community, our analysis found a notable lack of diversity reporting in OSS projects. Applying the card to real-world examples highlighted challenges like balancing anonymity with transparency, managing sensitive data, and ensuring authenticity. Conclusion: We believe that our proposal can enhance diversity practices in software development, support public administrations in software assessment, and help businesses promote diversity as a key asset.

Advances in online text-to-image (T2I) generator models allow users and organizations alike to generate millions of images from text prompts. Unfortunately, recent studies have demonstrated how elementary prompts result in displaying noticeable social biases in the models' output imagery. The potential representational harm of T2I models could lead to further marginalize minority groups. Thus, a systematic approach is required to comprehensively and continuously assess the absence of bias in T2I generators. In this paper, we present ImageBiTe, a framework to systematically and thoroughly evaluate representational discrimination in T2I-generated images, seamlessly integrable into AI engineering processes. ImageBiTe enables development teams to customize their test scenarios and automatically create and run test cases based on user-defined non-discrimination requirements. We have implemented an open-source tool, available on GitHub, to support the application of our approach.

Abstract Generative artificial intelligence (AI) systems are capable of synthesizing complex artifacts such as text, source code or images according to the instructions provided in a natural language prompt. The quality of the input prompt, in terms of both content and structure, has a large impact on the quality of the output. This has given rise to prompt engineering , the process of designing natural language prompts to best take advantage of the capabilities of generative AI systems. This paper describes , a model-driven engineering framework to support the creation, management and reuse of prompts for generative AI. offers a domain-specific language (DSL) to define multimodal prompts in a modular and tool-independent way. The language offers additional features such as versioning, prompt chaining and multi-language support. Moreover, it provides tool support to adapt prompts for specific generative AI systems, execute those prompts on a generative AI system and validate the quality of the response that is generated. is available as a Langium-based Visual Studio Code plugin.

Large language models (LLMs) are increasingly integrated into software systems to enhance them with generative AI capabilities. But LLMs may reflect a biased behavior, resulting in systems that could discriminate against gender, age or ethnicity, among other ethical concerns. Society and upcoming regulations will force companies and development teams to ensure their AI-enhanced software is ethically fair. To facilitate such ethical assessment, we propose LangBiTe, a model-driven solution to specify ethical requirements, and customize and automate the testing of ethical biases in LLMs. The evaluation can raise awareness on the biases of the LLM-based components of the system and/or trigger a change in the LLM of choice based on the requirements of that particular application. The model-driven approach makes both the requirements specification and the test generation platform-independent, and provides end-to-end traceability between the requirements and their assessment. We have implemented an open-source tool set, available on GitHub, to support the application of our approach.

The increasing complexity of modern systems poses numerous challenges at all stages of system development and operation. Continuous software and system engineering processes, e.g., DevOps, are increasingly adopted and spread across organizations. In parallel, many leading companies have begun to apply artificial intelligence (AI) principles and techniques, including Machine Learning (ML), to improve their products. However, there is no holistic approach that can support and enhance the growing challenges of DevOps. In this paper, we propose a software architecture that provides the foundations of a model-based framework for the development of AI-augmented solutions incorporating methods and tools for continuous software and system engineering and validation. The key characteristic of the proposed architecture is that it allows leveraging the advantages of both AI/ML and Model Driven Engineering (MDE) approaches and techniques in a DevOps context. This architecture has been designed, developed and applied in the context of the European large collaborative project named AIDOaRt. In this paper, we also report on the practical evaluation of this architecture. This evaluation is based on a significant set of technical solutions implemented and applied in the context of different real industrial case studies coming from the AIDOaRt project. Moreover, we analyze the collected results and discuss them according to both architectural and technical challenges we intend to tackle with the proposed architecture.

Large Language Models (LLMs) are being quickly integrated in a myriad of software applications. This may introduce a number of biases, such as gender, age or ethnicity, in the behavior of such applications. To face this challenge, we explore the automatic generation of tests suites to assess the potential biases of an LLM. Each test is defined as a prompt used as input to the LLM and a test oracle that analyses the LLM output to detect the presence of biases.