Context: The quality of requirements specifications is a critical success factor in software development. Assuring high-quality requirements, specifically in an automated way, poses a significant challenge due to their unstructured and multi-modal character. With the rise of deep learning and large language models (LLMs), new opportunities have developed to assess the quality of requirements automatically, particularly user stories in the context of agile software engineering, where short development cycles require efficient tool support. Objective: This study aims to systematically review and investigate the current landscape of approaches based on artificial intelligence techniques such as natural language processing and deep learning for assessing the quality of software requirements. The investigation focuses on the artificial intelligence techniques adopted, quality aspects considered, datasets used to tune and evaluate the proposed approaches, and their performance. Method: We conducted a systematic literature review of 26 peer-reviewed papers published between 2019 and 2025. We selected the papers after a title and abstract review of 353 papers identified through a literature databases query and forward–backward snowballing. Results: The results reveal significant overlap among considered quality aspects, which can be mapped onto the higher-order requirements quality model INVEST. Most studies focus on assessing requirement quality rather than improving requirements and rely heavily on synthetic and public datasets. LLMs have rapidly gained popularity since 2023, though model evaluation strategies remain inconsistent. Metrics such as accuracy, precision, recall, and F1-Score are common, yet a few studies use semantic or expert-based evaluations. Conclusion: The field is evolving toward LLM-driven, semantically rich models, yet lacks methodological standardization, reproducible datasets for evaluating the models, and integration of the approaches with real-world requirements engineering processes. Future work should address these limitations by developing benchmark datasets, standardizing evaluation metrics, and exploring hybrid systems that combine AI-based and traditional requirements quality assurance approaches.

Abstract Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary qualities of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to re-interpret existing qualities for ML systems or add new ones (such as trustworthiness). We have to be very precise about which quality property is relevant for which entity of interest (such as completeness of training data or correctness of trained model), and how to objectively evaluate adherence to quality requirements. In this article, we present how to systematically construct quality models for ML systems based on an industrial use case. This quality model enables practitioners to specify and assess qualities for ML systems objectively. In addition to the overall construction process described, the main outcomes include a meta-model for specifying quality models for ML systems, reference elements regarding relevant views, entities, quality properties, and measures for ML systems based on existing research, an example instantiation of a quality model for a concrete industrial use case, and lessons learned from applying the construction process. We found that it is crucial to follow a systematic process in order to come up with measurable quality properties that can be evaluated in practice. In the future, we want to learn how the term quality differs between different types of ML systems and come up with reference quality models for evaluating qualities of ML systems.

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.

As the applications of machine learning algorithms in various fields are widely demanded, the development of machine learning software systems (MLS) is rapidly increasing. The quality of MLS is different from that of conventional software systems, in the sense that it depends on the amount and distribution of training data in a model learning and input data during operation. This is a major challenge in quality assurance of MLS development for the enterprise. In this paper, we propose a requirements-driven method to determine the quality characteristics of the MLS. Major contributions of this paper include: (1) Extending the quality characteristics of ISO 25010, which defines the conventional software quality, to those unique to MLS; this paper also defines its measuring method. (2) A method to identify requirements, i.e., issues to be determined in the requirements definition, in order to derive the quality characteristics and measurement methods for MLS, since the quality characteristics and the measurement method depend on the goals of the system under development. In order to evaluate the proposed method, we carried out an empirical study of the quality characteristics and measurement methods related to functional correctness and the maturity of the MLS for the enterprise. Based on the study, we compare the quality characteristics and measurement methods derived by the proposed method with those suggested by developers, and demonstrate the effectiveness of the proposed method.