Requirements Engineering (RE) has been extensively refined for traditional software systems, but AI-based software (AIS)11In this work, AI-based software (AIS) refers to software that relies exclusively on vision-based perception, meaning its understanding of the environment is derived solely from camera input. introduces unique challenges that necessitate novel approaches. This paper addresses the gap in RE practices for AIS by proposing a framework that leverages partial specifications of domain concepts from RE and employs eXplainable AI (XAI) to verify AIS's perception of these specifications. The purpose of this framework is to demonstrate that systematically engineering AIS, according to RE practices, rather than fully relying on AI capabilities, will enhance the perception capabilities of resultant AIS. This work aims to enhance RE4AI by offering a structured approach for managing and evaluating requirements specifications in AIS, ultimately leading to improved performance in these systems. Evaluation results showed that our framework improves AIS perception of variants of two domain concepts-pedestrian and aircraft-within the automotive and aviation domains.

Operational Design Domain (ODD) attributes define the environmental conditions under which Automated Driving Systems (ADS) can safely operate. These attributes include factors such as road and lighting conditions, as well as infrastructure elements, such as lane markings and road conditions. However, existing ODD definitions are often ambiguous and lack specificity, making it challenging to validate their presence in datasets.The absence of precise ODD definitions and robust validation mechanisms poses significant challenges, as it remains unclear whether ADS training and testing datasets adequately represent real-world operating conditions. This gap introduces risks that could compromise the safe deployment of ADS in diverse environments.To address this issue, we introduce FODSE (Framing ODDs as Domain Specifications for Evaluation), a semi-automated AI-powered approach that refines ODD attributes into structured, context-aware domain specifications and systematically evaluates their presence in datasets. FODSE leverages Retrieval-Augmented Generation (RAG), multimodal AI, and prompt learning to enhance specification clarity and dataset completeness.Experimental evaluation on two commonly adopted datasets in ADS demonstrates that FODSE significantly improves dataset validation accuracy, achieving up to 96.8% classification accuracy for an extended set of lane marking variants and 97.8% for roadway users—two key ODD attributes. Expert assessments confirm that FODSE effectively reduces ambiguity and enhances contextual adaptability, reinforcing its potential to improve dataset integrity and ensure safer, more reliable ADS training and validation.

While the conventional software components implement pre-defined specifications, Machine Learning (ML)-enabled Software Components (MLSC) learn the domain specifications from the training samples. Thus, the MLSC's data-driven and inductive reasoning becomes highly reliant on the quality of the training dataset, which are often arbitrarily collected in ad hoc manners. The random collection of samples leads to a significant gap between the actual specifications of a real-world concept, and the picture that a dataset represents of the concept, reducing MLSC generalizability, particularly in perceptual tasks where understanding the environment is an important factor of accurate prediction.

AI-enabled software systems (AIS) are prevalent in a wide range of applications, such as visual tasks of autonomous systems, extensively deployed in automotive, aerial, and naval domains. Hence, it is crucial for humans to evaluate the model’s intelligence before AIS is deployed to safety-critical environments, such as public roads.

The inductive nature of artificial neural models makes dataset quality a key factor of their proper functionality. For this reason, multiple research studies proposed metrics to assess the quality of the models’ datasets, such as dataset correctness, completeness, and consistency. However, these studies commonly lack a point of reference against which the proposed quality metrics could be assessed. To this end, this paper proposes a generic process that extracts the necessary knowledge to build a reliable reference point for the purpose of explanation, assessment, and augmentation of the AI-software dataset. This process automatically builds a benchmark specific to the software operational domain, interprets the training and validation datasets of AI-enabled perception software systems, and evaluates the dataset semantic quality and completeness relative to the benchmark. We implemented this process within a framework called Concept Augmentation and Dataset Evaluation (CADE), which leverages a series of novel natural language and image processing techniques to construct a semantic benchmark with respect to the domain specifications. The application of CADE to three commonly-used autonomous driving datasets showed several common weaknesses present in the arbitrarily-collected datasets against the encoded domain specifications, demonstrating dataset divergence from the domain concepts and under-represented variances of the concepts in the data. The qualitative evaluation results showed an average of about 75% relevancy of CADE generated topics.