Requirements Engineering (RE) has been extensively refined for traditional software systems, but AI-based software (AIS)11In this work, AI-based software (AIS) refers to software that relies exclusively on vision-based perception, meaning its understanding of the environment is derived solely from camera input. introduces unique challenges that necessitate novel approaches. This paper addresses the gap in RE practices for AIS by proposing a framework that leverages partial specifications of domain concepts from RE and employs eXplainable AI (XAI) to verify AIS's perception of these specifications. The purpose of this framework is to demonstrate that systematically engineering AIS, according to RE practices, rather than fully relying on AI capabilities, will enhance the perception capabilities of resultant AIS. This work aims to enhance RE4AI by offering a structured approach for managing and evaluating requirements specifications in AIS, ultimately leading to improved performance in these systems. Evaluation results showed that our framework improves AIS perception of variants of two domain concepts-pedestrian and aircraft-within the automotive and aviation domains.

Operational Design Domain (ODD) attributes define the environmental conditions under which Automated Driving Systems (ADS) can safely operate. These attributes include factors such as road and lighting conditions, as well as infrastructure elements, such as lane markings and road conditions. However, existing ODD definitions are often ambiguous and lack specificity, making it challenging to validate their presence in datasets.The absence of precise ODD definitions and robust validation mechanisms poses significant challenges, as it remains unclear whether ADS training and testing datasets adequately represent real-world operating conditions. This gap introduces risks that could compromise the safe deployment of ADS in diverse environments.To address this issue, we introduce FODSE (Framing ODDs as Domain Specifications for Evaluation), a semi-automated AI-powered approach that refines ODD attributes into structured, context-aware domain specifications and systematically evaluates their presence in datasets. FODSE leverages Retrieval-Augmented Generation (RAG), multimodal AI, and prompt learning to enhance specification clarity and dataset completeness.Experimental evaluation on two commonly adopted datasets in ADS demonstrates that FODSE significantly improves dataset validation accuracy, achieving up to 96.8% classification accuracy for an extended set of lane marking variants and 97.8% for roadway users—two key ODD attributes. Expert assessments confirm that FODSE effectively reduces ambiguity and enhances contextual adaptability, reinforcing its potential to improve dataset integrity and ensure safer, more reliable ADS training and validation.

Context: Prior research revealed that code components with similar structures tend to require structurally similar test cases and they often co-evolve over time. Objective: Leveraging this pattern, we implemented a prototype tool, Test Suite Evolver (TSE), to support the generation of test cases for structurally similar methods. Our prototype tool is applicable to both, incomplete test suites with the purpose of test case augmentation, as well as evolving test suites when a newer version of a software becomes available.

While the conventional software components implement pre-defined specifications, Machine Learning (ML)-enabled Software Components (MLSC) learn the domain specifications from the training samples. Thus, the MLSC's data-driven and inductive reasoning becomes highly reliant on the quality of the training dataset, which are often arbitrarily collected in ad hoc manners. The random collection of samples leads to a significant gap between the actual specifications of a real-world concept, and the picture that a dataset represents of the concept, reducing MLSC generalizability, particularly in perceptual tasks where understanding the environment is an important factor of accurate prediction.

Software systems are characterized by continual change which often occurs concurrently across various artifact types. While prior work has focused on the evolution of individual artifacts, this paper studies the patterns of co-evolution between source and test code. In this research, with a reference to the literature, as well as our manual analysis of several open-source software systems we first, patternize and document common patterns of co-evolution between source code and test suites. Leveraging the proposed patterns, we further infer the necessary remedies in the test suite in response to source code changes. Our approach enables to add missing test cases to the current version of a system (augmentation), but additionally allows to reuse and evolve the existing test suite for a modified version of the system (evolution). Furthermore, identifying patterns of concurrent evolution provides opportunities for a bi-directional change detection and remediation for both artifacts, source code and test cases, and additionally automates the process of maintaining code-to-test trace links. The evaluation of the patterns and remedies in five large open-source applications indicated the patterns contained up to 42% of the source code changes and the remediation recovered up to 100% of the impacted test cases in certain cases.

AI-enabled software systems (AIS) are prevalent in a wide range of applications, such as visual tasks of autonomous systems, extensively deployed in automotive, aerial, and naval domains. Hence, it is crucial for humans to evaluate the model’s intelligence before AIS is deployed to safety-critical environments, such as public roads.

The inductive nature of artificial neural models makes dataset quality a key factor of their proper functionality. For this reason, multiple research studies proposed metrics to assess the quality of the models’ datasets, such as dataset correctness, completeness, and consistency. However, these studies commonly lack a point of reference against which the proposed quality metrics could be assessed. To this end, this paper proposes a generic process that extracts the necessary knowledge to build a reliable reference point for the purpose of explanation, assessment, and augmentation of the AI-software dataset. This process automatically builds a benchmark specific to the software operational domain, interprets the training and validation datasets of AI-enabled perception software systems, and evaluates the dataset semantic quality and completeness relative to the benchmark. We implemented this process within a framework called Concept Augmentation and Dataset Evaluation (CADE), which leverages a series of novel natural language and image processing techniques to construct a semantic benchmark with respect to the domain specifications. The application of CADE to three commonly-used autonomous driving datasets showed several common weaknesses present in the arbitrarily-collected datasets against the encoded domain specifications, demonstrating dataset divergence from the domain concepts and under-represented variances of the concepts in the data. The qualitative evaluation results showed an average of about 75% relevancy of CADE generated topics.

From financial services platforms to social networks to vehicle control, software has come to mediate many activities of daily life. Governing bodies and standards organizations have responded to this trend by creating regulations and standards to address issues such as safety, security and privacy. In this environment, the compliance of software development to standards and regulations has emerged as a key requirement. Compliance claims and arguments are often captured in assurance cases, with linked evidence of compliance. Evidence can come from testcases, verification proofs, human judgment, or a combination of these. That is, experts try to build (safety-critical) systems carefully according to well justified methods and articulate these justifications in an assurance case that is ultimately judged by a human. Yet software is deeply rooted in uncertainty; most complex open-world functionality (e.g., perception of the state of the world by a self-driving vehicle), is either not completely specifiable or it is not cost-effective to do so; software systems are often to be placed into uncertain environments, and there can be uncertainties that need to be We argue that the role of assurance cases is to be the grand unifier for software development, focusing on capturing and managing uncertainty. We discuss three approaches for arguing about safety and security of software under uncertainty, in the absence of fully sound and complete methods: assurance argument rigor, semantic evidence composition and applicability to new kinds of systems, specifically those relying on ML.

Safety Assurance Cases (SACs) are increasingly used to guide and evaluate the safety of software-intensive systems. They are used to construct a hierarchically organized set of claims, arguments, and evidence in order to provide a structured argument that a system is safe for use. However, as the system evolves and grows in size, a SAC can be difficult to maintain. In this paper we utilize design science to develop a novel solution for identifying areas of a SAC that are affected by changes to the system. Moreover, we generate actionable recommendations for updating the SAC, including its underlying artifacts and trace links, in order to evolve an existing safety case for use in a new version of the system. Our approach, Safety Artifact Forest Analysis (SAFA), leverages traceability to automatically compare software artifacts from a previously approved or certified version with a new version of the system. We identify, visualize, and explain changes in a Delta Tree. We evaluate our approach using the Dronology system for monitoring and coordinating the actions of cooperating, small Unmanned Aerial Vehicles. Results from a user study show that SAFA helped users to identify changes that potentially impacted system safety and provided information that could be used to help maintain and evolve a SAC.

Problems with the correctness and completeness of environmental assumptions contribute to many accidents in safety-critical systems. The problem is exacerbated when products are modified in new releases or in new products of a product line. In such cases existing sets of environmental assumptions are often carried forward without sufficiently rigorous analysis. This paper describes a new technique that exploits the traceability required by many certifying bodies to reason about the likelihood that environmental assumptions are omitted or incorrectly retained in new products. An analysis of over 150 examples of environmental assumptions in historical systems informs the approach. In an evaluation on three safety-related product lines the approach caught all but one of the assumption-related problems. It also provided clearly defined steps for mitigating the identified issues. The contribution of the work is to arm the safety analyst with useful information for assessing the validity of environmental assumptions for a new product.

To facilitate research into trace link evolution we present 27 versions of Cassandra source code, feature descriptions for each version, deltas between versions, structured descriptions of each version, and trace links between a subset of 48 features and source code for the starting and ending versions.

Trace links provide critical support for numerous software engineering activities including safety analysis, compliance verification, test-case selection, and impact prediction. However, as the system evolves over time, there is a tendency for the quality of trace links to degrade into a tangle of inaccurate and untrusted links. This is especially true with the links between source-code and upstream artifacts such as requirements - because developers frequently refactor and change code without updating the links. We present TLE (Trace Link Evolver), a solution for automating the evolution of trace links as changes are introduced to source code. We use a set of heuristics, open source tools, and information retrieval methods to detect common change scenarios across different versions of software. Each change scenario is then associated with a set of link evolution heuristics which are used to evolve trace links. We evaluate our approach through a controlled experiment and also through applying it across 27 releases of the Cassandra Database System. Results show that the trace links evolved using our approach are significantly more accurate than those generated using information retrieval alone.

Software project artifacts such as source code, requirements, and change logs represent a gold-mine of actionable information. As a result, software analytic solutions have been developed to mine repositories and answer questions such as "who is the expert?," "which classes are fault prone?," or even "who are the domain experts for these fault-prone classes?" Analytics often require training and configuring in order to maximize performance within the context of each project. A cold-start problem exists when a function is applied within a project context without first configuring the analytic functions on project-specific data. This scenario exists because of the non-trivial effort necessary to instrument a project environment with candidate tools and algorithms and to empirically evaluate alternate configurations. We address the cold-start problem by comparatively evaluating 'best-of-breed' and 'profile-driven' solutions, both of which reuse known configurations in new project contexts. We describe and evaluate our approach against 20 project datasets for the three analytic areas of artifact connectivity, fault-prediction, and finding the expert, and show that the best-of-breed approach outperformed the profile-driven approach in all three areas; however, while it delivered acceptable results for artifact connectivity and find the expert, both techniques underperformed for cold-start fault prediction.

When organizations enter domains that are entirely new to them, they need to invest significant time and effort to acquire domain knowledge. This typically involves searching through a broad set of domain documents, retrieving relevant ones, and analyzing the textual content in order to discover and specify pertinent requirements. Depending on the nature of the domain and the availability of documentation, this task can be extremely time-consuming and may require non-trivial human effort. Furthermore, the task must often be performed repeatedly throughout early phases of the project. In this paper we first explore the effort needed to manually build a high-level domain model capturing the functional components. We then present MaRK (Mining Requirements Knowledge), which identifies and retrieves the documents containing descriptions of functional components in the domain model. Domain analysts can use this information to to specify requirements. We introduce and evaluate an algorithm which ranks domain documents according to their relevance to a component and then highlights sections of text which are likely to contain requirements-related information. We describe our process within the context of the Positive Train Control (PTC) domain with a repository of of 523 documents, representing 852MB of data. We empirically evaluate the MaRK relevance algorithm and its ability to retrieve relevant requirements knowledge for requirements related to PTC's On-Board Unit.

Research projects tend to evolve through multiple phases of incubation and experimentation, before maturing to levels of full industry adoption. Practice has shown that successful research solutions often take over 20 years to achieve full technology transfer. However, many projects never leave the incubation phase either because the new technique fails to perform well, or because researchers lack the knowledge, skills, or time to transition the idea to practice. A healthy research community could be expected to produce a steady stream of innovative solutions that positively impact industrial practice. To achieve these goals, we need ongoing, rigorous, and mutually beneficial conversations between academics and practitioners. Such exchanges are a desirable part of the research process, and will help the requirements engineering community to integrate technology transfer plans into the ongoing research plans. In this interactive panel, teams of researchers, representing different requirements engineering research areas, will present their research solutions to a panel of seasoned industrial practitioners. The practitioners provide insightful feedback that can help with the transition to practice. While Ready-Set-Transfer is presented as an interactive game-show, it has the serious goal of fostering collaboration and conversations between practitioners and researchers in the requirements engineering community.

Many software systems utilize forums to allow a broad set of stakeholders to request features. However the resulting mass of ideas and comments can make prioritization and management of feature requests challenging. In this paper we propose a novel approach for partially automating the creation of personas from a set of feature requests in open forums. Our approach utilizes topic clustering, classification, and association rules to identify meaningful groupings of feature requests and then uses them to guide the construction of personas. Once created, these personas are leveraged to coordinate feature requests, track changes, and to provide stakeholder communication mechanisms. We illustrate our approach with examples taken from the health-insurance domain and then evaluate it against feature requests in the SugarCRM project.

Software requirements specifications often focus on functionality and fail to adequately capture quality concerns such as security, performance, and usability. In many projects, quality-related requirements are either entirely lacking from the specification or intermingled with functional concerns. This makes it difficult for stakeholders to fully understand the quality concerns of the system and to evaluate their scope of impact. In this paper we present a data mining approach for automating the extraction and subsequent modeling of quality concerns from requirements, feature requests, and online forums. We extend our prior work in mining quality concerns from textual documents and apply a sequence of machine learning steps to detect quality-related requirements, generate goal graphs contextualized by project-level information, and ultimately to visualize the results. We illustrate and evaluate our approach against two industrial health-care related systems.

Despite the fact that traceability is a required element of almost all safety-critical software development processes, the trace data is often incomplete, inaccurate, redundant, conflicting, and outdated. As a result, it is neither trusted nor trustworthy. In this vision paper we propose a philosophical change in the traceability landscape which transforms traceability from a heavy-weight process producing untrusted trace links, to a light-weight results-oriented trustworthy solution. Current traceability practices which retard agility are cast away and replaced with a disciplined, just-in-time approach. The novelty of our solution lies in a clear separation of trusted trace links from untrusted ones, the change in perspective from `living-with' inacurate traces toward rigorous and ongoing debridement of stale links from the trusted pool, and the notion of synthesizing available `project exhaust' as evidence to systematically construct or reconstruct purposed, highly-focused trace links.