The development of high-quality software depends on precise and comprehensive requirements that meet the objectives of stakeholders. Goal modeling techniques have been developed to fill this gap by capturing and analyzing stakeholders' needs and allowing them to make trade-off decisions; yet, goal modeling analysis is often difficult for stakeholders to interpret. Recent work found that when subjects are given minimal training on goal modeling and access to a color visualization, called EVO, they are able to use EVO to make goal modeling decisions faster without compromising quality. In this paper, we evaluate the robustness of the empirical evidence for EVO and question the underlying color choices made by the initial designers of EVO. We conduct a pseudo-exact replication$(n=60)$of the original EVO study, varying the experimental site and the study population. Even in our heterogeneous sample with less a priori familiarity with requirements and goal modeling, we find that individuals using EVO answered the goal-modeling questions significantly faster than those using the control, expanding the external validity of the original results. However, we find some evidence that the chosen color scheme is not intuitive and make recommendations for the goal modeling community.

As software systems increasingly interact with humans in application domains such as transportation and healthcare, they raise concerns related to the social, legal, ethical, empathetic, and cultural (SLEEC) norms and values of their stakeholders. Normative non-functional requirements (N-NFRs) are used to capture these concerns by setting SLEEC-relevant boundaries for system behavior. Since N-NFRs need to be specified by multiple stakeholders with widely different, non-technical expertise (ethicists, lawyers, regulators, end users, etc.), N-NFR elicitation is very challenging. To address this difficult task, we introduce N-Check, a novel tool-supported formal approach to N-NFR analysis and debugging. N-Check employs satisfiability checking to identify a broad spectrum of N-NFR well-formedness issues, such as conflicts, redundancy, restrictiveness, and insufficiency, yielding diagnostics that pinpoint their causes in a user-friendly way that enables non-technical stakeholders to understand and fix them. We show the effectiveness and usability of our approach through nine case studies in which teams of ethicists, lawyers, philosophers, psychologists, safety analysts, and engineers used N-Check to analyse and debug 233 N-NFRs, comprising 62 issues for the software underpinning the operation of systems, such as, assistive-care robots and tree-disease detection drones to manufacturing collaborative robots.

Normative non-functional requirements specify con-straints that a system must observe in order to avoid violations of social, legal, ethical, empathetic, and cultural norms. As these requirements are typically defined by non-technical system stakeholders with different expertise and priorities (ethicists, lawyers, social scientists, etc.), ensuring their well-formedness and consistency is very challenging. Recent research has tackled this challenge using a domain-specific language to specify normative requirements as rules whose consistency can then be analysed with formal methods. In this paper, we propose a complemen-tary approach that uses Large Language Models to extract semantic relationships between abstract representations of system capabilities. These relations, which are often assumed implicitly by non-technical stakeholders (e.g., based on common sense or domain knowledge), are then used to enrich the automated reasoning techniques for eliciting and analyzing the consistency of normative requirements. We show the effectiveness of our approach to normative requirements elicitation and operational-ization through a range of real-world case studies. An extended version of this paper, which includes appendices is available at https://arxiv.org/abs/2404.12335

Recent approaches have investigated assisting users in making early trade-off decisions when the future evolution of project elements is uncertain. These approaches have demon-strated promise in their analytical capabilities; yet, stakeholders have expressed concerns about the readability of the models and resulting analysis, which builds upon Tropos. Tropos is based on formal semantics enabling automated analysis; however, this creates a problem of interpreting evidence pairs. The aim of our broader research project is to improve the process of model comprehension and decision making by improving how analysts interpret and make decisions. We extend and evaluate a prior approach, called EVO, which uses color to visualize evidence pairs. In this scientific evaluation paper, we explore the effectiveness and usability of EVO. We conduct an experiment (n = 32) to measure any effect of using colors to represent evidence pairs. We find that with minimal training, untrained modelers were able to use the color visualization for decision making. The visualization significantly improves the speed of model comprehension and users found it helpful.

Automated analysis has been used in goal-oriented requirements engineering (GORE) to evaluate scenarios and make trade-off decisions. For higher complexity problems (e.g., backwards analysis), using a search-based solver may be more efficient than custom algorithms. When these black-box solvers produce a single solution, users may be suspicious about whether the given answer is ideal or believable. Users would like to explore the potential solutions but are prevented from doing so because these inquiries often suffer from a state explosion problem. In this RE@Next! paper, we introduce the use of valuation-based filtering and coloring to assist users in understanding a solution space and selecting custom states from it. We use the concrete semantics of modeling requirements in the Evolving Intentions framework and its associated goal modeling tool, BloomingLeaf, to explore the application of these visualization techniques. In our initial evaluation, we demonstrate how these techniques can be used on a fully worked out example. We conduct initial measurements of the time savings and state space reduction created by the valuations and color filtering, and discuss future directions of this project.

Constraint satisfaction problems (CSPs) are problems with a set of variables that have constraints and the goal is to assign a value to each of the variables so the constraints are satisfied. CSPs are found in many areas and domains of computer science. When given a single solution out of the solution space from a general-purpose solver, users may want to further explore other alternatives to find ones that better meet their aims or to verify the believability of the existing result. However, users attempting to examine the solution space are often confronted with a state explosion problem, leaving them with the unrealistic task of manually sifting through each state. We aimed to assist users in exploring state spaces more efficiently. Using the concrete semantics of requirements modeling in the Evolving Intentions framework, we contribute two visualization techniques: (1) valuation-based coloring to assist users in interpreting the results of the path-based analysis and selecting states from the solution space more efficiently, and (2) valuation-based filtering to reduce the number of states in the solution space. Using a motivating example, we demonstrate the need for state space visualization and we provide an initial validation of our results. These visualization techniques can be used beyond the Evolving Intentions framework and adapted to other areas in software engineering.