With the growing adoption of AI-based systems, effective risk management is more important than ever. Obstacle analysis is a requirements engineering technique introduced three decades ago for designing dependable software systems despite failures, exceptions, and unforeseen behaviors in both the software and its environment. An obstacle is an undesirable situation that violates a stakeholder goal, an environment assumption, or a software requirement. Obstacles include safety hazards, security threats, user errors, and other adverse situations. Obstacle analysis provides a structured, systematic approach for identifying, analyzing, and resolving obstacles at the requirements level. In this retrospective paper, we summarize the original technique and discuss its impacts on research and practice. We also propose a research agenda to extend obstacle analysis to address emerging challenges in AI systems engineering.

App reviews provide a rich source of feature-related information that can support requirement engineering activities. Analyzing them manually to find this information, however, is challenging due to their large quantity and noisy nature. To overcome the problem, automated approaches have been proposed for ‘feature-specific analysis’. Unfortunately, the effectiveness of these approaches has been evaluated using different methods and datasets. Replicating these studies to confirm their results and to provide benchmarks of different approaches is a challenging problem. We address the problem by extending previous evaluations and performing a comparison of these approaches. In this paper, we present two empirical studies. In the first study, we evaluate opinion mining approaches; the approaches extract features discussed in app reviews and identify their associated sentiments. In the second study, we evaluate approaches searching for feature-related reviews. The approaches search for users’ feedback pertinent to a particular feature. The results of both studies show these approaches achieve lower effectiveness than reported originally, and raise an important question about their practical use.

App reviews can provide valuable information about user needs but analyzing them manually is challenging due to their large quantity and noisy nature. To overcome this problem, a variety of app review mining techniques have been proposed. So far, however, research in this area has paid little attention to the software engineering use cases of the mining techniques. This limits the understanding of their usefulness, applications and desired future developments. We address this problem by elaborating a reference model relating app review mining techniques to specific software engineering activities. In this paper, we present a unified description of software engineering use cases for mining app reviews and define a reference architecture realizing these use cases through a combination of natural language processing and data mining techniques. The use cases provide a novel systematic exposition of the envisioned applications and benefits of app review mining for software engineers. The reference architecture synthesises the diversity of research to realise these benefits and provide a general framework guiding the development and evaluation of future research and tools.

Abstract App reviews found in app stores can provide critically valuable information to help software engineers understand user requirements and to design, debug, and evolve software products. Over the last ten years, a vast amount of research has been produced to study what useful information might be found in app reviews, and how to mine and organise such information as efficiently as possible. This paper presents a comprehensive survey of this research, covering 182 papers published between 2012 and 2020. This survey classifies app review analysis not only in terms of mined information and applied data mining techniques but also, and most importantly, in terms of supported software engineering activities. The survey also reports on the quality and results of empirical evaluation of existing techniques and identifies important avenues for further research. This survey can be of interest to researchers and commercial organisations developing app review analysis techniques and to software engineers considering to use app review analysis.

Release planning—deciding what features to implement in upcoming releases of a software system—is a critical activity in iterative software development. Many release planning methods exist, but most ignore the inevitable uncertainty in estimating software development effort and business value. The article’s objective is to study whether analyzing uncertainty during release planning generates better release plans than if uncertainty is ignored. To study this question, we have developed a novel release planning method under uncertainty, called BEARS, that models uncertainty using Bayesian probability distributions and recommends release plans that maximize expected net present value and expected punctuality. We then compare release plans recommended by BEARS to those recommended by methods that ignore uncertainty on 32 release planning problems. The experiment shows that BEARS recommends release plans with higher expected net present value and expected punctuality than methods that ignore uncertainty, thereby indicating the harmful effects of ignoring uncertainty during release planning. These results highlight the importance of eliciting and analyzing uncertainty in software effort and value estimations and call for increased research in these areas.

Uncertainty and conflicting stakeholders' objectives make many requirements and architecture decisions particularly hard. Quantitative probabilistic models allow software architects to analyse such decisions using stochastic simulation and multi-objective optimisation, but the difficulty of elaborating the models is an obstacle to the wider adoption of such techniques. To reduce this obstacle, this paper presents a novel modelling language and analysis tool, called RADAR, intended to facilitate requirements and architecture decision analysis. The language has relations to quantitative AND/OR goal models used in requirements engineering and to feature models used in software product lines. However, it simplifies such models to a minimum set of language constructs essential for decision analysis. The paper presents RADAR's modelling language, automated support for decision analysis, and evaluates its application to four real-world examples.

Uncertainty is inevitable in real world requirement engineering. It has a significant impact on the feasibility of proposed solutions and thus brings risks to the software release plan. This paper proposes a multi-objective optimization technique, augmented with Monte-Carlo Simulation, that optimizes requirement choices for the three objectives of cost, revenue, and uncertainty. The paper reports the results of an empirical study over four data sets derived from a single real world data set. The results show that the robust optimal solutions obtained by our approach are conservative compared to their corresponding optimal solutions produced by traditional Multi-Objective Next Release Problem. We obtain a robustness improvement of at least 18% at a small cost (a maximum 0.0285 shift in the 2D Pareto-front in the unit space). Surprisingly we found that, though a requirement's cost is correlated with inclusion on the Pareto-front, a requirement's expected revenue is not.

Uncertainty complicates early requirements and architecture decisions and may expose a software project to significant risk. Yet software architects lack support for evaluating uncertainty, its impact on risk, and the value of reducing uncertainty before making critical decisions. We propose to apply decision analysis and multi-objective optimisation techniques to provide such support. We present a systematic method allowing software architects to describe uncertainty about the impact of alternatives on stakeholders' goals; to calculate the consequences of uncertainty through Monte-Carlo simulation; to shortlist candidate architectures based on expected costs, benefits and risks; and to assess the value of obtaining additional information before deciding. We demonstrate our method on the design of a system for coordinating emergency response teams. Our approach highlights the need for requirements engineering and software cost estimation methods to disclose uncertainty instead of hiding it.

User experience (UX) is difficult to quantify and thus more challenging to require and guarantee. It is also difficult to gauge the potential impact on users' lived experience, especially at the earlier stages of the development life cycle, particularly before hi fidelity prototypes are developed. We believe that the enrolment process is a major hurdle for e-government service adoption and badly designed processes might result in negative repercussions for both the policy maker and the different user groups involved; non-adoption and resentment are two risks that may result in low return on investment (ROI), lost political goodwill and ultimately a negative lived experience for citizens. Identity assurance requirements need to balance out the real value of the assets being secured (risk) with the user groups' acceptance thresholds (based on a continuous cost-benefit exercise factoring in cognitive and physical workload). Sentire is a persona-centric requirements framework built on and extending the Volere requirements process with UX-analytics, reusable user behavioural models and simulated user feedback through calibrated personas. In this paper we present a story on how Sentire was adopted in the development of a national public-facing e-service. Daily journaling was used throughout the project and a custom built cloud-based CASE tool was used to manage the whole process. This paper outlines our experiences and lessons learnt.

Requirements modelling helps software engineers understand a system's required behaviour and explore alternative system designs. It also generates a formal software specification that can be used for testing, verification, and debugging. However, elaborating such models requires expertise and significant human effort. The paper aims at reducing this effort by automating an essential activity of requirements modelling which consists in deriving a machine specification satisfying a set of goals in a domain. It introduces deontic input-output automata - an extension of input-output automata with permissions and obligations - and an automated synthesis technique over this formalism to support such derivation. This technique helps modellers identifying early when a goal is not realizable in a domain and can guide the exploration of alternative models to make goals realizable. Synthesis techniques for input-output or interface automata are not adequate for requirements modelling.

Scalability is a critical concern for many software systems. Despite the recognized importance of considering scalability from the earliest stages of development, there is currently little support for reasoning about scalability at the requirements level. This paper presents a goal-oriented approach for eliciting, modeling, and reasoning about scalability requirements. The approach consists of systematically identifying scalability-related obstacles to the satisfaction of goals, assessing the likelihood and severity of these obstacles, and generating new goals to deal with them. The result is a consolidated set of requirements in which important scalability concerns are anticipated through the precise, quantified specification of scaling assumptions and scalability goals. The paper presents results from applying the approach to a complex, large-scale financial fraud detection system.

Online feature request management systems are popular tools for gathering stakeholder requirements during system evolution. Deciding which feature requests require attention and how much upfront analysis to perform on them is an important problem in this context: too little upfront analysis may result in inadequate functionalities being developed, costly changes, and wasted development effort; too much upfront analysis is a waste of time and resources. Early predictions about which feature requests are most likely to fail due to insufficient or inadequate upfront analysis could facilitate such decisions. Our objective is to study whether it is possible to make such predictions automatically from the characteristics of the online discussions on feature requests. The paper presents a tool-implemented framework that automatically constructs failure prediction models using machine-learning classification algorithms and compares the performance of the different techniques for the Firefox and Netbeans projects. The comparison relies on a cost-benefit model for assessing the value of additional upfront analysis. In this model, the value of additional upfront analysis depends on its probability of success in preventing failures and on the relative cost of the failures it prevents compared to its own cost. We show that for reasonable estimations of these two parameters automated prediction models provide more value than a set of baselines for some failure types and projects. This suggests that automated failure prediction during online requirements elicitation may be a promising approach for guiding requirements engineering efforts in online settings.

Making decisions among a set of alternative system designs is an essential activity of requirements engineering. It involves evaluating how well each alternative satisfies the stakeholders' goals and selecting one alternative that achieves some optimal tradeoffs between possibly conflicting goals. Quantitative goal models support such activities by describing how alternative system designs - expressed as alternative goal refinements and responsibility assignments - impact on the levels of goal satisfaction specified in terms of measurable objective functions. Analyzing large numbers of alternative designs in such models is an expensive activity for which no dedicated tool support is currently available. This paper takes a first step towards providing such support by presenting automated techniques for (i) simulating quantitative goal models so as to estimate the levels of goal satisfaction contributed by alternative system designs and (ii) optimising the system design by applying a multi-objective optimisation algorithm to search through the design space. These techniques are presented and validated using a quantitative goal model for a well-known ambulance service system.

Multi-objective decisions problems are ubiquitous in requirements engineering. A common approach to solve them is to apply search-based techniques to generate a set of non-dominated solutions, formally known as the Pareto front, that characterizes all solutions for which no other solution performs better on all objectives simultaneously. Analysing the shape of the Pareto front helps decision makers understand the solution space and possible tradeoffs among the conflicting objectives. Interpreting the optimal solutions, however, remains a significant challenge. It is in particular difficult to identify whether solutions that have similar levels of goals attainment correspond to minor variants within a same design or to very different designs involving completely different sets of decisions. Our goal is to help decision makers identify groups of strongly related solutions in a Pareto front so that they can understand more easily the range of design choices, identify areas where strongly different solutions achieve similar levels of objectives, and decide first between major groups of solutions before deciding for a particular variant within the chosen group. The benefits of the approach are illustrated on a small example and validated on a larger independently-produced example representative of industrial problems.

Computational reflection is a well-established technique that gives a program the ability to dynamically observe and possibly modify its behaviour. To date, however, reflection is mainly applied either to the software architecture or its implementation. We know of no approach that fully supports requirements reflection- that is, making requirements available as runtime objects. Although there is a body of literature on requirements monitoring, such work typically generates runtime artefacts from requirements and so the requirements themselves are not directly accessible at runtime. In this paper, we define requirements reflection and a set of research challenges. Requirements reflection is important because software systems of the future will be self-managing and will need to adapt continuously to changing environmental conditions. We argue requirements reflection can support such self-adaptive systems by making requirements first-class runtime entities, thus endowing software systems with the ability to reason about, understand, explain and modify requirements at runtime.

Requirements are sensitive to the context in which the system-to-be must operate. Where such context is well understood and is static or evolves slowly, existing RE techniques can be made to work well. Increasingly, however, development projects are being challenged to build systems to operate in contexts that are volatile over short periods in ways that are imperfectly understood. Such systems need to be able to adapt to new environmental contexts dynamically, but the contextual uncertainty that demands this self-adaptive ability makes it hard to formulate, validate and manage their requirements. Different contexts may demand different requirements trade-offs. Unanticipated contexts may even lead to entirely new requirements. To help counter this uncertainty, we argue that requirements for self-adaptive systems should be run-time entities that can be reasoned over in order to understand the extent to which they are being satisfied and to support adaptation decisions that can take advantage of the systems' self-adaptive machinery. We take our inspiration from the fact that explicit, abstract representations of software architectures used to be considered design-time-only entities but computational reflection showed that architectural concerns could be represented at run-time too, helping systems to dynamically reconfigure themselves according to changing context. We propose to use analogous mechanisms to achieve requirements reflection. In this paper we discuss the ideas that support requirements reflection as a means to articulate some of the outstanding research challenges.

Our objective is to provide automated support for assisting designers in fixing inconsistencies in UML models. We have previously developed techniques for efficiently detecting inconsistencies in such models and identifying where changes need to occur in order to fix problems detected by these means. This paper extends previous work by describing a technique for automatically generating a set of concrete changes for fixing inconsistencies and providing information about the impact of each change on all consistency rules. The approach is integrated with the design tool IBM Rational Rose . We demonstrate the computational scalability and usability of the approach through the empirical evaluation of 39 UML models of sizes up to 120,000 elements.

Scalability is widely recognized as an important software quality, but it is a quality that historically has lacked a consistent and systematic treatment. To address this problem, we recently presented a framework for the characterization and analysis of software systems scalability. That initial work did not provide means to instantiate the variables and functions to be used in the analysis, which could compromise its results. This risk can be mitigated through a systematic exploration of system scalability goals in the application domain during requirements engineering. This paper describes our application of goal-oriented requirements engineering (GORE) for eliciting the scalability requirements of a large, real-world financial fraud detection system. The case study reveals both the suitability and the limitations of GORE as a technique for eliciting the information needed by stakeholders to specify scalability goals of a system. In the paper, we describe these findings in detail and chart a course for future research in extending goal-oriented techniques to scalability requirements.

Scenarios are an effective means for eliciting, validating and documenting requirements. At the requirements level, scenarios describe sequences of interactions between the software-to-be and agents in the environment. Interactions correspond to the occurrence of an event that is controlled by one agent and monitored by another.This paper presents a technique to analyse requirements-level scenarios for unforeseen, potentially harmful, consequences. Our aim is to perform analysis early in system development, where it is highly cost-effective. The approach recognises the importance of monitoring and control issues and extends existing work on implied scenarios accordingly. These so-called input-output implied scenarios expose problematic behaviours in scenario descriptions that cannot be detected using standard implied scenarios. Validation of these implied scenarios supports requirements elaboration. We demonstrate the relevance of input-output implied scenarios using a number of examples.

Fluent model checking is an automated technique for verifying that an event-based operational model satisfies some state-based declarative properties. The link between the event-based and state-based formalisms is defined through "fluents" which are state predicates whose value are determined by the occurrences of initiating and terminating events that make the fluents values become true or false, respectively.The existing fluent temporal logic is convenient for reasoning about untimed event-based models but difficult to use for timed models. The paper extends fluent temporal logic with temporal operators for modelling timed properties of discrete-time event-based models. It presents two approaches that differ on whether the properties model the system state after the occurrence of each event or at a fixed time rate. Model checking of timed properties is made possible by translating them into the existing untimed framework.

Exploring alternative options is at the heart of the requirements and design processes. Different alternatives contribute to different degrees of achievement of non-functional goals about system safety, security, performance, usability, and so forth. Such goals in general cannot be satisfied in an absolute, clear-cut sense. Various qualitative and quantitative frameworks have been proposed to support the assessment of alternatives for design decision making. In general they lead to limited conclusions due to the lack of accuracy and measurability of goal formulations and the lack of impact propagation rules along goal contribution links.

Goal-oriented methods are increasingly popular for elaborating software requirements. They provide systematic support for incrementally building intentional, structural and operational models of the software and its environment together with various techniques for early analysis, e.g., to manage conflicting goals or anticipate abnormal environment behaviors that prevent goals from being achieved. On the other hand, tabular event-based methods are well-established for specifying operational requirements for control software. They provide sophisticated techniques and tools for late analysis of software behavior models through, e.g., simulation, model checking or table exhaustiveness checks. We propose to take the best out of these two worlds to engineer requirements for control software. It presents a technique for deriving event-based specifications, written in the SCR tabular language, from operational specifications built according to the KAOS goal-oriented method. The technique consists in a series of transformation steps each of which resolves semantic, structural or syntactic differences between the KAOS source language and the SCR target language. Some of these steps need human intervention and illustrate the kind of semantic subtleties that need to be taken into account when integrating multiple formalisms. As a result of our technique SCR specifiers may use upstream goal-based processes a la KAOS for the incremental elaboration, early analysis, organization and documentation of their tables while KAOS modelers may use downstream tables a la SCR for later analysis of the behavior models derived from goal specifications.

Goal orientation is an increasingly recognized paradigm for eliciting, structuring, analyzing and documenting system requirements. Goals are statements of intent ranging from high-level, strategic concerns to low-level, technical requirements on the software-to-be and assumptions on its environment. Achieving goals require the cooperation of agents such as software components, input/output devices and human agents. The assignment of responsibilities for goals to agents is a critical decision in the requirements engineering process as alternative agent assignments define alternative system proposals.The paper describes a systematic technique to support the process of refining goals, identifying agents, and exploring alternative responsibility assignments. The underlying principles are to refine goals until they are assignable to single agents, and to assign a goal to an agent only if the agent can realize the goal.There are various reasons why a goal may not be realizable by an agent, e.g., the goal may refer to variables that are not monitorable or controllable by the agent. The notion of goal realizability is first defined on formal grounds; it provides a basis for identifying a complete taxonomy of realizability problems. From this taxonomy we systematically derive a catalog of tactics for refining goals and identifying agents so as to resolve realizability problems. Each tactics corresponds to the application of a formal refinement pattern that relieves the specifier from verifying the correctness of refinements in temporal logic.Our techniques have been used in two case studies of significant size; excerpts are shown to illustrate the main ideas.

Goal orientation is an increasingly recognized paradigm for eliciting, modeling, specifying and analyzing software requirements. Goals are statements of intent organized in AND/OR refinement structures; they range from high-level, strategic concerns to low-level, technical requirements on the software-to-be and assumptions on its environment. The operationalization of system goals into specifications of software services is a core aspect of the requirements elaboration process for which little systematic and constructive support is available. In particular, most formal methods assume such operational specifications to be given and focus on their a posteriori analysis.The paper considers a formal, constructive approach in which operational software specifications are built incrementally from higher-level goal formulations in a way that guarantees their correctness by construction. The operationalization process is based on formal derivation rules that map goal specifications to specifications of software operations; more specifically, these rules map real-time temporal logic specifications to sets of pre-, post- and trigger conditions. The rules define operationalization patterns that may be used for guiding and documenting the operationalization process while hiding all formal reasoning details; the patterns are formally proved correct once and for all. The catalog of operationalization patterns is structured according to a rich taxonomy of goal specification patterns.

Requirements engineering is concerned with the elicitation of high-level goals to be achieved by the envisioned system, the refinement of such goals and their operationalization into specifications of services and constraints and the assignment of responsibilities for the resulting requirements to agents such as humans, devices and software. Requirements engineering processes often result in goals, requirements, and assumptions about agent behavior that are too ideal; some of them are likely not to be satisfied from time to time in the running system due to unexpected agent behavior. The lack of anticipation of exceptional behaviors results in unrealistic, unachievable, and/or incomplete requirements. As a consequence, the software developed from those requirements will not be robust enough and will inevitably result in poor performance or failures, sometimes with critical consequences on the environment. This paper presents formal techniques for reasoning about obstacles to the satisfaction of goals, requirements, and assumptions elaborated in the requirements engineering process. The techniques are based on a temporal logic formalization of goals and domain properties; they are integrated into an existing method for goal-oriented requirements elaboration with the aim of deriving more realistic, complete, and robust requirements specifications. A key principle is to handle exceptions at requirements engineering time and at the goal level, so that more freedom is left for resolving them in a satisfactory way. The various techniques proposed are illustrated and assessed in the context of a real safety-critical system.

Requirements engineering is concerned with the elicitation of high-level goals to be achieved by the system envisioned, the refinement of such goals and their operationalization into services and constraints, and the assignment of responsibilities for the resulting requirements to agents such as humans, devices and software. Requirements engineering processes may often result in requirements and assumptions about agent behaviour that are too ideal; some of them are likely to be violated from time to time in the running system due to unexpected agent behaviour. The lack of anticipation of exceptional behaviors results in unrealistic, unachievable and/or incomplete requirements. As a consequence, the software developed from those requirements will inevitably result in poor performance, sometimes with critical consequences on the environment. This paper proposes systematic techniques for reasoning about obstacles to the satisfaction of goals, requirements, and assumptions elaborated in the requirements engineering process. These techniques are integrated into an existing method for goal-driven requirements elaboration with the aim of deriving more complete and realistic requirements. The concept of obstacle is first defined precisely. Formal techniques and domain-independent heuristics are then proposed for identifying obstacles from goal/assumption formulations and domain properties. The paper then discusses techniques for resolving obstacles by transformation of the goals, requirements and assumptions elaborated so far in the process, or by introduction of new ones. Numerous examples are given throughout the paper to suggest how the techniques can be usefully applied in practice.

A wide range of inconsistencies can arise during requirements engineering as goals and requirements are elicited from multiple stakeholders. Resolving such inconsistencies sooner or later in the process is a necessary condition for successful development of the software implementing those requirements. The paper first reviews the main types of inconsistency that can arise during requirements elaboration, defining them in an integrated framework and exploring their interrelationships. It then concentrates on the specific case of conflicting formulations of goals and requirements among different stakeholder viewpoints or within a single viewpoint. A frequent, weaker form of conflict called divergence is introduced and studied in depth. Formal techniques and heuristics are proposed for detecting conflicts and divergences from specifications of goals/requirements and of domain properties. Various techniques are then discussed for resolving conflicts and divergences systematically by the introduction of new goals or by transforming the specifications of goals/objects toward conflict-free versions. Numerous examples are given throughout the paper to illustrate the practical relevance of the concepts and techniques presented. The latter are discussed in the framework of the KAOS methodology for goal-driven requirements engineering.