Thirty years ago, when the TCAS II modeling effort was undertaken, the notion of model-based design and model-based systems engineering were new concepts. The TCAS II modeling effort demonstrated that creating a formal model of a complex system and doing so in collaboration with a diverse group of application experts was eminently feasible and laid the groundwork for future research. In this retrospective, we revisit the effort in the context of model-based systems engineering, summarize the most relevant lessons learned, and discuss the state of model-based techniques today and steps to the future.

When I was EIC of TSE and before, software design approaches were matched with the type of application. Since that time, software has become a major component in the control of potentially dangerous systems and has grown enormously in size. Most of these systems require high assurance of important properties, such as safety and security, and sometimes, require regulatory certification before they can be used. Today, however, such assurance is usually very expensive and limited in its power. This short paper proposes a change in how software is designed that can greatly simplify the development, assurance, and maintenance and evolution of critical control software.

As transportation systems become increasingly complex and the roles of human operators and autonomous software continue to evolve, traditional safety-related analytical methods are becoming inadequate. Traditional hazard analysis tools are based on an accident causality model that does not capture many of the complex behaviors found in modern engineered systems. Additionally, these traditional approaches are most effective during the late stages of system development, when detailed design information is available. However, system safety cannot be cost-effectively assured by discovering problems at these late stages and adding expensive updates to the design. Rather, safety should be designed into complex intelligent transportation systems from their very conception, which can be achieved by integrating powerful hazard analysis techniques into the general systems engineering process. The primary barrier to achieving this objective is the lack of effectiveness of the existing analytical tools during early concept development. This paper introduces a new technique, which is based on a systems- and control-theoretic model of accident causality that can capture behaviors that are prevalent in these complex software-intensive systems. The goals are to (1) develop rigorous systematic tools for the analysis of future concepts to identify potentially hazardous scenarios and undocumented assumptions and to (2) extend these tools to assist stakeholders in the development of concepts using a safety-driven approach. Current work focuses on air transportation, but future goals of this research are to extend to and generalize all modes of transportation.

The fundamental challenge facing security professionals is preventing losses, be they operational, financial or mission losses. As a result, one could argue that security professionals share this challenge with safety professionals. Despite their shared challenge, there is little evidence that recent advances that enable one community to better prevent losses have been shared with the other for possible implementation. Limitations in current safety approaches have led researchers and practitioners to develop new models and techniques. These techniques could potentially benefit the field of security. This paper describes a new systems thinking approach to safety that may be suitable for meeting the challenge of securing complex systems against cyber disruptions. Systems-Theoretic Process Analysis for Security (STPA-Sec) augments traditional security approaches by introducing a top-down analysis process designed to help a multidisciplinary team consisting of security, operations, and domain experts identify and constrain the system from entering vulnerable states that lead to losses. This new framework shifts the focus of the security analysis away from threats as the proximate cause of losses and focuses instead on the broader system structure that allowed the system to enter a vulnerable system state that the threat exploits to produce the disruption leading to the loss.

Too often, systems are designed and then an attempt is made to add safety features or to prove that the design is safe after the fact. Safety has to be designed into a system from the start-it cannot be effectively added on to a mature design. In addition, the increasing use of software is changing the nature of accident causation in software-intensive systems and our safety engineering techniques must change accordingly. This article will describe a new hazard analysis technique, called STPA, which is effective on software-intensive systems. An advantage of this technique is that it can be used to drive the earliest design decisions and then proceed in parallel with ensuing design decisions and design refinement. Not only is this approach more effective, but the cost is no more than a more conventional design process and potentially much cheaper.

Diverse stakeholders--clinicians, researchers, business leaders, policy makers, and the public--have good reason to believe that the effective use of electronic health care records (EHRs) is essential to meaningful advances in health care quality and patient safety. However, several reports have documented the potential of EHRs to contribute to health care system flaws and patient harm. As organizations (including small hospitals and physician practices) with limited resources for care-process transformation, human-factors engineering, software safety, and project management begin to use EHRs, the chance of EHR-associated harm may increase. The authors propose a coordinated set of steps to advance the practice and theory of safe EHR design, implementation, and continuous improvement. These include setting EHR implementation in the context of health care process improvement, building safety into the specification and design of EHRs, safety testing and reporting, and rapid communication of EHR-related safety flaws and incidents.

Reuse of application software has been limited and sometimes has led to accidents. This paper suggests some requirements for successful and safe application software reuse and demonstrates them using a case study on a real spacecraft.

Traditional accident models were devised to explain losses caused by failures of physical devices in relatively simple systems. They are less useful for explaining accidents in software-intensive systems and for nontechnical aspects of safety such as organizational culture and human decision-making. This paper describes how systems theory can be used to form new accident models that better explain system accidents (accidents arising from the interactions among components rather than individual component failure), software-related accidents, and the role of human decision-making. Such models consider the social and technical aspects of systems as one integrated process and may be useful for other emergent system properties such as security. The loss of a Milstar satellite being launched by a Titan/Centaur launch vehicle is used as an illustration of the approach.

Software engineering degrees and classes are sprouting up everywhere – particularly outside computer science departments. Attempts to extend Professional Engineer licensing, while wellintended, may make it impossible for those with computer science degrees to be licensed. At the same time, the proper content of software engineering education is debatable. For example, do those who will work on embedded software to control spacecraft require the same education as those who will be writing business software? Does software engineering education rightly belong within computer science departments or would it be better outside traditional CS departments? This talk will explore alternative views of the future of software engineering education and training in order to encourage discussion of these and other questions.

The readability of formal requirements specification languages is hypothesized as a limiting factor in the acceptance of formal methods by the industrial community. An empirical study was conducted to determine how various factors of state-based requirements specification language design affect readability using aerospace applications. Six factors were tested in all, including the representation of the overall state machine structure, the expression of triggering conditions, the use of macros, the use of internal broadcast events, the use of hierarchies, and transition perspective (going-to or coming-from). Subjects included computer scientists as well as aerospace engineers in an effort to determine whether background affects notational preferences. Because so little previous experimentation on this topic exists on which to build hypotheses, the study was designed as a preliminary exploration of what factors are most important with respect to readability. It can serve as a starting point for more thorough and carefully controlled experimentation in specification language readability.

A limiting factor in the industrial acceptance of formal specifications is their readability, particularly for large, complex engineering systems. We hypothesize that multiple visualizations generated from a common model will improve the, requirements creation, reviewing and understanding, process. Visual representations, when effective, provide cognitive support by highlighting the most relevant interactions and aspects of a specification for a particular use. In this paper, we propose a taxonomy and some preliminary principles for designing visual representations of formal specifications. The taxonomy and principles are illustrated by sample visualizations we created while trying to understand a formal specification of the MD-11 flight management system.

This paper examines and proposes an approach to writing software specifications, based on research in systems theory, cognitive psychology and human-machine interaction. The goal is to provide specifications that support human problem solving and the tasks that humans must perform in software development and evolution. A type of specification, called intent specifications, is constructed upon this underlying foundation.

Validation of software requirements is an important part of software engineering.This paper describes a new safety analysis technique called software deviation analysis to help identify weaknesses in how software handles an imperfect environment.The technique propagates deviations in software inputs to output deviations.A qualitative analysis is used to improve the search efficiency.

This paper describes an integrated approach to safety analysis of software requirements and demonstrates the feasibility and utility of applying the individual techniques and the integrated approach on the requirements specification of a guidance system for a high-speed civil transport being developed at NASA Ames. Each analysis found different types of errors in the specification; thus together the techniques provided a more comprehensive safety analysis than any individual technique. We also discovered that the more the analyst knew about the application and the model, the more successful they were in finding errors. Our findings imply that the most effective safety-analysis tool will assist rather than replace the analyst.

No abstract available.

This paper describes methods for automatically analyzing formal, state-based requirements specifications for some aspects of completeness and consistency. The approach uses a low-level functional formalism, simplifying the analysis process. State-space explosion problems are eliminated by applying the analysis at a high level of abstraction; i.e., instead of generating a reachability graph for analysis, the analysis is performed directly on the model. The method scales up to large systems by decomposing the specification into smaller, analyzable parts and then using functional composition rules to ensure that verified properties hold for the entire specification. The analysis algorithms and tools have been validated on TCAS II, a complex, airborne, collision-avoidance system required on all commercial aircraft with more than 30 passengers that fly in U.S. Airspace.

This paper describes methods for automatically analyzing formal, state-based requirements specifications for completeness and consistency.The approach uses a low-level functional formalism, simplifying the analysis process.State space exploslon problems are eliminated by applying the analysis at a high level of abstraction;i.e, instead of generating a reachability graph for analysis, the analysis is performed directly on the model.The method scales up to large systems by decomposing the specification into smaller, analyzable parts and then using functional composition rules to ensure that verified properties hold for the entire specification.The analysis algorithms and tools have been validated on TCAS II, a complex, airborne, collision-avoidance system reqmred on all commercial aircraft with more than 30 passengers that fly in U.S. airspace.

The paper describes an approach to writing requirements specifications for process-control systems, a specification language that supports this approach, and an example application of the approach and the language on an industrial aircraft collision avoidance system (TCAS II). The example specification demonstrates: the practicality of writing a formal requirements specification for a complex, process-control system; and the feasibility of building a formal model of a system using a specification language that is readable and reviewable by application experts who are not computer scientists or mathematicians. Some lessons learned in the process of this work, which are applicable both to forward and reverse engineering, are also presented.>

Article High-pressure steam engines and computer software Share on Author: Nancy G. Leveson View Profile Authors Info & Claims ICSE '92: Proceedings of the 14th international conference on Software engineeringJune 1992 Pages 2–14https://doi.org/10.1145/143062.143076Published:01 June 1992 6citation753DownloadsMetricsTotal Citations6Total Downloads753Last 12 Months20Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

A set of criteria is defined to help find errors in, software requirements specifications. Only analysis criteria that examine the behavioral description of the computer are considered. The behavior of the software is described in terms of observable phenomena external to the software. Particular attention is focused on the properties of robustness and lack of ambiguity. The criteria are defined using an abstract state-machine model for generality. Using these criteria, analysis procedures can be defined for particular state-machine modeling languages to provide semantic analysis of real-time process-control software requirements.>

The authors compared two major approaches to the improvement of software-software fault elimination and software fault tolerance-by examination of the fault detection (and tolerance, where applicable) of five techniques: run-time assertions, multiversion voting, functional testing augmented by structural testing, code reading by stepwise abstraction, and static data-flow analysis. The focus was on characterizing the sets of faults detected by the techniques and on characterizing the relationships between these sets of faults. Two categories of questions were investigated: (1) comparison between fault elimination and fault tolerance techniques and (2) comparisons among various testing techniques. The results provide information useful for making decisions about the allocation of project resources, show strengths and weaknesses of the techniques studies, and indicate directions for future research.>

The authors have conducted a large-scale experiment in N-version programming. A total of 27 versions of a program were prepared independently from the same specification at two universities. The results of executing the versions revealed that the versions were individually extremely reliable but that the number of input cases in which more than one failed was substantially more than would be expected if they were statistically independent. After the versions had been executed, the failures of each version were examined and the associated faults located. It appears that minor differences in the software development environment would not have a major impact in reducing the incidence of faults that cause correlated failures.>

The results of an empirical study of software error detection using self checks and N-version voting are presented. Working independently, each of 24 programmers first prepared a set of self checks using just the requirements specification of an aerospace application, and then each added self checks to an existing implementation of that specification. The modified programs were executed to measure the error-detection performance of the checks and to compare this with error detection using simple voting among multiple versions. The analysis of the checks revealed that there are great differences in the ability of individual programmers to design effective checks. It was found that some checks that might have been effective failed to detect an error because they were badly placed, and there were numerous instances of checks signaling nonexistent errors. In general, specification-based checks alone were not as effective as specification-based checks combined with code-based checks. Self checks made it possible to identify faults that had not been detected previously by voting 28 versions of the program over a million randomly generated inputs. This appeared to result from the fact that the self checks could examine the internal state of the executing program, whereas voting examines only final results of computations. If internal states had to be identical in N-version voting systems, then there would be no reason to write multiple versions.>

This paper presents an approach to providing a rigorous basis for ascertaining whether or not a given set of software requirements is internally complete, i.e., closed with respect to questions and inferences that can be made on the basis of information included in the specification. Emphasis is placed on aspects of software requirements specifications that previously have not been adequately handled, including timing abstractions, safety, and robustness.

The authors have identified a difficulty in the implementation of N-version programming. The problem, called the consistent comparison problem, arises for applications in which decisions are based on the results of comparing finite-precision numbers. It is shown that when versions make comparisons involving the results of finite-precision calculations, it is impossible to guarantee the consistency of their results. It is therefore possible that correct versions may arrive at completely different outputs for an application that does not apparently have multiple correct solutions. If this problem is not dealt with explicitly, an N-version system may be unable to reach consensus even when none of its component versions falls.>

The application of Time Petri net modeling and analysis techniques to safety-critical real-time systems is explored and procedures described which allow analysis of safety, recoverability, and fault-tolerance.

N-version programming has been proposed as a method of incorporating fault tolerance into software. Multiple versions of a program (i.e. `N') are prepared and executed in parallel. Their outputs are collected and examined by a voter, and, if they are not identical, it is assumed that the majority is correct. This method depends for its reliability improvement on the assumption that programs that have been developed independently will fail independently. An experiment is described in which the fundamental axiom is tested. In all, 27 versions of a program were prepared independently from the same specification at two universities and then subjected to one million tests. The results of the tests revealed that the programs were individually extremely reliable but that the number of tests in which more than one program failed was substantially more than expected. The results of these tests are presented along with an analysis of some of the faults that were found in the programs. Background information on the programmers used is also summarized.

With the increased use of software controls in critical realtime applications, a new dimension has been introduced into software reliability–the "cost" of errors. The problems of safety have become critical as these applcations have increasingly included areas where the consequences of failure are serious and may involve grave dangers to human life and property. This paper defines software safety and describes a technique called software fault tree analysis which can be used to analyze a design as to its safety. The technique has been applied to a program which controls the flight and telemetry for a University of California spacecraft. A critical failure scenario was detected by the technique which had not been revealed during substantial testing of the program. Parts of this analysis are presented as an example of the use of the technique and the results are discussed.