Sensor selection is critical for designing effective online monitoring systems for safety–critical applications. This paper proposes a novel sensor selection framework that utilizes qualitative system models to evaluate various sensor configurations based on multiple criteria. The criteria assess capabilities like fault diagnostics, risk reduction, observability, functionality, integrability, and cost. The framework uses the Integrated System Failure Analysis to generate signal features from qualitative system models. These features are used to evaluate sensor configurations against the selection criteria. The criteria can be applied as constraints or objectives for optimization. The Non-dominated Sorting Genetic Algorithm handles the multi-objective optimization to find Pareto optimal sensor deployment solutions. The framework is demonstrated on a reactor cavity cooling system case study, generating optimal configurations considering temperature, flow, pressure, density, and radiation sensors. The framework aids online monitoring system design by recommending sensor deployment configurations that balance critical capabilities. Qualitative models provide effective analysis despite the lack of operational data. The flexible criteria and multi-objective optimization enable extensive exploration of configurations in early development stages.

In the software development life cycle, the quality of the requirements specification affects the overall quality of the subsequent phases and hence, the software product. The requirements specification is usually inspected by an inspection team to detect defects. To enhance the quality of the requirements specification, one conventional strategy usually used is adding redundancies to the inspection team. However, this strategy suffers from the problem of dependent failures of the redundant inspectors which was not studied systematically in previous research. To analyze the dependent failures and independent failures in an inspection team, this paper first defines the independent failures and dependent failures in an inspection team from the perspective of human errors. Then a quantification model, i.e., the Zone-based Model, is proposed to analyze the dependent and independent failures. The Zone-based Model considers the following situations: 1) the probability of failures of an inspector may be high; 2) the probability of failures of the inspectors may be different; 3) the failures in an inspection team can be a combination of dependent failures and independent failures. By considering all those situations, the Z model has a meaningful interpretation and a convincing assessment of the failures of an inspection team. To verify the effectiveness of the new model, the Zone-based model is compared to conventional models using simulation data. The results show that the Zone-based model is significantly better than the traditional models in analyzing the independent and dependent failures.

Hardware-in-the-loop systems are a type of Hard real-time system that require a faster than real-time simulation. A method of running hard real-time systems with slower than real-time simulations is devised. Instead of waiting for the real-time subsystem output to a simulation, multiple simulation input predictions are generated and used to run several parallel simulation paths independent of the real-time subsystem. The simulation path results are stored until needed by the real-time subsystem at which time one is selected and returned to it to continue the execution of the system. A test case using a small modular reactor nuclear power plant hardware-in-the-loop system was used to demonstrate the effects of slower than real-time simulations on a hard real-time system and to show that this method allows such a simulation to be used.

Reliability and safety assurance are of supreme importance in the implementation of digital safety-critical control systems. A deductive method integrated with simulation-based fault injection and testing is presented for out-of-range permanent software fault localization. For fault modeling, an input-output mapping scheme is proposed to characterize the behavior of software modules and represent failure modes in an analogous manner to hardware state definitions. The Markov/cell-to-cell-mapping scheme is used for diagnostics. The diagnostic process is illustrated by several case studies for a boiling water reactor feedwater control system. The case study results show that the diagnostic algorithm is capable of software fault localization in the presence of both single and multiple faults.

The papers in this special section were presented at the 2014 International Symposium on Software Reliability Engineering (ISSRE) that was held from 3 to 6 November 2014 in Naples, Italy.

Software-driven hardware configurations account for the majority of modern safety-critical complex systems. The often costly failures of such systems can be attributed to software specific, hardware specific, or software/hardware interaction failures. The understanding of how failures propagate in such complex systems might provide critical information to designers, because, while a software component may not fail in terms of loss of function, a software operational state can cause an associated hardware failure. The least expensive phase of the product life cycle to address failures is during the design stage. This research presents a means to evaluate how a combined software/hardware system behaves and how such failures propagate to result in potential failures downstream, during the conceptual design stage. In particular, this paper proposes the use of high-level system modeling and model-based reasoning approaches to model failure propagation in combined software-hardware systems, introducing the Function-Failure Identification and Propagation (FFIP) analysis framework to help formalize the design of safety-critical systems. The fact that the hardware and software designers do not share the same background, knowledge, methods, or language contributes significantly to software/hardware interaction failures. A high-level systems analysis method, such as FFIP, is geared toward the unification of language and modeling concepts and may help to more seamlessly bridge such a gap. The technique is applied to the design of the Reaction Control System Jet Selection of the NASA space shuttle to evaluate failure propagation within the Reaction Control System Jet selection, specifically for the redundancy management system. The paper concludes with the extensions and mappings to the software domain that are required for a truly integrated methodology.

The influence of the software, and its interaction and interdependency with the hardware in the creation and propagation of hardware failures, are usually neglected in reliability analyses of safety critical systems. The software operation is responsible for the usage of semiconductor devices along the system lifetime. This usage consists of voltage changes and current flows that steadily degrade the materials of circuit devices until the degradation becomes permanent, and the device can no longer perform its intended function. At the circuit level, these failures manifest as stuck-at values, signal delays, or circuit functional changes. These failures are permanent in nature. Due to the extremely high scaling of complementary metal-oxide-semiconductor (CMOS) technology into deep submicron regimes, permanent hardware failures are a key concern, and can no longer be neglected compared to transient failures in radiation-intense applications. Our work proposes a methodology for the reliability analysis of permanent failure manifestations of hardware devices due to the usage induced by the execution of embedded software applications. The methodology is illustrated with a case study based on a safety critical application.

Model-based testing is an effective black-box test generation technique for applications. Existing model-based testing techniques, however, fail to capture implicit domain-specific properties, as they overtly rely on software artifacts such as design documents, requirement specifications, etc., for completeness of the test model. This article presents a technique, HOTTest, which uses a strongly typed domain-specific language to model the system under test. This allows extraction of type-related system invariants, which can be related to various domain-specific properties of the application. Thus, using HOTTest, it is possible to automatically extract and embed domain-specific requirements into the test models. In this article we describe HOTTest, its principles and methodology, and how it is possible to relate domain-specific properties to specific type constraints. HOTTest is described using the example of HaskellDB, which is a Haskell-based embedded domain-specific language for relational databases. We present an example application of the technique and compare the results to some other commonly used Model-based test automation techniques like ASML-based testing, UML-based testing, and EFSM-based testing.

As software plays increasingly important roles in modern society, reliable software becomes desirable for all stakeholders. One of the root causes of software failure is the failure of the computer hardware platform on which the software resides. Traditionally, fault injection has been utilized to study the impact of these hardware failures. One issue raised with respect to the use of fault injection is the lack of prior knowledge on the faults injected, and the fact that, as a consequence, the failures observed may not represent actual operational failures. This paper proposes a simulation-based approach to explore the distribution of hardware failures caused by three primary failure mechanisms intrinsic to semiconductor devices. A dynamic failure probability for each hardware unit is calculated. This method is applied to an example Z80 system and two software segments. The results lead to the conclusion that the hardware failure profile is location related, time dependent, and software-specific.

Software reliability has progressively become critical issues due to the number and the nature of the fields invaded. Software development companies need to be provided with the capability to improve their software development process and decrease its related costs. The Software Reliability Engineering Curriculum and in particular a graduate level software reliability modeling course at the University of Maryland was created to respond to this need. In this paper, the contents of the course in the context of a particularly diverse environment are discussed

Software-based digital systems are progressively replacing analog systems in safety-critical applications. However the ability to predict their reliability is not well understood and needs further study. A first step towards systematic resolution of this issue was presented in a recent software engineering measure study. In that study a set of software engineering measures were ranked with respect to their ability in predicting software reliability through an expert opinion elicitation process. This study also proposed a concept of reliability prediction system (RePS) to bridge the gap between software engineering measures and software reliability. The research presented in this paper validates the rankings obtained and the concept of RePS proposed in the previous study.

Probabilistic Risk Assessment is a technique to assess the probability of failure or success of a mission. Results provided by the risk assessment methodology are used to make decisions concerning choice of upgrades, scheduling of maintenance, decision to launch, etc. However, current PRA neglects the contribution of software to the risk of failure of the mission. This paper presents a framework for "Integrating Software into PRA", a methodology for systematic integration of the software contribution to risk in system failure analysis. In particular, we established a software-related failure mode taxonomy and a three-level PRA sub-model to account for the impact of software to the classical PRA structure. Application and validation of the taxonomy are discussed in this paper. Future research is also summarized.

Adequate testing is necessary and important to ensure reliability of software. Most test models are specification-based and fail to capture implicit domain specific properties. This paper presents a technique, which uses a HaskellDB specification of the software to extract domain specific properties and embed them into the test generation model. HaskellDB is an embedded domain specific functional and strongly typed language for database related applications. Specifying using HaskellDB ensures that a set of axioms based on type safeness of the database queries hold for the specification. The implementation of the application should also satisfy these properties and should be tested accordingly. We therefore propose a technique that extracts the axioms automatically from the HaskellDB specification and embeds additional test paths in the test model leading to an enriched test suite. We present an example application of the technique and compare the results against a manual testing technique.

This research proposes a framework based on expert opinion elicitation, developed to select the software engineering measures which are the best software reliability indicators. The current research is based on the top 30 measures identified in an earlier study conducted by Lawrence Livermore National Laboratory. A set of ranking criteria and their levels were identified. The score of each measure for each ranking criterion was elicited through expert opinion and then aggregated into a single score using multiattribute utility theory. The basic aggregation scheme selected was a linear additive scheme. A comprehensive sensitivity analysis was carried out. The sensitivity analysis included: variation of the ranking criteria levels, variation of the weights, variation of the aggregation schemes. The top-ranked measures were identified. Use of these measures in each software development phase can lead to a more reliable quantitative prediction of software reliability.

Two broad categories of human error occur during software development: (1) development errors made during requirements analysis, design, and coding activities; (2) debugging errors made during attempts to remove faults identified during software inspections and dynamic testing. This paper describes a stochastic model that relates the software failure intensity function to development and debugging error occurrence throughout all software life-cycle phases. Software failure intensity is related to development and debugging errors because data on development and debugging errors are available early in the software life-cycle and can be used to create early predictions of software reliability. Software reliability then becomes a variable which can be controlled up front, viz, as early as possible in the software development life-cycle. The model parameters were derived based on data reported in the open literature. A procedure to account for the impact of influencing factors (e.g., experience, schedule pressure) on the parameters of this stochastic model is suggested. This procedure is based on the success likelihood methodology (SLIM). The stochastic model is then used to study the introduction and removal of faults and to calculate the consequent failure intensity value of a small-software developed using a waterfall software development.

The field of software engineering measurement appears to the unfamiliar eye as a chaotic environment lacking unifying principles and rigor. The number of software engineering measures developed over the years is stupefying and keeps increasing. Software engineering measures relate to multiple aspects of the software development process and product. Software development organizations typically select a small number of such software engineering measures to manage their development processes and products. The research presented in this paper is an attempt to help software development organizations identify the software engineering measures that are best predictors of software reliability. The research is based on the top 30 measures identified in an earlier study carried out by Lawrence Livermore National Laboratory (J.D. Lawrence et al., Technical Report UCRL-ID-136035, 1998). The set of ranking criteria was modified to fit the needs of the study. The score of each measure for each ranking criterion was elicited through expert opinion and then aggregated into a single score using multi-attribute utility theory. The basic aggregation scheme selected was a linear additive scheme. A comprehensive sensitivity analysis was carried out. The sensitivity analysis included variation of levels, variation of weights and variation of aggregation schemes.

Most existing software reliability models estimate the failure intensity function during dynamic testing. While these models are useful engineering tools, they can not be applied to earlier life-cycle phases where pay-off is maximum in terms of avoiding later failures. Few models have been developed to capture phenomena occurring early in the life cycle and their impact on reliability and much research effort is still needed in this area. Our previous research efforts have led to the development of a stochastic model, which relates human errors committed during software development and debugging activities to the software failure intensity function. The software development schedule and other influencing factors (e.g., experience, schedule pressure, etc.) were used to predict human error rates. The model is based on several assumptions. In particular, it is assumed that repair is instantaneous. The present paper attempts to remove this assumption and assesses the impact of repair times on previous findings.

Software reliability assessment models in use today treat software as a monolithic block. An aversion towards "atomic" models seems to exist. These models appear to add complexity to the modeling, to the data collection and seem intrinsically difficult to generalize. This paper presents an architecturally based software reliability model and underlines its benefits. The model is based on an architecture derived from the requirements which captures both functional and non-functional requirements and on a generic classification of functions, attributes and failure modes. The model focuses on evaluation of failure mode probabilities and uses a Bayesian quantification framework. Failure mode probabilities of functions and attributes are propagated to the system level using fault trees. The model accounts explicitly for the type of software development life-cycle. It can incorporate any type of prior information, such as results of developers' testing or historical information on a specific functionality and its attributes, and is ideally suited for reusable software. By building an architecture and deriving its potential failure modes, the model forces early appraisal and understanding of the weaknesses of the software, allows reliability analysis of the structure of the system, and provides assessments at a functional level as well as at the system level. The model is applied to a simple example. The model's assessment of reliability tends to be conservative and the model appears to be scalable to larger, more complex problems.

Software reliability prediction models are of paramount importance since they provide early identification of cost overruns, software development process issues, optimal development strategies, etc. Existing prediction models were developed mostly during the past 5 to 10 years and, hence, have become obsolete. Furthermore, they are not based on a deep knowledge and understanding of the software development process. This limits their predictive power. This paper presents an approach to the prediction of software reliability based on a systematic identification of software process failure modes and their likelihoods. A direct consequence of the approach and its supporting data collection efforts is the identification of weak areas in the software development process. A Bayesian framework for the quantification of software process failure mode probabilities is recommended since it allows usage of historical data that are only partially relevant to the software at hand. The approach is applied to the requirements analysis phase.