This practical experience report explores Neural Machine Translation (NMT) models’ capability to generate offensive security code from natural language (NL) descriptions, highlighting the significance of contextual understanding and its impact on model performance. Our study employs a dataset comprising real shellcodes to evaluate the models across various scenarios, including missing information, necessary context, and unnecessary context. The experiments are designed to assess the models’ resilience against incomplete descriptions, their proficiency in leveraging context for enhanced accuracy, and their ability to discern irrelevant information. The findings reveal that the introduction of contextual data significantly improves performance. However, the benefits of additional context diminish beyond a certain point, indicating an optimal level of contextual information for model training. Moreover, the models demonstrate an ability to filter out unnecessary context, maintaining high levels of accuracy in the generation of offensive security code. This study paves the way for future research on optimizing context use in AI-driven code generation, particularly for applications requiring a high degree of technical precision such as the generation of offensive code.

AI-based code generators are an emerging solution for automatically writing programs starting from descriptions in natural language, by using deep neural networks (Neural Machine Translation, NMT). In particular, code generators have been used for ethical hacking and offensive security testing by generating proof-of-concept attacks. Unfortunately, the evaluation of code generators still faces several issues. The current practice uses output similarity metrics, i.e., automatic metrics that compute the textual similarity of generated code with ground-truth references. However, it is not clear what metric to use, and which metric is most suitable for specific contexts. This work analyzes a large set of output similarity metrics on offensive code generators. We apply the metrics on two state-of-the-art NMT models using two datasets containing offensive assembly and Python code with their descriptions in the English language. We compare the estimates from the automatic metrics with human evaluation and provide practical insights into their strengths and limitations.

Presents the conference keynote speech, plenary speech, or messages from conference chairs.

Abstract Writing software exploits is an important practice for offensive security analysts to investigate and prevent attacks. In particular, shellcodes are especially time-consuming and a technical challenge, as they are written in assembly language. In this work, we address the task of automatically generating shellcodes, starting purely from descriptions in natural language, by proposing an approach based on Neural Machine Translation (NMT). We then present an empirical study using a novel dataset ( Shellcode_IA32 ), which consists of 3200 assembly code snippets of real Linux/x86 shellcodes from public databases, annotated using natural language. Moreover, we propose novel metrics to evaluate the accuracy of NMT at generating shellcodes. The empirical analysis shows that NMT can generate assembly code snippets from the natural language with high accuracy and that in many cases can generate entire shellcodes with no errors.

Writing exploits for security assessment is a challenging task. The writer needs to master programming and obfuscation techniques to develop a successful exploit. To make the task easier, we propose an approach (EVIL) to automatically generate exploits in assembly/Python language from descriptions in natural language. The approach leverages Neural Machine Translation (NMT) techniques and a dataset that we developed for this work. We present an extensive experimental study to evaluate the feasibility of EVIL, using both automatic and manual analysis, and both at generating individual statements and entire exploits. The generated code achieved high accuracy in terms of syntactic and semantic correctness.

The paper is part of the Difference Makers Track of FIE 2020.Many institutions of higher learning are in the process of defining and implementing Data Science programs. The emerging field is revolutionizing scientific discovery and many industries. Broad availability of data sets and emerging analytical techniques for their processing are changing our economies and societies. While there is a broad agreement about underlying principles in this new discipline, defining data science as a pedagogically independent discipline has proven to be a challenge.In this paper, we present the university-wide effort that led to the creation of a new undergraduate interdisciplinary Data Science program. In particular, we point to the critical role of inclusive pedagogical design and broadening participation criteria in program development. Current and projected job market data clearly indicate that data science programs need to expand traditional STEM workforce and attract students with varying backgrounds and degrees or preparation. Such a focus has led to a broad agreement across our campus regarding the new program and its curriculum.

This Research-to-Practice Work-in-Progress Paper presents a midway report on a change initiative underway in the College of Computing and Informatics (CCI) at UNC Charlotte. Comprised of approximately 100 faculty members and nearing 2,000 undergraduate majors, CCI was awarded a 5-year, $2 million grant from the National Science Foundation in 2015 to revolutionize computer science education at the collegiate level. This initiative, which seeks to simultaneously achieve both pedagogical and organizational change, is built upon a foundation of educational innovation through engaged teaching practices. To sustain educational innovation in CCI beyond the funded scope of the project, pedagogical change is thus strategically embedded in CCI's organizational structure through a 3-stage model of faculty adoption, redesigned student course evaluations, and realigned values in the reappointment, promotion, and tenure process. By recognizing that sustained organizational change takes root when there is concurrent buy-in from organizational members—both bottom-up and top-down— this reform initiative seeks to embed two-pronged change not only via educational innovation (pedagogical change), but also in the day-to-day practices, policies, and physical environment of the College itself (organizational change).

The Connected Learner is a re-orientation of undergraduate computing education that focuses on connecting students to peers, the profession, and purpose. The College of Computing and Informatics at UNC Charlotte—comprised of three departments with an undergraduate enrollment of approximately 1,500 students—is a large research institution in an urban setting with a diverse student population. Within this unique context, the project aim is to build a sustainable practice of educational innovation across the undergraduate computing curriculum by increasing faculty awareness of teaching innovations, resources for pedagogical change, and support for teaching practices that engage students. The vision is to create an active learning environment that transforms the student entering the computing undergraduate program from a person with an interest in computing to a person with an affinity identity [1] as a computing professional through these ongoing connections. We employ a systems theory of change for the Connected Learner based on two foundational concepts: flipped classrooms and engagement theory. To achieve related goals of improving student retention and time to graduation, Connected Learner teaching strategies are being integrated across the undergraduate curriculum with an initial focus on introductory gateway courses. Change is occurring via infrastructure supports to sustain learning practices across our college through faculty development initiatives (hiring, training, mentoring, and incentives) that are designed to inform faculty about engagement pedagogies, motivate faculty to adopt these practices, and shift pedagogical attitudes. During the first year of the five-year project, quantitative and qualitative data from students and faculty was collected and analyzed. This data provides a baseline of student attitudes and performance as well as a baseline of faculty attitudes and teaching practices. In this paper, we present the background, context, organizational structure, and research questions for the project. Findings from Year 1 are discussed, including academic outcomes, project milestones, and student attitudes towards new teaching practices. Long-term project goals and expectations are presented.

Accurate detection of software components that need to be exposed to additional verification and validation offers the path to high quality products while minimizing non essential software assurance expenditures. In this type of quality modeling we assume that software modules with known fault content developed in similar environment are available. Supervised learning algorithms are the traditional methods of choice for training on existing modules. The models are then used to predict fault content for newly developed software components prior to product release. However, one needs to realize that establishing whether a module contains a fault or not, only to be used for model training, can be expensive. The basic idea behind semi-supervised learning is to learn from a small number of software modules with known fault content and supplement model training with modules for which the fault information is not available, thus reducing the overall cost of quality assurance. In this study, we investigate the performance of semi-supervised learning for software fault prediction. A preprocessing strategy, multidimensional scaling, is embedded in the approach to reduce the dimensional complexity of software metrics used for prediction. Our results show that the dimension-reduction with semi-supervised learning algorithm preforms significantly better than one of the best performing supervised learning algorithm - random forest - in situations when few modules with known fault content are available. We compare our results with the published benchmarks and clearly demonstrate performance benefits.

Fingerprints are likely the most widely used biometric in commercial as well as law enforcement applications. With the expected rapid growth of fingerprint authentication in mobile devices their importance justifies increased demands for dependability. An increasing number of new sensors, applications and a diverse user population also intensify concerns about the interoperability in fingerprint authentication. In most applications, fingerprints captured for user enrollment with one device may need to be "matched" with fingerprints captured with another device. We have performed a large-scale study with 494 participants whose fingerprints were captured with 4 different industry-standard optical fingerprint devices. We used two different image quality algorithms to evaluate fingerprint images, and then used three different matching algorithms to calculate match scores. In this paper we present a comprehensive analysis of dependability and interoperability attributes of fingerprint authentication and make empirically-supported recommendations on their deployment strategies.

Security of fingerprint authentication systems remains threatened by the presentation of spoof artifacts. Most current mitigation approaches rely upon the fingerprint liveness detection as the main anti-spoofing mechanisms. However, liveness detection algorithms are not robust to sensor variations. In other words, typical liveness detection algorithms need to be retrained and adapted to each and every sensor used for fingerprint capture. In this paper, inspired by popular invariant feature descriptors such as histograms of oriented gradients (HOG) and the scale invariant feature transform (SIFT), we propose a new invariant descriptor of fingerprint ridge texture called histograms of invariant gradients (HIG). The proposed descriptor is designed to preserve robustness to variations in gradient positions. Spoofed fingerprints are detected using multiple histograms of invariant gradients computed from spatial neighborhoods within the fingerprint. Results show that proposed method achieves an average accuracy comparable to the best algorithms of the Fingerprint Liveness Detection Competition 2013, while being applicable with no change to multiple acquisition sensors.

Accurate detection of defects prior to product release helps software engineers focus verification activities on defect prone modules, thus improving the effectiveness of software development. A common scenario is to use the defects from prior releases to build the prediction model for the upcoming release, typically through a supervised learning method. As software development is a dynamic process, fault characteristics in subsequent releases may vary. Therefore, supplementing the defect information from prior releases with limited information about the defects from the current release detected early seems to offer intuitive and practical benefits. We propose active learning as a way to automate the development of models which improve the performance of defect prediction between successive releases. Our results show that the integration of active learning with uncertainty sampling consistently outperforms the corresponding supervised learning approach. We further improve the prediction performance with feature compression techniques, where feature selection or dimensionality reduction is applied to defect data prior to active learning. We observe that dimensionality reduction techniques, particularly multidimensional scaling with random forest similarity, work better than feature selection due to their ability to identify and combine essential information in data set features. We present the improvements offered by this methodology through the prediction of defective modules in the three successive versions of Eclipse.

Issue tracking systems play a critical role in software maintenance by allowing users and developers to submit problem reports for observed failures. A major problem in these systems is that two or more users can, and do, submit reports describing the same issue. Automated classification of such duplicate problem reports is an area of active research. The corpus of existing research shows a slow improvement in classification accuracy using relatively small subsets of problem report data. When applied to an entire project's problem repository, they exhibit a reduction in performance. In this paper we propose a novel duplicate report detection approach using multi-label classification. We use a suite of 24 duplicate classification techniques and MULAN software package to train a multi-label classifier. This multi-label classifier selects a set of similarity measures (from a pool of measures) that are most likely to find the true primary report. To demonstrate its effectiveness the method was tested on the entire Firefox repository. This data set encompasses 12+ years of problem reports and contains over 30,000 duplicate reports. Our results indicate that multi-label classification boosts the performance of the individual measures by up to 40% while returning overall results that match or outperform existing methods. The proposed method uses less than 1% of the dataset for training.

The identification of fault-prone modules has a significant impact on software quality assurance. In addition to prediction accuracy, one of the most important goals is to detect fault prone modules as early as possible in the development lifecycle. Requirements, design, and code metrics have been successfully used for predicting fault-prone modules. In this paper, we investigate the benefits of the incremental development of software fault prediction models. We compare the performance of these models as the volume of data and their life cycle origin (design, code, or their combination) evolve during project development. We analyze 14 data sets from publicly available software engineering data repositories. These data sets offer both design and code metrics. Using a number of modeling techniques and statistical significance tests, we confirm that increasing the volume of training data improves model performance. Further models built from code metrics typically outperform those that are built using design metrics only. However, both types of models prove to be useful as they can be constructed in different phases of the life cycle. Code-based models can be used to increase the effectiveness of assigning verification and validation activities late in the development life cycle. We also conclude that models that utilize a combination of design and code level metrics outperform models which use either one metric set exclusively.

In this paper we study the problems of intra-spectral and cross-spectral face recognition (FR) in homogeneous and heterogeneous environments. Specifically we investigate the advantages and limitations of matching (i) short wave infrared (SWIR) face images to visible images under controlled or uncontrolled conditions, (ii) mid-wave infrared (MWIR) to MWIR or visible images under controlled conditions, and (iii) intra-distance near infrared (NIR) to NIR images and cross-distance, cross-spectral NIR to visible images. All NIR images were captured night-time, outdoors and at mid-ranges (from 30 up to 120 meters). We utilized both commercial and academic face matchers and performed a set of experiments indicating that our cross-photometric score level fusion rule can be utilized to improve SWIR cross-spectral matching performance across all FR scenarios investigated. We also show that intra-spectral matching results, using either MWIR or NIR images, are comparable to the baseline results, i.e., when comparing visible to visible face images. Our experiments also indicate that the level of improvement in recognition performance is scenario dependent. Experiments also show that cross-spectral matching (the heterogeneous problem, where gallery and probe sets have face images acquired in different spectral bands) is a very challenging problem and it requires further investigation to address real-world law enforcement or military situations.

Accurate detection of fault prone modules offers the path to high quality software products while minimizing non essential assurance expenditures. This type of quality modeling requires the availability of software modules with known fault content developed in similar environment. Establishing whether a module contains a fault or not can be expensive. The basic idea behind semi-supervised learning is to learn from a small number of software modules with known fault content and supplement model training with modules for which the fault information is not available. In this study, we investigate the performance of semi-supervised learning for software fault prediction. A preprocessing strategy, multidimensional scaling, is embedded in the approach to reduce the dimensional complexity of software metrics. Our results show that the semi-supervised learning algorithm with dimension-reduction preforms significantly better than one of the best performing supervised learning algorithms, random forest, in situations when few modules with known fault content are available for training.

In this paper we study the problem of cross spectral face recognition in heterogeneous environments. Specifically we investigate the advantages and limitations of matching short wave infrared (SWIR) face images to visible images under controlled or uncontrolled conditions. The contributions of this work are three-fold. First, three different databases are considered, which represent three different data collection conditions, i.e., images acquired in fully controlled (indoors), semi-controlled (indoors at standoff distances ≥ 50m), and uncontrolled (outdoor operational conditions) environments. Second, we demonstrate the possibility of SWIR cross-spectral matching under controlled and challenging scenarios. Third, we illustrate how photometric normalization and our proposed cross-photometric score level fusion rule can be utilized to improve cross-spectral matching performance across all scenarios. We utilized both commercial and academic (texture-based) face matchers and performed a set of experiments indicating that SWIR images can be matched to visible images with encouraging results. Our experiments also indicate that the level of improvement in recognition performance is scenario dependent.

When designing critical applications, trade offs between different security solutions and their performance implications are common. Unfortunately, understanding the precise implications of such tradeoffs early in the system development lifecycle is difficult. This paper proposes a methodology for combined analysis of performance and security risk. We transform system requirements into a Layered Queueing Network (LQN) model that subsequently provides analytical performance analysis feedback when considering a set of security mechanisms and incurred security risks. We quantify security risks using cost curves. The proposed approach is illustrated through a realistic case study of a border management application.

The problem of face verification across the short wave infrared spectrum (SWIR) is studied in order to illustrate the advantages and limitations of SWIR face verification. The contributions of this work are two-fold. First, a database of 50 subjects is assembled and used to illustrate the challenges associated with the problem. Second, a set of experiments is performed in order to demonstrate the possibility of SWIR cross-spectral matching. Experiments also show that images captured under different SWIR wavelengths can be matched to visible images with promising results. The role of multispectral fusion in improving recognition performance in SWIR images is finally illustrated. To the best of our knowledge, this is the first time cross-spectral SWIR face recognition is being investigated in the open literature.

Self-organization in learning systems refers to the ability of the system to adapt and respond data as it is presented without outside intervention. The ability to self-organize is desirable and is critical in realizing on-line and real-time adaptive systems for applications including control systems, navigation, vision, and speech. In this paper, we focus on self-organizing learning systems which utilize the addition and subtraction of receptor nodes or neurons in some way. Typically, these algorithms store error information and use it to modify the neural network dynamically so that this error will be decreased. Examples of these learning systems include self-organizing maps, growing cell structures, and dynamic cell structures. We describe current methods for growing the number nodes in the case of the Dynamic Cell Structures neural network and discuss issues via examples that could lead to potentially incorrect data representation during the implementation of the algorithm. A new algorithm is provided that overcomes the observed flaw and enables these learning systems to grow and operate in an optimal and robust manner. The analysis of the proposed optimal growing algorithm indicates that this modified algorithm is more reliable for use in on-line and real-time adaptive systems.

Software as a Service (SaaS) has gained momentum in the past few years and businesses have been increasingly moving to SaaS model for their IT solutions. SaaS is a newer and transformed model where software is delivered to customers as a service over the web. With the SaaS model, there is a need for service providers to ensure that the services are available and reliable for end users at all times, which introduces significant pressure on the service provider to ensure right test processes and methodologies to minimize any impact to the provisions in Service Level Agreements (SLA). There is lack of research on the unique approaches to reliability analysis of SaaS suites. In this paper, we expand traditional approaches to reliability analysis of traditional web servers and propose methods tailored towards assessing the workload and reliability of SaaS applications. In addition we show the importance of data filtration when assessing SaaS reliability from log files. Finally, we discuss the suitability of reliability measures with respect to their relevance in the context of SLAs.

Iris recognition, the ability to recognize and distinguish individuals by their iris pattern, is one of the most reliable biometrics in terms of recognition and identification performance. However, the performance of these systems is affected by poor-quality imaging. In this paper, we extend iris quality assessment research by analyzing the effect of various quality factors such as defocus blur, off-angle, occlusion/specular reflection, lighting, and iris resolution on the performance of a traditional iris recognition system. We further design a fully automated iris image quality evaluation block that estimates defocus blur, motion blur, off-angle, occlusion, lighting, specular reflection, and pixel counts. First, each factor is estimated individually, and then, the second step fuses the estimated factors by using a Dempster-Shafer theory approach to evidential reasoning. The designed block is evaluated on three data sets: Institute of Automation, Chinese Academy of Sciences (CASIA) 3.0 interval subset, West Virginia University (WVU) non-ideal iris, and Iris Challenge Evaluation (ICE) 1.0 dataset made available by National Institute for Standards and Technology (NIST). Considerable improvement in recognition performance is demonstrated when removing poor-quality images selected by our quality metric. The upper bound on computational complexity required to evaluate the quality of a single image is O(n2log n).

Software fault prediction models play an important role in softwarequality assurance. They identify software subsystems (modules,components, classes, or files) which are likely to contain faults.These subsystems, in turn, receive additional resources forverification and validation activities. Fault prediction models arebinary classifiers typically developed using one of the supervisedlearning techniques from either a subset of the fault data from thecurrent project or from a similar past project. In practice, itis critical that such models provide a reliable predictionperformance on the data not used in training. Variance is animportant reliability indicator of software fault prediction models.However, variance is often ignored or barely mentioned in manypublished studies. In this paper, through the analysis of twelvedata sets from a public software engineering repository from theperspective of variance, we explore the following five questionsregarding fault prediction models:(1) Do different types ofclassification performance measures exhibit different variance? (2)Does the size of the data set imply a more (or less) accurateprediction performance? (3) Does the size of training subset impactmodel's stability? (4) Do different classifiers consistently exhibitdifferent performance in terms of model's variance? (5) Are theredifferences between variance from 1000 runs and 10 runs of 10-fold crossvalidation experiments? Our results indicate that variance is avery important factor in understanding fault prediction models andwe recommend the best practice for reporting variance in empiricalsoftware engineering studies.

Prediction of fault prone software components is one of the most researched problems in software engineering. Many statistical techniques have been proposed but there is no consensus on the methodology to select the "best model" for the specific project. In this paper, we introduce and discuss the merits of cost curve analysis of fault prediction models. Cost curves allow software quality engineers to introduce project-specific cost of module misclassification into model evaluation. Classifying a software module as fault-prone implies the application of some verification activities, thus adding to the development cost. Misclassifying a module as fault free carries the risk of system failure, also associated with cost implications. Through the analysis of sixteen projects from public repositories, we observe that software quality does not necessarily benefit from the prediction of fault prone components. The inclusion of misclassification cost in model evaluation may indicate that even the "best" models achieve performance no better than trivial classification. Our results support a recommendation to adopt cost curves as one of the standard methods for software quality model performance evaluation.

The prediction of fault-prone modules in a software project has been the topic of many studies. In this paper, we investigate whether metrics available early in the development lifecycle can be used to identify fault-prone software modules. More precisely, we build predictive models using the metrics that characterize textual requirements. We compare the performance of requirements-based models against the performance of code-based models and models that combine requirement and code metrics. Using a range of modeling techniques and the data from three NASA projects, our study indicates that the early lifecycle metrics can play an important role in project management, either by pointing to the need for increased quality monitoring during the development or by using the models to assign verification and validation activities.

Most computer systems rely on usernames and passwords as a mechanism for authentication and access control. These credential sets offer weak protection to a broad scope of applications with differing levels of sensitivity. Traditional physiological biometric systems such as fingerprint, face, and iris recognition are not readily deployable in remote authentication schemes. Keystroke dynamics provide the ability to combine the ease of use of username/password schemes with the increased trustworthiness associated with biometrics. Our research extends previous work on keystroke dynamics by incorporating shift-key patterns. The system is capable of operating at various points on a traditional ROC curve depending on application specific security needs. A 1% false accept rate is attainable at a 14% false reject rate. An equal error rate of 5% is suitable for systems requiring a relatively low security. As a username password authentication scheme, our approach decreases the system penetration rate associated with compromised passwords by 95%-99%. Said performance measures can be further improved through optimization of the classification algorithm on a user specific basis

Increased security risk in international travel has resulted in the creation of new programs to determine the admissibility of foreign travelers at official ports of entry within a country. Primary program goals are improving border security and, at the same time, facilitating the flow of legitimate travelers. Major program requirements include the adoption of machine readable travel documents (i.e., passports, visas, etc.), the use of biometric identifiers, and the interoperability among multiple information systems for travelersy identity verification and background checks. Performance analysis of a border inspection system early in its development life-cycle is essential to predict its ability to meet established performance goals, to identify key performance drivers and potential bottlenecks and to suggest possible design improvements. This paper presents our experience with performance evaluation of a hypothetical inspection system. We adopt an analytical modeling technique based on layered queuing networks. Compared with similar studies which use extensive simulations, we observe that our methodology achieves comparably accurate results while being simpler and less costly

In this paper we describe an experiment in which inconsistent results between two tools for testing formal models (and a third used to determine which of the two was correct) led us to a more careful look at the way each tool was being used and a clearer understanding of the output of the tools. For the experiment, we created error-seeded versions of an SCR specification representing a real-world personnel access control system. They were checked using the model checker SPIN and Lurch, our random testing tool for finite-state models. In one case a property violation was detected by Lurch, an incomplete tool, but missed by SPIN, a model checking tool designed for complete verification. We used the SCR Toolset and the Salsa invariant checker to determine that the violation detected by Lurch was indeed present in the specification. We then looked more carefully at how we were using SPIN in conjunction with the SCR Toolset and, eventually, made adjustments so that SPIN also detected the property violation initially detected only by Lurch. Once it was clear the tools were being used correctly and would give consistent results, we did an experiment to determine how they could be combined to optimize completeness and efficiency. We found that combining tools made it possible to verify the specifications faster and with much less memory in most cases

Formal methods for verification of software systems often face the problem of state explosion and complexity. We propose a divide and conquer methodology which leads to component-based verification and analysis of formal requirements specifications expressed using software cost reduction (SCR) models. This paper presents a novel decomposition methodology which identifies components in the given SCR specification and automates related abstraction methods. Further, we propose a verification strategy for modular and decomposable software models. Efficient verification of SCR models is achieved through the use of invariants and proof compositions. Experimental validation of our methodology brought to light the importance of modularity, encapsulation, information hiding and the avoidance of global variables in the context of formal specification models. The advantages of the compositional verification strategy are demonstrated in the analysis of the personnel access control system. Our approach offers significant savings in terms of time and memory requirements needed to perform formal system verification

The last few decades are marked by an unprecedented increase in the complexity and consequence of information technology systems. High assurance software systems must satisfy basic functional service properties that the system intends to deliver, as well as guarantee desirable system properties such as security, safety, timeliness and reliability. Examples of high assurance software systems include command and control systems, nuclear power plants, electronic banking, aerospace systems, automated manufacturing and medical systems. One of the major challenges in high assurance software engineering is to develop well-founded methods for system construction, verification and validation, so they provide critical services with a high degree of confidence in their correctness and quality. The goal of the special issue is to bring together innovative research ideas and advances in the field of high assurance software systems to address these challenges. To this end, the guest editors invited six papers published in the 29th IEEE Annual International Computer Software and Applications Conference (COMPSAC 2005) in the conference theme of High Assurance Software Systems to submit to this special issue, from which four papers after a rigorous review process have been selected to appear in the special issue.

Abstract Traditional software validation methods cannot guarantee the safe behavior of online self‐adaptive systems. These systems are characterized by continual adaptation to changing environmental conditions. We present a novel methodology for validating adaptive software systems based ononline operational monitoring. The methodology inherits its theoretical underpinnings from the generic stability and convergence analysis ofLyapunov's theory. Lyapunov theory is well established in mathematics and control theory, but has not been used before for software validation. The presented validation technique is applied to a neural network‐based online self‐adaptive system,the intelligent flight control system. In this application, environmental changes include system failure modes, such as a stuck stabilator, broken aileron and/or rudder, sensor failure, etc. Our case study for validation is a specific online self‐adaptive system, the intelligent flight control that utilizes dynamic cell structures' neural networks to perform online adaptation. The theoretical foundation and practicability of the presented validation technique make the presented validation approach generally applicable to other types of online adaptive systems. Copyright © 2006 John Wiley & Sons, Ltd.

Practical iris-based identification systems are easily accessible for data collection at the matching score level. In a typical setting, a video camera is used to collect a single frontal view image of good quality. The image is then preprocessed, encoded, and compared with all entries in the biometric database resulting in a single highest matching score. In this paper, we assume that multiple scans from the same iris are available and design the decision rules based on this assumption. We consider the cases where vectors of matching scores may be described by a Gaussian model with dependent components under both genuine and imposter hypotheses. Two test statistics: the plug-in loglikelihood ratio and the average Hamming distance are designed. We further analyze the performance of filter-based iris recognition systems. The model fit is verified using the Shapiro-Wilk test for normality. We show that the loglikelihood ratio with well-estimated maximum-likelihood parameters in it often outperforms the average Hamming distance statistic. The problem of identification with M iris classes is further stated as an (M+1)ary hypothesis testing problem. We use empirical approach, Chernoff bound, and Large Deviations approach to predict the performance of the iris-based identification system. The bound on the probability of error is evaluated as a function of the number of classes and the number of iris scans per class.

The appeal of including biologically inspired soft computing systems such as neural networks in complex computational systems is in their ability to cope with a changing environment. Unfortunately, continual changes induce uncertainty that limits the applicability of conventional verification and validation (V&V) techniques to assure the reliable performance of such systems. At the system input layer, novel data may cause unstable learning behavior, which may contribute to system failures. Thus, the changes at the input layer must be observed, diagnosed, accommodated and well understood prior to system deployment. Moreover, at the system output layer, the uncertainties/novelties existing in the neural network predictions also need to be well analyzed and detected during system operation. Our research tackles the novelty detection problem at both layers using two different methods. We use a statistical learning tool, support vector data description (SVDD), as a one-class classifier to examine the data entering the adaptive component and detect unforeseen patterns that may cause abrupt system functionality changes. At the output layer, we define a reliability-like measure, the validity index. The validity index reflects the degree of novelty associated with each output and thus can be used to perform system validity checks. Simulations demonstrate that both techniques effectively detect unusual events and provide validation inferences in a near-real time manner.

The ability to ensure reliable adaptation is important in safety-critical applications. Traditional software verification and validation techniques cannot account for the time-evolving nature of a system, making them inapplicable for adaptive computing system assurance. In this paper, we propose considering stability of adaptation as a heuristic measure of reliability. We present a stability monitoring technique that detects unstable learning behavior during online operation of adaptive systems. The stability monitoring relies upon Lyapunov-like functions that detect distinct states in learning that bifurcate away from stable behavior. Dempster-Shafer theory is used for combining stability estimates provided by the monitors into an easily interpretable stability belief function. The proposed analysis technique is evaluated using online learning experiments based on data generated by an actual adaptive flight control system. Results indicate that the stability monitoring successfully detects unstable learning conditions. Our approach is one of the first that can be used for the verification, validation and monitoring of adaptive computing applications.

We analyze the performance of an iris based recognition system. We consider a practical setting where matching scores are accessible for collecting data. We assume that multiple scans from the same iris are available and design the decision rules based on this assumption. We show that vectors of matching scores are described by a Gaussian model with dependent components both under the genuine and imposter hypotheses. Two test statistics: the average Hamming distance and the log-likelihood ratio are designed. We show that the log-likelihood ratio with well estimated maximum likelihood parameters in it outperforms the first test statistic. We further use an empirical approach, Chernoff bound, and large deviations approach to predict the performance of the recognition system.

This paper describes a methodology for generating indicators of performance for the dynamic cell structures neural network, a type of growing self-organizing map. The performance indicators are based on the learning architecture of the neural network and are validated using correlation measures of Murphy's rule. Time estimates for neural network convergence are generated based on the current data conditions and the confidence in the neural network, which is provided by the performance indicators. Analytical and experimental results are presented for the dynamic cell structures neural network during its training from the Carnegie Mellon University two-spirals benchmark data.

The appeal of including adaptive components in complex computational systems, such as flight control, is in their ability to cope with a changing environment. Neural networks are adopted as a popular soft-computing paradigm to carry out the adaptive learning. The dynamic cell structures (DCS) network is derived as a dynamically growing structure to achieve better adaptability and employed for online learning of the intelligent flight control system (IFCS). As a crucial component of a safety critical system, the DCS networks need to be validated. Within the scope of validating adaptive systems, the validation of neural networks is particularly challenging due to their complexity and nonlinearity. The predictions of DCS networks are difficult to warrant because of the locally poor fitting during a relatively short time of adaptive learning. In this paper, we present the validity index, an estimated confidence interval associated with each output, as a reliability-like measure of the network's prediction performance. Experimental results of validity index on the flight condition data collected from an IFCS simulator demonstrate an effective validation scheme for DCS networks.

Component based development is gaining popularity in the software engineering community. The reliability of components affects the reliability of the system. Different models and theories have been developed to estimate system reliability given the information about system architecture and the quality of the components. Almost always in these models a key attribute of component-based systems, the error propagation between the components, is overlooked and not taken into account in the reliability prediction. We extend our previous work on Bayesian reliability prediction of component based systems by introducing the error propagation probability into the model. We demonstrate the impact of the error propagation in a case study of an automated personnel access control system. We conclude that error propagation may have a significant impact on the system reliability prediction and, therefore, future architecture-based models should not ignore it.

A result checker is a program that checks the output of the computation of the observed program for correctness. Introduced originally by Blum, the result checking paradigm has provided a powerful platform assuring the reliability of software. However, constructing result checkers for most problems requires not only significant domain knowledge but also ingenuity and can be error prone. In this paper we present our experience in validating result checkers using formal methods. We have conducted several case studies in validating result checkers from the commercial LEDA system for combinatorial and geometric computing. In one of our case studies, we detected a logical error in a result checker for a program computing max flow of a graph.

An increasing number of applications operate in heterogeneous computing environments, often with mobile components. Methodologies that help developers assess the ability of such applications to meet their performance requirements throughout the software life-cycle are needed. In particular, early in the design phases, analysis techniques are critical for ensuring the future system's behavior, evaluating and comparing design alternatives. A performability evaluation is the most appropriate means to assess the expected system's ability to perform, including the effects of component failures and repairs. This paper focuses on model-based analysis of performability of mobile software systems. We propose a general methodology that starts from design artifacts expressed in a UML-based notation. Inferred performability models are based on the stochastic activity networks notation. The viability of the proposed approach is demonstrated through its application in a case study.

The application of biometric devices and systems is experiencing significant growth, primarily due to the need for reliable authentication. Verification and validation techniques applicable to these systems are rather immature and ad hoc, yet the consequences of the wide deployment of biometric systems could be significant. In this paper we discuss an approach to validation and reliability estimation of a fingerprint registration software. Our validation approach includes the following three steps; a) The validation of the source code with respect to the system requirements specification; b) the validation of the optimization algorithm, which is in the core of the registration system and c) the automation of testing. Since the optimization algorithm is heuristic in nature, mathematical analysis and test results are used to estimate the reliability of the image registration module.

Accurate prediction of fault prone modules (a module is equivalent to a C function or a C+ + method) in software development process enables effective detection and identification of defects. Such prediction models are especially beneficial for large-scale systems, where verification experts need to focus their attention and resources to problem areas in the system under development. This paper presents a novel methodology for predicting fault prone modules, based on random forests. Random forests are an extension of decision tree learning. Instead of generating one decision tree, this methodology generates hundreds or even thousands of trees using subsets of the training data. Classification decision is obtained by voting. We applied random forests in five case studies based on NASA data sets. The prediction accuracy of the proposed methodology is generally higher than that achieved by logistic regression, discriminant analysis and the algorithms in two machine learning software packages, WEKA [I. H. Witten et al. (1999)] and See5. The difference in the performance of the proposed methodology over other methods is statistically significant. Further, the classification accuracy of random forests is more significant over other methods in larger data sets.

Software-based digital systems are progressively replacing analog systems in safety-critical applications. However the ability to predict their reliability is not well understood and needs further study. A first step towards systematic resolution of this issue was presented in a recent software engineering measure study. In that study a set of software engineering measures were ranked with respect to their ability in predicting software reliability through an expert opinion elicitation process. This study also proposed a concept of reliability prediction system (RePS) to bridge the gap between software engineering measures and software reliability. The research presented in this paper validates the rankings obtained and the concept of RePS proposed in the previous study.

This paper introduces a reliability model, and a reliability analysis technique for component-based software. The technique is named Scenario-Based Reliability Analysis (SBRA). Using scenarios of component interactions, we construct a probabilistic model named Component-Dependency Graph (CDG). Based on CDG, a reliability analysis algorithm is developed to analyze the reliability of the system as a function of reliabilities of its architectural constituents. An extension of the proposed model and algorithm is also developed for distributed software systems. The proposed approach has the following benefits: 1) It is used to analyze the impact of variations and uncertainties in the reliability of individual components, subsystems, and links between components on the overall reliability estimate of the software system. This is particularly useful when the system is built partially or fully from existing off-the-shelf components; 2) It is suitable for analyzing the reliability of distributed software systems because it incorporates link and delivery channel reliabilities; 3) The technique is used to identify critical components, interfaces, and subsystems; and to investigate the sensitivity of the application reliability to these elements; 4) The approach is applicable early in the development lifecycle, at the architecture level. Early detection of critical architecture elements, those that affect the overall reliability of the system the most, is useful in delegating resources in later development phases.

We investigate the dynamics of monitored memory resource utilizations in an operating system under stress using quantitative methods of fractal analysis. In the experiments, we recorded the time series representing various memory related parameters of the operating system. We observed that parameters demonstrate clear multifractal behavior. The degree of fractality of these time series tends to increase as the system workload increases. We conjecture that the H¨ older exponent that measures the local rate of fractality may be used as a quantitative measure of software aging. We propose a simple proactive computer crash avoidance strategy based on the online fractal analysis of system memory resource observations.

One of the goals of verification and validation (V&V) activities for online adaptive control systems is providing assurance that they are able to detect novel system behaviors and provide adequate (safe) control actions. Novel (or abnormal) system behaviors cannot be enumerated or fully and explicitly described in requirements documentation. Therefore, they have to be observed and recognized during the operation. Novelty detection methods, therefore, provide an adequate approach for the V&V purposes. We propose a novelty detection method based on support sector data description (SVDD) as a candidate approach for validating adaptive control systems. As a one-class classifier, the support vector data description is able to form a decision boundary around the learned data domain with very little or no knowledge of data points outside the boundary (outliers). We apply the SVDD techniques for novelty detection as part of the validation on an intelligent flight control system (IFCS). Experimental results show that the SVDD can be adopted as an effective tool for finding indications of the safe region for the learned domain, whereby we are able to separate faulty behavior from normal events.

In this paper we show that the quantization error for Dynamic Cell Structures (DCS) Neural Networks (NN) as defined by Bruske and Sommer provides a measure of the Lyapunov stability of the weight centers of the neural net. We also show, however, that this error is insufficient in itself to verify that DCS neural networks provide stable topological representation of a given fixed input feature manifold. While it is true that DCS generates a topology preserving feature map, it is unclear when and under what circumstances DCS will have achieved an accurate representation. This is especially important in safety critical systems where it is necessary to understand when the topological representation is complete and accurate. The stability analysis here shows that there exists a Lyapunov function for the weight adaptation of the DCS NN system applied to a fixed feature manifold. The Lyapunov function works in parallel during DCS learning, and is able to provide a measure of the effective placement of neural units during the NN's approximation. It does not, however, guarantee the formation of an accurate representation of the feature manifold. Simulation studies from a selected CMU-Benchmark involving the use of the constructed Lyapunov function indicate the existence of a Globally Asymptotically Stable (GAS) state for the placement of neural units, but an example is given where the topology of the constructed network fails to mirror that of the input manifold even though the quantization error continues to decrease monotonically.

This paper describes a novel methodology for predicting fault prone modules. The methodology is based on Dempster-Shafer (D-S) belief networks. Our approach consists of three steps: First, building the Dempster-Shafer network by the induction algorithm; Second, selecting the predictors (attributes) by the logistic procedure; Third, feeding the predictors describing the modules of the current project into the inducted Dempster-Shafer network and identifying fault prone modules. We applied this methodology to a NASA dataset. The prediction accuracy of our methodology is higher than that achieved by logistic regression or discriminant analysis on the same dataset.

The comparison of partition and random sampling methods for software testing has received considerable attention in the literature. A standard criterion for comparisons between random and partition testing based on their expected efficacy in program debugging is the probability of detecting at least one failure causing input in the program's domain. We investigate the relative effectiveness of partition testing versus random testing through the powerful mathematical technique of majorization, which was introduced by Hardy et al. (1952). The tools of majorization and the concepts of Schur (convex and concave) functions (1923) enable us to derive general conditions under which partition testing is superior to random testing and, consequently, to give further insights into the value of partition testing strategies.

Adaptive systems are systems whose function evolves while adapting to current environmental conditions, Due to the real-time adaptation, newly learned data have a significant impact on system behavior When online adaptation is included in system control, anomalies could cause abrupt loss of system functionality and possibly result in a failure. In this paper we present a framework for reasoning about the online adaptation problem. We describe a machine learning tool that sniffs data and detects anomalies before they are passed to the adaptive components for learning. Anomaly detection is based on distance computation. An algorithm for framework evaluation as well as sample implementation and empirical results are discussed. The method we propose is simple and reasonably effective, thus it can be easily adopted for testing.

Formal analysis of software is a powerful analysis tool, but can be too costly. Random search of formal models can reduce that cost, but is theoretically incomplete. However, random search of finite-state machines exhibits an early saturation effect, i.e., random search quickly yields all that can be found, even after a much longer search. Hence, we avoid the theoretical problem of incompleteness, provided that testing continues until after the saturation point. Such a random search is rapid, consumes little memory, is simple to implement, and can handle very large formal models (in one experiment shown here, over 10/sup 178/ states).

This paper studies how details of a particular model can effect the efficacy of a search for detects. We find that if the test method is fixed, we can identity classes of software that are more or less testable. Using a combination of model mutators and machine learning, we find that we can isolate topological features that significantly change the effectiveness of a defect detection tool. More specifically, we show that for one defect detection tool (a stochastic search engine) applied to a certain representation (finite state machines), we can increase the average odds of finding a defect from 69% to 91%. The method used to change those odds is quite general and should apply to other defect detection tools being applied to other representations.

Automated generation of test cases is a prerequisite for fast testing. Whereas the research in automated test data generation addressed the creation of individual test points, test trajectory generation has attracted limited attention. In simple terms, a test trajectory is defined as a series of data points, with each (possibly multidimensional) point relying upon the value(s) of previous point(s). Many embedded systems use data trajectories as inputs, including closed-loop process controllers, robotic manipulators, nuclear monitoring systems, and flight control systems. For these systems, testers can either handcraft test trajectories, use input trajectories from older versions of the system or, perhaps, collect test data in a high fidelity system simulator. While these are valid approaches, they are expensive and time-consuming, especially if the assessment goals require many tests. We developed a framework for expanding a small, conventionally developed set of test trajectories into a large set suitable, for example, for system safety assurance. Statistical regression is the core of this framework. The regression analysis builds a relationship between controllable independent variables and closely correlated dependent variables, which represent test trajectories. By perturbing the independent variables, new test trajectories are generated automatically. Our approach has been applied in the safety assessment of a fault tolerant flight control system. Linear regression, multiple linear regression, and autoregressive techniques are compared. The performance metrics include the speed of test generation and the percentage of "acceptable" trajectories, measured by the domain specific reasonableness checks.

Embedded systems have become ubiquitous and essential entities in our ever growing high-tech world. The backbone of today's information-highway infrastructure are embedded systems such as telecommunication systems. They demand high reliability, so as to prevent severe consequences of failures including costly repairs at remote sites. Technology changes mandate that embedded systems evolve, resulting in a demand for techniques for improving reliability of their future system releases. Reliability models based on software metrics can be effective tools for software engineering of embedded systems, because quality improvements are so resource-consuming that it is not feasible to apply them to all modules. Identification of the likely fault-prone modules before system testing, can be effective in reducing the likelihood of faults discovered during operations. A software quality classification model is calibrated using software metrics from a past release, and is then applied to modules currently under development to estimate which modules are likely to be fault-prone. This paper presents and demonstrates an effective case-based reasoning approach for calibrating such classification models. It is attractive for software engineering of embedded systems, because it can be used to develop software reliability models using a faster, cheaper, and easier method. We illustrate our approach with two large-scale case studies obtained from embedded systems. They involve data collected from telecommunication systems including wireless systems. It is indicated that the level of classification accuracy observed in both case studies would be beneficial in achieving high software reliability of subsequent releases of the embedded systems.

Adaptive systems are systems whose function evolves over time, as they improve their performance through learning. The advantage of adaptive systems is that they can, through judicious learning, react to situations that were never individually identified and analyzed by the designer. If learning and adaptation are allowed to occur after the control system is deployed, the system is called an online adaptive system. Online adaptive systems are attracting increasing attention in application domains where autonomy is an important feature, or where it is virtually impossible to analyze ahead of time all the possible combinations of environmental conditions that may arise. An archetype of the former are long term space missions where communication delays to ground stations are prohibitively long, and we have to depend on the systems' local capabilities to deal with unforeseen circumstances. An archetype of the latter are flight control systems, which deal with a wide range of parameters, and a wide range of environmental factors. In recent years NASA conducted experiments evaluating adaptive computational paradigms (neural networks, AI planners) for providing fault tolerance capabilities in control systems following sensor and/or actuator faults. Experimental success suggests significant potential for future use. The critical factor limiting wider use of neural networks and other soft-computing paradigms in process control applications, is our (in)ability to provide a theoretically sound and practical approach to their verification and validation.

It is generally believed that component-based software development leads to improved application quality, maintainability and reliability. However most software reliability techniques model integrated systems. These models disregard system's internal structure, taking into account only the failure data and interactions with the environment. We propose a novel approach to reliability analysis of component-based systems. Reliability prediction algorithm allows system architects to analyze reliability of the system before it is built, taking into account component reliability estimates and their anticipated usage. Fully integrated with the UML, this step can guide the process of identifying critical components and analyze the effect of replacing them with the more/less reliable ones. Reliability assessment algorithm, applicable in the system test phase, utilizes these reliability predictions as prior probabilities. In the Bayesian estimation. framework, posterior probability of failure is calculated from the priors and test failure data.

An essential type of "evidence" of the correctness of the requirements formalization process can be provided by human-based calculation. Human calculation can be significantly amplified by shifting from symbolic representations to graphical representations. Having a formally-defined system model, we can visualize formulas that represent the functional behavior of the system and associated safety constraints. This, in turn, provides an environment for the validation of system models. The visual model strengthens the specifier's ability to grasp the system's complexity, build a correct mental model of the system and write the symbolic specification. The visual model can also serve as a bridge between the domain expert and symbolic specification, and provide evidence of the correctness of the formalization process. As a case study, we developed a virtual environment model for the Production Cell robotic system. The model runs in the ImmersaDesk virtual reality environment. We used this model to evaluate the applicability of virtual reality environments for software requirements validation. Although it introduces higher cost in the requirements formalization phase, this approach can be very beneficial in the development of high-consequence systems.

The behavior of nondeterminate systems can be hard to predict, since similar inputs at different times can generate different outputs. In other words, the behavior seen during the testing process may not be seen at runtime. Due to the uncertainties associated with nondeterminism, the standard view is that we should avoid such nondeterminate systems, especially for systems requiring high reliability. While this is a valid guideline, at least in two application areas such nondeterminacy is unavoidable. Early life-cycle requirements and AI software are becoming widely used, yet both are imprecise and may exhibit nondeterminate behaviour if explored rigorously by a test device. Based on a literature review and some theoretical studies, we argue that many stable properties exist within the space of all possible nondeterminate behaviors. However, we also show that seemingly trivial changes to a nondeterministic system can turn an easily testable system into an impossibly hard system to test. Finally, we stress that this analysis does not imply a correlation between stable zones of nondeterminate testability and the ultimate maintainability of nondeterminate systems. That is, while we are optimistic about testing nondeterminate systems, we remain cautious about the maintenance of such systems.

Automated generation of test cases is a prerequisite for fast testing. Whereas the research has addressed the creation of individual test points, test trajectory generation has attracted limited attention. In simple terms, a test trajectory is defined as a series of data points, with each (possibly multidimensional) point relying upon the value(s) of previous point(s). Software systems that use data trajectories as inputs include closed-loop process controllers. For these systems, software testers can either handcraft test trajectories, use input trajectories from older versions of the system or, perhaps, collect test data in a high fidelity system simulator. While these are valid approaches, they are expensive and time-consuming, especially if the assessment goals require substantial number of tests. We propose a framework for expanding a small, conventionally developed set of test trajectories into a large set suitable, for example, for system safety assurance. In the core of this framework is statistical regression analysis. The regression analysis builds a relationship between controllable independent variables and closely correlated dependent variables, which represent test trajectories. By perturbing the independent variables, new test trajectories can be generated automatically. Automated test trajectory generation has been applied in the safety assessment of a fault tolerant flight control system. We compare the performance of simple linear regression, multiple linear regression and autoregressive techniques.

Knowledge-based engineering and computational intelligence are expected to become core technologies in the design and manufacturing for the next generation of space exploration missions. Yet, if one is concerned with the reliability of knowledge based systems, studies indicate significant disagreement regarding the amount of testing needed for system assessment. The sizes of standard black-box test suites are impracticably large since the black-box approach neglects the internal structure of knowledge-based systems. On the contrary, practical results repeatedly indicate that only a few tests are needed to sample the range of behaviors of a knowledge-based program. In this paper, we model testing as a search process over the internal state space of the knowledge-based system. When comparing different test suites, the test suite that examines larger portion of the state space is considered more complete. Our goal is to investigate the trade-off between the completeness criterion and the size of test suites. The results of testing experiment on tens of thousands of mutants of real-world knowledge based systems indicate that a very limited gain in completeness can be achieved through prolonged testing. The use of simple (or random) search strategies for testing appears to be as powerful as testing by more thorough search algorithms.

Software designers are motivated to utilize off-the-shelf software components for rapid application development. Such applications are expected to have high reliability as a result of deploying trusted components. The claims of high reliability need further investigation based on reliability analysis techniques that are applicable to component-based applications. This paper introduces a probabilistic model and a reliability analysis technique that is applicable to high-level designs. The technique is named scenario-based reliability analysis (SBRA). SBRA is specific to component-based software whose analysis is strictly based on execution scenarios. Using scenarios, we construct a probabilistic model named a "component-dependency graph" (CDG). CDGs are directed graphs that represent components, component reliabilities, link and interface reliabilities, transitions and transition probabilities. In CDGs, component interfaces and link reliabilities are treated as first-class elements of the model. Based on CDGs, an algorithm is presented to analyze the reliability of the application as the function of reliabilities of its components and interfaces. A case study illustrates the applicability of the algorithm. The SBRA is used to identify critical components and critical component interfaces, and to investigate the sensitivity of the application reliability to changes in the reliabilities of components and their interfaces.

Fault avoidance, fault removal and fault tolerance represent three successive lines of defense against the contingency of faults in software systems and their impact on system reliability. Beyond the colorful discussions of the relative merits of these techniques, the law of diminishing returns advocates that they be used in concert, where each is applied whenever it is most effective. Such a premise remains an idle act of faith so long as these techniques cannot be captured by a uniform model. This paper proposes such a model and illustrates how it can be used in practice to improve the quality of software products.

This paper proposes a reliability analysis technique for distributed software systems. The technique is based on scenarios that are modeled as sequence diagrams. Using scenarios, we construct component-dependency graphs (CDG). CDGs have been introduced for reliability analysis of component-based systems. They are extended to serve the complex nature of distributed systems by applying nesting and hierarchy. CDGs include component and link reliabilities, which are treated as first class elements of the model. Based on CDGs, we present an algorithm to analyze the sensitivity of system reliability to reliabilities of its components, subsystems, and links. The proposed analysis technique is useful in identifying critical components and critical component links. An example based on medical informatics standard is presented to illustrate our methodology.

Scenario based dynamic analysis is an important technique used in the verification of specification models for complex real time systems. One of the important problems facing developers of these systems is conducting risk analysis at early stages of development. Our methodology for risk assessment uses colored Petri net (CPN) models for predicting risk factors of system components, based on severity and complexity measures. CPN models are developed from system requirements specifications, and risk analysis provides guidance for identifying high risk components prior to their actual design and implementation. The analysis of the specification models is performed through scenario based simulations. Even though the set of scenarios used for simulation is very important for the success of risk analysis, the scenarios are chosen in an ad hoc fashion, usually guided by the experience of domain experts. Therefore, it is likely that some important scenarios are overlooked, due to the complexity of the system. We propose a technique that increases the likelihood that high risk scenarios are identified. The technique is based on input domain partitioning. Partitions can be determined from the given CPN model automatically. Predicates, which describe subdomains of the input space, assist users in revealing interesting scenarios. This methodology is applied to the assessment of a commanding component of NASA's Earth Observing System (EOS).

In today's world of computers, dealing with huge amounts of data is not unusual. The need to distribute this data in order to increase its availability and increase the performance of accessing it is more urgent than ever. For these reasons it is necessary to develop scalable distributed data structures. We propose EH*, a distributed variant of the Extendible Hashing data structure. It consists of buckets of data that are spread across multiple servers and autonomous clients that can access these buckets in parallel. EH* is scalable in the sense that it grows gracefully, one bucket at a time, to a large number of servers. The communication overhead is relatively independent of the number of servers and clients in the system. EH* offers a high query efficiency and good storage space utilization. The simulation results reveal that the method is comparable to the LH* introduced by W. Litwin (1993).

In the statistical sampling method, as in any other statistical approaches for measuring software reliability, the inputs to the program are chosen according to the estimated probability with which they occur in field use, forming the operational profile. However, in practice it is very difficult to accurately assess the operational distribution of input points. Furthermore, a variety of factors can cause the operational distribution to change during field use making the estimation even more difficult. Musa (1993) has suggested that reducing the size of the input domain simplifies the task of determining operational profiles. We present a class of techniques that reduce the dimensionality of input domains and describe their application. These techniques do not limit the functionality or change the input-output behavior of the program. An additional benefit of these techniques is the insensitivity of the reliability estimate to variations in the operational profile of variables eliminated from the input domain.