Recently there has been a surge of interest in several prototype software systems that can embed a cloud computing image with user applications into a supercomputer’s hardware architecture. This position paper will summarize these efforts and comment on the advantages of each design and will also discuss some of the challenges that one faces with such software systems. This paper takes the position that specific types of user applications may favor one type of design over another. Different designs may have potential advantages for specific user applications and each design also brings a considerable cost to assure operability and overall computer security. A “one size fits all design” for a cost effective and portable solution for Supercomputer/cloud delivery is far from being a solved problem. Additional research and development should continue exploring various design approaches. In the end several different types of supercomputer/cloud implementations may be needed to optimally satisfy the complexity and diversity of user needs, requirements and security concerns. The authors also recommend that the community recognize a distinction when discussing cluster-type HPC/Cloud versus Supercomputer/Cloud implementations because of the substantive differences between these systems.

All research topics of IEEE NCA*, for more than one decade, have been contributing to the creation of the foundation of the Network/Cloud Computing research and application do domain. Now, Network/Cloud Computing is becoming the major mode of operation of the Information Technology Industry over the Internet. BIG Data, Software Defined Networks (SDN), Network Functions Virtualization (NFI), Internet of Things (IoT) have profound effect on the innovations and modern society. The challenge for Network Cloud Computing is to realize a true Internet of Things, a network capable of supporting potentially trillions of wireless connected devices and with overall bandwidth one thousand times higher that today's wireless networks. Current IT technologies are approaching their limits.

With the rapid growth of data centers and the unprecedented increase in storage demands, the traditional storage control techniques are considered unsuitable to deal with this large volume of data in an efficient manner. The Software Defined Storage (SDStore) comes as a solution for this issue by abstracting the storage control operations from the storage devices and set it inside a centralized controller in the software layer. Building a real SDStore system without any simulation and emulation is considered an expensive solution and may have a lot of risks. Thus, there is a need to simulate such systems before the real-life implementation and deployment. In this paper we present SDStorage, an experimental framework to provide a novel virtualized test bed environment for SDStore systems. The main idea of SDStorage is based on the Mininet Software Defined Network (SDN) Open Flow simulator and is built over of it. The main components of Mininet, which are the host, the switch and the controller, are customized to serve the needs of SDStore simulation environments.

The Computer Science and Software Engineering (CS/SE) profession reports that new college graduates lack the communication skills needed for personal and organizational success. Many CS/SE faculty may omit communication instruction from their courses because they do not want to reduce technical content. We experimented in a software-engineering-intensive second-semester programming course with strategies for improving students' writing of black box test plans that included no instruction on writing the plans beyond the standard lecture on testing. The treatment version of the course used 1) a modified assignment that focused on the plan's readers, 2) a model plan students could consult online, and 3) a modified grading rubric that identified the readers' needs. Three external raters found that students in the treatment sections outperformed students in the control sections on writing for five of nine criteria on rubrics for evaluating the plans and on the raters' holistic impression of the students' technical and communication abilities from the perspectives of a manager and a tester.

Access control policies (ACPs) are necessary mechanisms for protection of critical resources and applications. As operational and security requirements of a system evolve, so do access control policies. It is important to help policy authors in effectively managing access control policies by providing insights into historical trends and evolution patterns of access control policies. We analyzed ACP evolution in three systems: Security Enhanced Linux (SELinux) operating system, Virtual Computing Laboratory (VCL) cloud, and a network intrusion detection (Snort) application. We propose an approach, which extracts evolution patterns based on the analysis of ACP historical change data. An evolution pattern indicates an abstraction of change in the permissions/privileges assigned to a group or a user. We then developed a model of ACPs evolution. We found eight frequently occurring evolution patterns across the three systems. In our context this model can predict evolution patterns in ACPs with a precision of 50-80%, a recall of 70-90% and an F-measure of 65-75%.

Communication skills, including reading, writing, speaking, and teaming, are among the most sought after by employers of recent Computer Science/Software Engineering (CS/SE) graduates. We have been conducting a project that has developed a general framework for program-level integration of communication into CS/SE education. Our framework has been developed through workshops and field evaluation of training materials (and other resources) by faculty from a wide variety of institutions under the guidance of experts in writing and communication across the curriculum. This paper discusses the current state of communication teaching in the CS/SE domain, and presents our framework and experiences for program-wide implementation of communication skills.

The fact that employers regularly list communication skills among their top requirements for graduates of Computer Science programs and that communication skills are specifically included in ABET program outcomes attests to their significance for Computer Science educators. To prepare students, Computer Science programs typically require students to take a course in technical communications or writing intensive courses. A recent trend emerging from the communication specialist suggests that integrating communication skills with content across the Computer Science curriculum is more effective strategy. The three-year NSF-funded CPATH II project (CCF-0939122 and CCF-0939081) "Incorporating Communication Skills into the Computer Science Curriculum" brought together 40 Computer Science educators and communications specialists to develop assignments and communications supports that implement content and communication skills integration. The results are available on the project website (http://cs-comm.lib.muohio.edu/). Computer Science educators who seek to provide graduates of Computer Science programs with excellent communication skills will find this approach relevant and valuable. Those who have developed integrated materials will be encouraged to contribute to the project repository. Those who are interested in increasing the amount of communication taught in their curriculum will be encouraged to adapt assignments from the repository to their needs.

“Incorporating Communication Learning Outcomes Across the Computer Science (CS) and Software Engineering (SE) Curriculum” is an NSF-funded project that aims to identify the communication skills, specifically those involving reading, writing, speaking, and teaming, needed by CS and SE graduates and then determine how these skills can be taught within CS and SE curricula. This paper presents the results of our first two years of studying the programs at Miami University and North Carolina State University to determine the kinds of communication utilized in our courses and how communication is taught and assessed. When looking at student attitudes towards communication our analyses suggest that students do not consider technical (professional) activities such as code reading, writing, and inspections as communication activities, and the same applies to more complex genres such as software testing or design. They also appear to have difficulty relating writing and reading that may take place in the context of the courses to what they might be doing as software engineers and computer scientists once they start working. On the other hand, initial results indicate that our students' communications skills increase steadily throughout their stay in the university. We have also identified potential strategies for improved instruction in communication.

To be successful, CS and SE graduates need strong communication skills (writing, speaking, and teaming), particularly within their discipline. Students exercise these skills during their classes but are not always given explicit domain-specific instruction on these skills, instead relying on instruction provided outside the program. CS and SE faculty are not always comfortable in evaluating these aspects of their assignments and are often unhappy with the results. In this workshop we will lead sessions on teaching writing, speaking, and teaming; situating assignments in workplace-scenarios (to motivate students); and writing communication rubrics that convey faculty expectations to students and support evaluation of student work. For more information, see www.muohio.edu/sigcse_workshop11. A laptop is recommended but not mandatory for this workshop.

A polynomial matrix is said to be locally invertible if it has an invertible subsequence map of equal size between its input and output sequence spaces. This paper examines the use of these matrices, which we call locally invertible encoders, for generating multidimensional convolutional codes. We discuss a novel method of encoding and inverting multidimensional sequences using the subsequence map. We also show that the overlapping symbols between consecutive input subsequences obtained during the sequence inversion can be used to determine if the received sequence is the same as the transmitted codeword.

One way to improve the communication abilities of new software engineering graduates in the workplace is to integrate communication more effectively in the software engineering curriculum. But faculty typically conceive of communication as outside their realm of expertise. Based on the results of an NSF-funded project, we use theories of situated learning and genre to make the case that communication is integral to software engineering and that faculty are in the best position to guide students in becoming better communicators in the field. We identify software engineering genres and show how those genres may be used to integrate communication in the classroom and throughout the curriculum.

According to a study commissioned by the National Institute of Standards and Technology in 2002, software bugs cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the nation's gross domestic product (GDP). The same study also found that more than one-third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure. These numbers would be significantly higher if the study were conducted today.

Software security has both an objective and a subjective component. A lot of the information available about that today is focused on the security vulnerabilities and their disclosure. It is less frequent that security breaches and failures rates are reported, even in open source projects. Disclosure of security problems can take several forms. A disclosure can be accompanied by a release of the fix for the problem, or not. The latter category can be further divided into ”voluntary” and ”involuntary” security issues. In widely used software there is also considerable variability in the operational profile under which the software is used. This profile is further modified by attacks on the software that may be triggered by security disclosures. Therefore a comprehensive model of software security qualities of a product needs to incorporate both objective measures, such as security problem disclosure, repair and, failure rates, as well as less objective metrics such as implied variability in the operational profile, influence of attacks, and subjective impressions of exposure and severity of the problems, etc. We show how a classical Bayesian model can be adapted for use in the security context. The model is discussed and assessed using data from three open source software projects. Our results show that the model is suitable for use with a certain subset of disclosed security faults, but that additional work will be needed to identify appropriate shape and scaling functions that would accurately reflect end-user perceptions associated with security problems.

Log files contain valuable information about the execution of a system. This information is often used for debugging, operational profiling, finding anomalies, detecting security threats, measuring performance etc. The log files are usually too big for extracting this valuable information manually, even though manual perusal is still one of the more widely used techniques. Recently a variety of data mining and machine learning algorithms are being used to analyze the information in the log files. A major road block for the efficient use of these algorithms is the inherent variability present in every log line of a log file. Each log line is a combination of a static message type field and a variable parameter field. Even though both these fields are required, the analyses algorithm often requires that these be separated out, in order to find correlations in the repeating log event types. This disentangling of the message and parameter fields to find the event types is called abstraction of log lines. Each log line is abstracted to a unique ID or event type and the dynamic parameter value is extracted to give an insight on the current state of the system. In this paper we present a technique based on a clustering technique used in the Simple Log file Clustering Tool for log file abstraction. This solution is especially useful when we don't have access to the source code of the application or when the lines in the log file do not conform to a rigid structure. We evaluated our implementation on log files from the Virtual Computing Lab, a cloud computer management system at North Carolina State University, and abstracted it to 727 unique event types.

This paper addresses the challenges of providing fault-tolerance in scientific workflow management. The specification and handling of faults in scientific workflows should be defined precisely in order to ensure the consistent execution against the process-specific requirements. We identified a number of typical failure patterns that occur in real-life scientific workflow executions. Following the intuitive recovery strategies that correspond to the identified patterns, we developed the methodologies that integrate recovery fragments into fault-prone scientific workflow models. Compared to the existing fault-tolerance mechanisms, the propositions reduce the effort of workflow designers by defining recovery fragments automatically. Furthermore, the developed framework implements the necessary mechanisms to capture the faults from the different layers of a scientific workflow management architecture. Experience indicates that the framework can be employed effectively to model, capture and tolerate the typical failure patterns that we identified.

Using effective teaching practices is a high priority for educators. One important pedagogical skill for computer science instructors is asking effective questions. This paper presents a set of instructional principles for effective question asking during guided problem solving. We illustrate these principles with results from classifying the questions that untrained human tutors asked while working with students solving an introductory programming problem. We contextualize the findings from the question classification study with principles found within the relevant literature. The results highlight ways that instructors can ask questions to 1) facilitate students' comprehension and decomposition of a problem, 2) encourage planning a solution before implementation, 3) promote self-explanations, and 4) reveal gaps or misconceptions in knowledge. These principles can help computer science educators ask more effective questions in a variety of instructional settings.

Recent years have seen a growing awareness in the computing education community that initiatives outside the classroom are vital for retaining students and preparing them for a collaborative and dynamic professional environment. Particularly important are programs that develop rich technical skills while increasing students' interest in computing disciplines. We present Computing Identity Mentoring, an intervention designed to increase commitment to computing while enhancing students' technical and leadership skills. This program was implemented at seven universities during 2008-2009. Preliminary results suggest that Computing Identity Mentoring contributes to students' self-efficacy regarding computing and leadership, and solidifies students' commitment to a career in computing. This paper presents early findings on the effectiveness of the approach and illustrates Computing Identity Mentoring in the context of three of the seven institutions where it has been implemented.

Identifying effective tutorial strategies is a key problem for tutorial dialogue systems research. Ongoing work in human-human tutorial dialogue continues to reveal the complex phenomena that characterize these interactions, but we have not yet seen the emergence of an automated approach to discovering tutorial dialogue strategies. This paper presents a first step toward establishing a methodology for such an approach. In this methodology, a corpus is first annotated with dialogue acts that are grounded in theories of tutoring and natural language dialogue. Hidden Markov modeling is then applied to discover tutorial strategies inherent in the structure of the sequenced dialogue acts. The methodology is illustrated by demonstrating how hidden Markov models can be learned from a corpus of human-human tutoring in the domain of introductory computer science.

Existing studies on problem reports in open source projects focus primarily on the analysis of the general category of problem reports, or limit their attention to observations on the number of security problem reports. To evaluate the security of a project, it is necessary to know not only how many security problem reports are logged but also how many are reported and how promptly they are corrected etc. In this paper, we study publicly disclosed security problem reports from eight releases of Fedora, nine releases of Ubuntu, four releases of RedHat Enterprise Linux (RHEL) and two releases of Suse Linux distributions, analyse and discuss which type of problem reports and how frequently they are reported, and how promptly they are corrected. Overall, Fedora and Suse show good results with high and medium severity security problem reports resolved without a backlog. On the other hand, RHEL and Ubuntu show less positive results with presence of backlogs.

Existing studies on the maintenance of open source projects focus primarily on the analyses of the overall maintenance of the projects and less on specific categories like the corrective maintenance. This paper presents results from an empirical study of bug reports from an open source project, identifies user participation in the corrective maintenance process through bug reports, and constructs a model to predict the corrective maintenance effort for the project in terms of the time taken to correct faults. Our study focuses on 72482 bug reports from over nine releases of Ubuntu, a popular Linux distribution. We present three main results: (1) 95% of the bug reports are corrected by people participating in groups of size ranging from 1 to 8 people, (2) there is a strong linear relationship (about 92%) between the number of people participating in a bug report and the time taken to correct it, (3) a linear model can be used to predict the time taken to correct bug reports.

To evaluate security in the context of software reliability engineering, it is necessary to analyse security problems, actual exploits, and their relationship with an understanding of the operational behaviour of the system. That can be done in terms of the effort involved in security exploits, through classic reliability factors such as calendar and inservice time, etc. Existing studies focus primarily on security problems and security exploits. Less attention has been given to the study of the relationship between security problems and security exploits. We present an analysis and classification of 43,710 vulnerabilities from the Open Source National Vulnerability Database and vulnerabilities for two specific products - Bugzilla and FEDORA. About 35% of the published vulnerabilities have been exploited. 34% of the vulnerabilities are disclosed as a result of an exploit and only 1.3% have been exploited after being publicly disclosed. We investigate a unifying approach, to understand security as a component of reliability. We focus on the disclosure and exploits of security problems with respect to calendar time and inservice time, and the impact of such exploits on the process of correcting the security problems, and discuss our approach using the collected data.

An important software reliability engineering tool is operational profiles. In this paper we propose a cost effective automated approach for creating second generation operational profiles using execution logs of a software product. Our algorithm parses the execution logs into sequences of events and produces an ordered list of all possible subsequences by constructing a suffix-array of the events. The difficulty in using execution logs is that the amount of data that needs to be analyzed is often extremely large (more than a million records per day in many applications). Our approach is very efficient. We show that our approach requires O(N) in space and time to discover all possible patterns in N events. We discuss a practical implementation of the algorithm in the context of the logs from a large cloud computing system.

Software repositories provide abundance of valuable information about open source projects. With the increase in the size of the data maintained by the repositories, automated extraction of such data from individual repositories, as well as of linked information across repositories, has become a necessity. In this paper we describe a framework that uses web scraping to automatically mine repositories and link information across repositories. We discuss two implementations of the framework. In the first implementation, we automatically identify and collect security problem reports from project repositories that deploy the Bugzilla bug tracker using related vulnerability information from the National Vulnerability Database. In the second, we collect security problem reports for projects that deploy the Launchpad bug tracker along with related vulnerability information from the National Vulnerability Database. We have evaluated our tool on various releases of Fedora, Ubuntu, Suse, RedHat, and Firefox projects. The percentage of security bugs identified using our tool is consistent with that reported by other researchers.

In the quest to find instructional approaches that benefit student learning, engagement, and retention, evidence suggests providing students with hands-on practice is a worthwhile use of class time. This paper presents results from an exploratory study of two different instructional approaches that were encountered in a study of experienced human tutors working with novice computing students engaged in a programming exercise. No difference in average learning gains was found between a moderate approach, in which students were given control of problem solving nearly half the time, and a proactive approach in which the tutor took initiative nearly three-fourths of the time. Implications of this finding for fine-grained instructional strategy, as well as for broader classroom management decisions, are discussed. This paper also makes the case for the value of one-on-one tutoring studies as an exploratory research methodology for the comparative evaluation of computer science teaching strategies.

Reliability analyses of software systems often focus only on the number of faults reported against the software. Using a broader set of metrics, such as problem resolution times and field software usage levels, can provide a more comprehensive view of the product. Some of these metrics are more readily available for open source products. We analyzed a suite of FEDORA releases and obtained some interesting findings. For example, we show that traditional reliability models may be used to predict problem rates across releases. We also show that security related reports tend to have a different profile than non-security related problem reporting and repair.

Operational profiles are an essential part of software reliability engineering. Typically they are created from the software requirements, and through customer reviews. Creation of operational profiles often is laborious and requires human intervention. Our approach builds an operational profile based on the actual usage from execution logs. The difficulty in using execution logs is that the amount of data to be analyzed is extremely large (more than a million records per day in many applications). Our solution constructs operational profiles by identifying all the possible clustered sequences of events (patterns) that exist in the logs. This is done very efficiently using suffix arrays data structure.

While collaborative approaches in the classroom have been shown to be highly beneficial for students of computer science, obstacles inherent in today's academic environment often prevent collocated collaborative approaches from being implemented. One solution to the collocation problem may lie with tools that facilitate distributed collaboration. This paper presents RIPPLE (Remote Interactive Pair Programming and Learning Environment), a development environment for distributed synchronous collaborative programming. RIPPLE is an open source software tool. Initial user tests demonstrate positive responses from students, and the potential for long term learning, motivation, and retention benefits is significant. In addition to its benefits for students, RIPPLE is a tool for computing education researchers who wish to collect data on collaborative programming.

In industrial practice, information on the software field quality of a product is available too late in the software lifecycle to guide affordable corrective action. An important step towards remediation of this problem lies in the ability to provide an early estimation of post-release field quality. This paper evaluates the Software Testing and Reliability Early Warning for Java (STREW-J) metric suite leveraging the software testing effort to predict post-release field quality early in the software development phases. The metric suite is applicable for software products implemented in Java for which an extensive suite of automated unit test cases are incrementally created as development proceeds. We validated the prediction model using the STREW-J metrics via a two-phase case study approach which involved 27 medium-sized open source projects, and five industrial projects. The error in estimation and the sensitivity of the predictions indicate the STREW-J metric suite can be used effectively to predict post-release software field quality.

Prompted by changes in the numbers and demographics of students enrolled and being retained in computer science, the Department of Computer Science at NC State University is revising its undergraduate curriculum to better meet the needs of its students, and increase student attraction and retention. One set of changes concerns introductory computer science courses (CS1). This paper reports on a study conducted to assess the impact of class size and active learning in our CS1 courses. We find that smaller classes with integrated laboratories improve both learning and retention, as well as satisfaction of the students. Among other benefits, we found retention rates in small classes to be about 20% better than large classes.

A scientific workflow combines data and processes into a configurable, structured set of steps that implement semi-automated computational solutions of a scientific data management or analysis problem. Scientific workflow systems provide graphical user interfaces to combine different technologies along with efficient methods for using them with the goal to increase the efficiency of the scientists. This tutorial provides an introduction to scientific workflow construction and management (Part I) and includes a detailed hands-on session (Part II) using the Kepler system. It is intended for an audience with a computational science background. It will cover principles and foundations of scientific workflows, Kepler environment installation, workflow construction using Kepler library components, and workflow execution management that uses Kepler facilities to provide process and data monitoring and provenance information, as well as high speed data movement solutions. This tutorial also incorporates hands-on exercises and application examples from different scientific disciplines.

In prokaryotes, Shine-Dalgarno (SD) sequences, nucleotides upstream from start codons on messenger RNAs (mRNAs) that are complementary to ribosomal RNA (rRNA), facilitate the initiation of protein synthesis. The location of SD sequences relative to start codons and the stability of the hybridization between the mRNA and the rRNA correlate with the rate of synthesis. Thus, accurate characterization of SD sequences enhances our understanding of how an organism's transcriptome relates to its cellular proteome. We implemented the Individual Nearest Neighbor Hydrogen Bond model for oligo-oligo hybridization and created a new metric, relative spacing (RS), to identify both the location and the hybridization potential of SD sequences by simulating the binding between mRNAs and single-stranded 16S rRNA 3' tails. In 18 prokaryote genomes, we identified 2,420 genes out of 58,550 where the strongest binding in the translation initiation region included the start codon, deviating from the expected location for the SD sequence of five to ten bases upstream. We designated these as RS+1 genes. Additional analysis uncovered an unusual bias of the start codon in that the majority of the RS+1 genes used GUG, not AUG. Furthermore, of the 624 RS+1 genes whose SD sequence was associated with a free energy release of less than -8.4 kcal/mol (strong RS+1 genes), 384 were within 12 nucleotides upstream of in-frame initiation codons. The most likely explanation for the unexpected location of the SD sequence for these 384 genes is mis-annotation of the start codon. In this way, the new RS metric provides an improved method for gene sequence annotation. The remaining strong RS+1 genes appear to have their SD sequences in an unexpected location that includes the start codon. Thus, our RS metric provides a new way to explore the role of rRNA-mRNA nucleotide hybridization in translation initiation.

No single software fault-detection technique is capable of addressing all fault-detection concerns. Similarly to software reviews and testing, static analysis tools (or automated static analysis) can be used to remove defects prior to release of a software product. To determine to what extent automated static analysis can help in the economic production of a high-quality product, we have analyzed static analysis faults and test and customer-reported failures for three large-scale industrial software systems developed at Nortel Networks. The data indicate that automated static analysis is an affordable means of software fault detection. Using the orthogonal defect classification scheme, we found that automated static analysis is effective at identifying assignment and checking faults, allowing the later software production phases to focus on more complex, functional, and algorithmic faults. A majority of the defects found by automated static analysis appear to be produced by a few key types of programmer errors and some of these types have the potential to cause security vulnerabilities. Statistical analysis results indicate the number of automated static analysis faults can be effective for identifying problem modules. Our results indicate static analysis tools are complementary to other fault-detection techniques for the economic production of a high-quality software product.

Although N-version programming has been employed in some mission-critical applications, the reliability and fault correlation issues remain a debatable topic in the research community. In this paper, we perform a comprehensive evaluation on our recent project data on N-version programming and present statistical investigations on coincident failures and correlated faults. Furthermore, we compare our project with NASA 4-University project to identify the "variants" and "invariants" with respect to failure rate, fault density, coincident failures, related faults, and reliability improvement for N-version programming. Our experimental results support fault tolerance as an effective software reliability engineering technique.

A classic question in software development is "How much testing is enough?" Aside from dynamic coverage-based metrics, there are few measures that can be used to provide guidance on the quality of an automatic test suite as development proceeds. This paper utilizes the software testing and reliability early warning (STREW) static metric suite to provide a developer with indications of changes and additions to their automated unit test suite and code for added confidence that product quality will be high. Retrospective case studies to assess the utility of using the STREW metrics as a feedback mechanism were performed in academic, open source and industrial environments. The results indicate at statistically significant levels the ability of the STREW metrics to provide feedback on important attributes of an automatic test suite and corresponding code

Summary form only given. Network and information security is of increasing concern as intruders utilize more advanced technologies, and attacks are occurring much more frequently. A simple intrusion can cause an enterprise financial disaster, a threat to national safety, or loss of human life. Network-based and computer-based intrusion detection systems (IDS's) started appearing some twenty years ago. Now, there are various synchronous and asynchronous tools for external and internal network and host intrusion detection and mitigation using models ranging from signature scanning and pattern matching, to statistical anomaly detection. Although modern tools are much more advanced, they still have many limitations, shortcomings, and open issues. Most tend to be focused on matching known patterns as opposed to discovery of new anomalies through, for example, inductive reasoning about potential anomaly signals. This talk discusses the issues, and the place of pro-active reasoning in the context of identification of security-related anomalies and issues.

Software testing is an integral part of the software development process. Some software developers, particularly those who use the Extreme Programming test-driven development practice, continuously write automated tests to verify their code. We present a tool to complement the feedback loops created by continuous testing. The tool combines static source code metrics with dynamic test coverage for use throughout the development phase to predict a reliability estimate based on a linear combination of these values. Implemented as an open source plug-in to the Eclipse IDE, the tool facilitates the rapid transition between unit test case completions and testing feedback. The color-coded results highlight inadequate testing efforts as well as weaknesses in overall program structure. To illustrate the tool's efficacy, we share the results of its use on university software engineering course projects.

Software inspection has been shown to be an effective defect removal practice, leading to higher quality software with lower field failures. Automated software inspection tools are emerging for identifying a subset of defects in a less labor-intensive manner than manual inspection. This paper investigates the use of automated inspection for a large-scale industrial software system at Nortel Networks. We propose and utilize a defect classification scheme for enumerating the types of defects that can be identified by automated inspections. Additionally, we demonstrate that automated code inspection faults can be used as efficient predictors of field failures and are effective for identifying fault-prone modules.

Test-driven development is a software development practice that has been used sporadically for decades. With this practice, test cases (preferably automated) are incrementally written before production code is implemented. Test-driven development has recently re-emerged as a critical enabling practice of the extreme programming software development methodology. We ran a case study of this practice at IBM. In the process, a thorough suite of automated test cases was produced after UML design. In this case study, we found that the code developed using a test-driven development practice showed, during functional verification and regression tests, approximately 40% fewer defects than a baseline prior product developed in a more traditional fashion. The productivity of the team was not impacted by the additional focus on producing automated test cases. This test suite aids in future enhancements and maintenance of this code. The case study and the results are discussed in detail.

We illustrate how a domain scientist can perform a complex scientific task by interleaving data access, querying, and manipulation, as well as analytical steps and computations in complex, problem specific ways. We show how our system is used by a geneticist for solving the problem of discovering the so-called "co-regulated" genes by interlinking data and computation from several Web sites, local computations, as well as local and remote databases. The main distinctive features of our system (compared, e.g., to the ZOO environment (Ioannidis et al., 1996)) include: (i) executable workflows run as Web services; (ii) abstract workflows employ concept names and semantic types that are higher-level (and thus more "scientist friendly") than executable workflows; and (iii) our system supports automatic translation of the latter into the former.

This paper reports on a teaching and research SLA test-bed that we have constructed using specially developed soft DiffServ routing nodes operating in a Linux environment. The test-bed, and the associated tools, are intended for various levels of study of SLA translation and mapping algorithms. It is instrumented to provide considerable insight into the DiffServ behavior. The software is more modular, flexible and easier to configure than the "standard" Linux DiffServ offerings. This paper presents the architecture of our system, and reports on the proof-of-concept experiments that show that it can be used to implement a general Services Level Agreement (SLA) solution and study related issues. Initial empirical studies involve implementation of some of the quality of service (QoS) related IETF drafts and RFCs in a five-node test-bed, including an implementation and empirical evaluation of expedited forwarding (EF) PHB, assured forwarding (AF) PHB, virtual wire (VW) PDB, and of assured forwarding PDB.

Simultaneous reachability analysis (SRA) is a recently proposed technique to alleviate the state space explosion problem in reachability analysis of concurrent systems. Its goal is to reduce the number of generated states while guaranteeing the detection of certain types of faults in the system such as deadlock and unexecutable transitions. The main idea of SRA is to allow a global transition in a reachability graph to contain a set of local transitions (i.e. transitions of individual processes) such that the state reached by the global transition is independent of the execution order of the associated local transitions. In this paper, we show how to apply the SRA approach to systems modeled as extended finite state machines (EFSM) with multiple ports. Empirical results from applying our SRA algorithm to the dining philosophers problem indicate that our algorithm reduces the number of generated states and the computation time by about 90%.

"Classical" operational-profile based testing strategies tend to advocate testing of product functions and operations according to the relative frequencies of their usage in the field. As such these strategies tend to explicitly or implicitly advocate re-execution of previously "covered" elements. This provides a built-in mechanism for letting the testers know that they are doing well through the resulting declining failure intensity. Unfortunately, in modern resolute constrained software testing environments, the re-execution of functions that have already been tested is often discourage business models that operate with limited budget and schedule. Consequently, in Resource Constrained Testing (RCT) environments failure intensity decay is open to interpretation, and may be an impractical guiding and decision tool. We present an approach based on the Testing Efficiency (TE) Metric that, for RCT environments, may be a more appropriate testing decision support aide. We also present supporting results from simulations. experiments and field application of the TE metric.

The introduction of object-oriented technology does not appear to hinder overall productivity on new large commercial projects, but nor does it seem to improve it in the first two product generations. In practice, the governing influence may be the business workflow, and not the methodology. Copyright © 1999 John Wiley & Sons, Ltd.

The option of obtaining education over networks is quickly becoming a reality for all those who have access to the Internet and the World Wide Web (WWW). However, at present, network-based education (NBE) over the WWW and the Internet in general faces a number of pitfalls. The problems range from inadequate end-user quality of service (QoS), to inadequate materials, to shortcomings in learning paradigms, and to missing or inappropriate student assessment and feedback mechanisms. In this paper, we discuss some major issues that, although mostly solved for NBE, still face Web-based education (WEE). These include the required workflow-oriented technological and quality of service support. In discussing the issues, we use examples from a wide-area NBE/WBE system called NovaNET and a WEE system called Web Lecture System (WLS). We recommend that WEE system developers construct operational user (workflow) profiles before building their content and interfaces. Our experience is that, especially for synchronous WEE systems, user-level round-trip keystroke delays should not exceed about 250 ms and the overall availability of the system (including network-related service failures) should be at least 0.95. We also suggest that a successful WEE system will have a sound auto-adaptive knowledge assessment component, a "virtual" laboratory capability, and a set of strong collaborative functions.

In "classical" testing approaches, "learning" is said to occur if testers dynamically improve the efficiency of their testing as they progress through a testing phase. However, the pressures of modern business and software development practices seem to favor an approach to testing which is very akin to a "sampling without replacement" of a relatively limited number of pre-determined structures and functions conducted under significant schedule and resource constraints. The primary driver is often the desire to "cover" ONLY previously "untested" functions, operations or code constructs, and to meet milestones. We develop and evaluate a model that describes the fault detection and removal process in such an environment. Results indicate that in environments where "coverage" based testing is promoted, but resources and decisions are constrained, very little dynamic "learning" takes place, and that it may be an artifact of program structure or of the test case sequencing policy.

A locally invertible convolutional encoder has a local inverse defined as a full rank w/spl times/w matrix that specifies a one-to-one mapping between equal-length blocks of information and encoded bits. In this correspondence, it is shown that a rate-1/n convolutional encoder is nondegenerate and noncatastrophic if and only if it is locally invertible. Local invertibility is used to obtain upper and lower bounds on the number of consecutive zero-weight branches in a convolutional codeword. Further, existence of a local inverse can be used as an alternate test for noncatastrophicity instead of the usual approach involving computation of the greatest common divisor of n polynomials.

An important class of quality of service (QoS)-dependent network-based applications are computer-based education systems. A successful network-based education (NBE) system needs to provide appropriate QoS at the user level. This includes adequate end-to-end response delay and adequate system reliability and availability. This paper presents results from a reliability and availability evaluation of NovaNET. NovaNET is a successful low-overhead multimedia education system which serves thousands of users on a daily basis. We analyze eight years of failure data and examine correlations among system failure events. The NovaNET data are used to discuss practical bounds on the reliability and availability of an NBE system.

The maximum likelihood voting (MLV) strategy was recently proposed as one of the most reliable voting methods. The strategy determines the most likely correct result based on the reliability history of each software version. However, the theoretical results were obtained under the assumption that inter-version failures are not correlated by common cause faults. We first discuss the issues that arise in practical implementation of MLV, and present an extended version of the MLV algorithm that uses component reliability estimates to break voting ties. We then empirically evaluate the implemented MLV strategy in a situation where the inter-version failures are highly correlated. Our results show that, although in real situations MLV carries no reliability guarantees, it tends to be statistically more reliable, even under high inter-version correlation conditions, than other voting strategies that we have examined. We also compare implemented MLV performance with that of Recovery Block and hybrid Consensus Recovery Block approaches. Our results show that MLV often outperforms Recovery Block and that it can successfully compete with more elaborate Consensus Recovery Block. To the best of our knowledge, this is the first empirical evaluation of the MLV strategy.

We propose a new technique for automatic generation of test cases for predicates. Earlier we proposed an efficient and effective test generation strategy for Boolean expressions. We now extend this strategy to predicates. Our new strategy addresses a number of issues, including: analysis of dependencies between relational expressions in a predicate P; generation of test constraints for P based on the detection of Boolean and relational operator faults in P; and generation of actual tests according to the generated test constraints for P. We propose the use of constraint logic programming (CLP) to automate test data generation for a predicate. Furthermore, we propose an incremental approach to apply CLP techniques to solve a constraint system. Since our technique is specification-based, it can facilitate generation of expected outputs for actual tests.

The authors propose a new technique for the automatic generation of test cases for predicates. Earlier, they proposed an efficient effective test generation strategy for Boolean expressions. They now extend this strategy to predicates. Their new strategy addresses several issues, including: analysis of dependencies between relational expressions in a predicate /spl Pscr/; generation of test constraints for /spl Pscr/ based on the detection of Boolean and relational operator faults in /spl Pscr/; and generation of actual tests according to the generated test constraints for /spl Pscr/. They propose: the use of constraint logic programming (CLP) to automate test-data generation for a predicate; and an incremental approach to apply CLP techniques to solve a constraint system. Since their technique is specification-based, it can facilitate generation of anticipated outputs for actual tests.

Table based error correction and decoding of rate one-half convolutional codes is described. A new class of fast-decodeable locally invertible convolutional codes based on a one-to-one mapping between information and encoded blocks of equal lengths is defined. The syndrome is used as an address to access a correction table which stores pre-computed correction information. The correction table generation process is described and a specific table based correction algorithm is given. Performance of this scheme is analyzed and simulation results are presented.>

We report the results of three empirical studies of fault detection and stability performance of the predicate-based BOR (Boolean Operator) testing strategy. BOR testing is used to develop test cases based on formal software specification, or based on the implementation code. We evaluated the BOR strategy with respect to some other strategies by using Boolean expressions and actual software. We applied it to software specification cause-effect graphs of a safety-related real-time control system, and to a set of N-version programs. We found that BOR testing is very effective at detecting faults in predicates, and that BOR-based approach has consistently better fault detection performance than branch testing, thorough (but informal) functional testing, simple state-based testing, and random testing. Our results indicate that BOR test selection strategy is practical and effective for detection of faulty predicates and is suitable for generation of safety-sensitive test-cases.>

Nonstationary channels (e.g., digital mobile communication channels) require adaptive error control schemes for reliable communication. On these channels, a particular transmission may encounter no errors. Hence, it is desirable to split an encoded sequence into subsequences that are sent in successive transmissions such that each subsequence contains all the information necessary to recover the original message in case of no errors. When errors are present, the subsequences are combined to perform error correction. We show that invertible convolutional codes have this property, and can be used to provide incremental redundancy in a variable-redundancy hybrid ARQ (VR-HARQ) scheme. Invertible convolutional codes provide an alternative to polynomial division otherwise required to extract the original message in convolutional VR-HARQ schemes.>

Modern telecommunications systems are dependent on software for their successful operation. In many cases, the software operates on an established hardware platform and subsequent releases of the system offer primarily in the software component. Because of the increased dependence of the society on advanced telecommunications systems, the reliability and availability of network switching elements are very important. Empirical information on the operational unavailability due to total system outages of a large software-based telecommunications system is presented and discussed. About 50% of these outages are reported as being caused by software. The data is used to show that the unavailability of this system can be described well using a classical two-state availability model.>

The problem of quantifying the field quality of wide-distribution commercial software is addressed. The authors argue that for this type of software the proper quality metric is an estimate of the number of defects remaining in the code. They observe that the apparent number of defects remaining in this type of software is a function of the number of users as well as the number of actual defects in the code. New releases of commercial software normally consist of some code from prior releases and some new or modified code. The authors continue to discover new defects in the code from prior releases even after a new release is in the field. In fact, the new release code appears to stimulate discovery of defects latent in the 'old' code and cause a 'next release effect.' Field defect data from several releases of a widely distributed commercial software product are shown to a demonstrate this effect.>

The strategy of using multiple versions of independently developed software as a means to tolerate residual software design faults is discussed. The effectiveness of multiversion software is studied by comparing estimates of the failure probabilities of these systems with the failure probabilities of single versions. The estimates are obtained under a model of dependent failures and compared with estimates obtained when failures are assumed to be independent. The experimental results are based on 20 versions of an aerospace application developed and independently validated by 60 programmers from 4 universities. Descriptions of the application and development process are given, together with an analysis of the 20 versions.>

The timing performance of N-version multi-stage software is analyzed for a strategy called expedient voting. In expedient voting the voting takes place as soon as an adequate number of components have finished the stage. The concept of a 'runahead' is introduced: the faster versions are allowed to run ahead of the rest of the slower versions by one or more stages, with synchronized re-start in the event of a failure. If the versions are highly reliable, inter-version failure dependence is small, and the difference between the fastest and the slowest successful components in each stage is large, then the execution speed-up through expedient voting may be substantial. Runaheads exceeding three stages offer diminishing returns. Speed-up deteriorates with reduction in the version reliability and independence.>