As one part of an NSF-sponsored Data Science Fellowship at Cal Poly, San Luis Obispo, a group of faculty offered a unique one-unit quarter-long seminar on the history of ideas behind the core principles of Data Science. We present an overview of this seminar, its learning objectives, and outcomes and lessons learned.

In this paper we discuss a Software Engineering Without Programming course we taught in the Winter 2020 quarter at Cal Poly, San Luis Obispo. We designed this course to target students outside of computing majors, and taught it with the purpose of getting them familiarized with the software development process. To provide software development support, we taught this course in concert with a traditional Software Engineering I course for Software Engineering majors. In this paper we concentrate on reporting our experiences related specifically to the teaching early stages of software development: development of software product vision, and requirements gathering. We discuss the topics covered in the course, the activities students participated in and the artifacts they created. Additionally, we report the results of two surveys (one conducted before the course started, and one conducted during Week 8 of the course) detailing student attitudes towards software development and the course material.

The Structured Query Language (SQL) is powerful, prevalent across many problem domains, and challenging to master. Sophisticated information requests require programmers to set aside familiar procedural and functional modes of thought. Instead, programmers must learn how to apply unfamiliar set-based relational manipulation rules. This paper analyzes the student learning process in an introductory database course. We do this using detailed analysis of problem-solving attempts by 114 students related to 116 SQL lab exercises assigned over a five-week period. We measure student success rates in mastering these SQL concepts, as well as effort expended by students in solving information retrieval problems.

This paper presents the REquirements TRacing On target (RETRO).NET dataset. The dataset includes the requirement specification, the source code files (C# and Visual Basic), the gold standard/answer set for tracing the artifacts to each other, as well as the script used to parse the requirements from the specification (to put in RETRO.NET format). The dataset can be used to support tracing and other tasks.

The goal of RE Cares is to apply our requirements engineering and design and prototyping skills to a problem of societal importance to stakeholders residing in the RE conference locale.

Since their introduction over a year ago, Google's TensorFlow package for learning with multilayer neural networks and their Word2Vec representation of words have both gained a high degree of notoriety. This paper considers the application of TensorFlow-guided learning and Word2Vec-based representations to the problems of classification in requirements documents. In this paper, we compare three categories of machine learning techniques for requirements identification for the SecReq and NFR datasets. The first category is the baseline method used in prior work: Naïve Bayes over word count and TF-IDF representations of requirements. The remaining two categories of techniques are the training of TensorFlow's convolutional neural networks on random and pre-trained Word2Vec embeddings of the words found in the requirements. This paper reports on the experiments we conducted and the accuracy results we achieved.

Fecal contamination in bodies of water is an issue that cities must combat regularly. Often, city governments must restrict access to water sources until the contaminants dissipate. Sourcing the species of the fecal matter helps curb the issue in the future, giving city governments the ability to mitigate the effects before they occur again. Microbial Source Tracking (MST) aims to determine source host species of strains of microbiological lifeforms and library-based MST is one method that can assist in sourcing fecal matter. Recently, the Biology Department in conjunction with the Computer Science Department at California Polytechnic State University San Luis Obispo (Cal Poly) teamed up to build a database called the Cal Poly Library of Pyroprints (CPLOP). Students collect fecal samples, culture and pyrosequence the E. coli in the samples, and insert this data, called pyroprints, into CPLOP. Using two intergenic transcribed spacer regions of DNA, Cal Poly biologists perform studies on strain differentiation. We propose using k-Nearest Neighbors, a straightforward machine learning technique, to classify the host species of a given pyroprint, construct four algorithms to resolve the regions, and investigate classification accuracy.

Can biology students without programming skills solve problems that require computational solutions? They can if they learn to cooperate effectively with computer science students. The goal of the in-concert teaching approach is to introduce biology students to computational thinking by engaging them in collaborative projects structured around the software development process. Our approach emphasizes development of interdisciplinary communication and collaboration skills for both life science and computer science students.

TraceLab is designed to empower future traceability research, through facilitating innovation and creativity, increasing collaboration between researchers, decreasing the startup costs and effort of new traceability research projects, and fostering technology transfer. To this end, it provides an experimental environment in which researchers can design and execute experiments in TraceLab's visual modeling environment using a library of reusable and user-defined components. TraceLab fosters research competitions by allowing researchers or industrial sponsors to launch research contests intended to focus attention on compelling traceability challenges. Contests are centered around specific traceability tasks, performed on publicly available datasets, and are evaluated using standard metrics incorporated into reusable TraceLab components. TraceLab has been released in beta-test mode to researchers at seven universities, and will be publicly released via CoEST.org in the summer of 2012. Furthermore, by late 2012 TraceLab's source code will be released as open source software, licensed under GPL. TraceLab currently runs on Windows but is designed with cross platforming issues in mind to allow easy ports to Unix and Mac environments.

Human analysts working with results from automated traceability tools often make incorrect decisions that lead to lower quality final trace matrices. As the human must vet the results of trace tools for mission- and safety-critical systems, the hopes of developing expedient and accurate tracing procedures lies in understanding how analysts work with trace matrices. This paper describes a study to understand when and why humans make correct and incorrect decisions during tracing tasks through logs of analyst actions. In addition to the traditional measures of recall and precision to describe the accuracy of the results, we introduce and study new measures that focus on analyst work quality: potential recall, sensitivity, and effort distribution. We use these measures to visualize analyst progress towards the final trace matrix, identifying factors that may influence their performance and determining how actual tracing strategies, derived from analyst logs, affect results.

Hierarchical clustering is used in computational biology as a method of comparing sequenced bacterial strain DNA and determining bacterial isolates that belong to the same strain. However, the results of the hierarchical clustering are, at times, difficult to read and interpret. This paper is a case study for the use of a modified hierarchical clustering algorithm, which takes into account the underlying structure of the bacterial DNA isolate collection to which it is applied.

Our research group recently discovered that human analysts, when asked to validate candidate traceability matrices, produce predictably imperfect results, in some cases less accurate than the starting candidate matrices. This discovery radically changes our understanding of how to design a fast, accurate and certifiable tracing process that can be implemented as part of software assurance activities. We present our vision for the new approach to achieving this goal. Further, we posit that human fallibility may impact other software engineering activities involving decision support tools.

Assisted requirements tracing is a process in which a human analyst validates candidate traces produced by an automated requirements tracing method or tool. The assisted requirements tracing process splits the difference between the commonly applied time-consuming, tedious, and error-prone manual tracing and the automated requirements tracing procedures that are a focal point of academic studies. In fact, in software assurance scenarios, assisted requirements tracing is the only way in which tracing can be at least partially automated. In this paper, we present the results of an extensive 12 month study of assisted tracing, conducted using three different tracing processes at two different sites. We describe the information collected about each study participant and their work on the tracing task, and apply statistical analysis to study which factors have the largest effect on the quality of the final trace.

The requirements traceability matrix (RTM) supports many software engineering and software verification and validation (V&V) activities such as change impact analysis, reverse engineering, reuse, and regression testing. The generation of RTMs is tedious and error-prone, though, thus RTMs are often not generated or maintained. Automated techniques have been developed to generate candidate RTMs with some success. When using RTMs to support the V&V of mission-or safety-critical systems, however, a human analyst must vet the candidate RTMs. The focus thus becomes the quality of the final RTM. This paper investigate show human analysts perform when vetting candidate RTMs. Specifically, a study was undertaken at two universities and had 26 participants analyze RTMs of varying accuracy for a Java code formatter program. The study found that humans tend to move their candidate RTM toward the line that represents recall = precision. Participants who examined RTMs with low recall and low precision drastically improved both.

This paper introduces the automation of satisfaction assessment: the process of determining the satisfaction mapping of natural language textual requirements to natural language design elements. Satisfaction assessment is useful because it assists in discovering unsatisfied requirements early in the lifecycle when such issues can be corrected with lower cost and impact than later. We define the basic terms and concepts for this process and explore the feasibility of developing baseline methods for its automation. This paper describes the satisfaction assessment approach algorithmically and then evaluates the effectiveness of two proposed information retrieval (IR) methods in two industrial studies - one based on a large dataset including a complete requirements specification and design specification for a NASA science instrument, and one based on a smaller dataset for an open source project management dataset. We found that both approaches have merit, and that the more sophisticated approach outperformed the simpler approach in terms of overall accuracy of the results.

Java Object XML Mapping (JOXM) is a library that supports the automated persistence and querying of Java objects in a native XML database. The goal of this library is to provide a suitable alternative to standard object-relational mapping (ORM) tools, most notably Hibernate. Unlike techniques such as XML-relational persistence, the storage mechanism provided by JOXM is transparent, allowing the developer to store, retrieve, and query typed Java objects as opposed to plain XML data.

In determining whether to permit a safety-critical software system to be certified and in performing independent verification and validation (IV&V) of safety- or mission-critical systems, the requirements traceability matrix (RTM) delivered by the developer must be assessed for accuracy. The current state of the practice is to perform this work manually, or with the help of general-purpose tools such as word processors and spreadsheets; Such work is error-prone and person-power intensive. In this paper, we extend our prior work in application of Information Retrieval (IR) methods for candidate link generation to the problem of RTM accuracy assessment. We build voting committees from five IR methods, and use a variety of voting schemes to accept or reject links from given candidate RTMs. We report on the results of two experiments. In the first experiment, we used 25 candidate RTMs built by human analysts for a small tracing task involving a portion of a NASA scientific instrument specification. In the second experiment, we randomly seeded faults in the RTM for the entire specification. Results of the experiments are presented.

Zhang and Zhang argue that predictors are useless unless they have high precison&recall. We have a different view, for two reasons. First, for SE data sets with large neg/pos ratios, it is often required to lower precision to achieve higher recall. Second, there are many domains where low precision detectors are useful.

Predicting future success of students as software engineers is an open research area. We posit that current grading means do not capture all the information that may predict whether students will become good software engineers. We use one such piece of information, traceability of project artifacts, to illustrate our argument. Traceability has been shown to be an indicator of software project quality in industry. We present the results of a case study of a University of Waterloo graduate-level software engineering course where traceability was examined as well as course grades (such as mid-term, project grade, etc.). We found no correlation between the presence of good traceability and any of the course grades, lending support to our argument

This paper addresses the issues related to improving the overall quality of the dynamic candidate link generation for the requirements tracing process for verification and validation and independent verification and validation analysts. The contribution of the paper is four-fold: we define goals for a tracing tool based on analyst responsibilities in the tracing process, we introduce several new measures for validating that the goals have been satisfied, we implement analyst feedback in the tracing process, and we present a prototype tool that we built, RETRO (REquirements TRacing On-target), to address these goals. We also present the results of a study used to assess RETRO's support of goals and goal elements that can be measured objectively.

Management of multihierarchical XML encodings has attracted attention of a number of researchers both in databases [8] and in humanities[10]. Encoding documents using multiple hierarchies can yield overlapping markup. Previously proposed solutions to management of document-centric XML with overlapping markup rely on the XML expertise of humans and their ability to maintain correct schemas for complex markup languages.We demonstrate a unified solution for management of complex, multihierarchical document-centric XML. Our framework includes software for storing, parsing, in-memory access, editing and querying, multihierarchical XML documents with conflicting structures.

The building of traceability matrices by those other than the original developers is an arduous, error prone, prolonged, and labor intensive task. Thus, after-the-fact requirements tracing is a process where the right kind of automation can definitely assist an analyst. Recently, a number of researchers have studied the application of various methods, often based on information retrieval after-the-fact tracing. The studies are diverse enough to warrant a means for comparing them easily as well as for determining areas that require further investigation. To that end, we present here an experimental framework for evaluating requirements tracing and traceability studies. Common methods, metrics and measures are described. Recent experimental requirements tracing journal and conference papers are catalogued using the framework. We compare these studies and identify areas for future research. Finally, we provide suggestions on how the field of tracing and traceability research may move to a more mature level.

The process of creation of document-centric XML documents often starts with a prepared textual content, into which the editor introduces markup. In such situations, intermediate XML is almost never valid with respect to the DTD/Schema used for the encoding. At the same time, it is important to ensure that at each moment of time, the editor is working with an XML document that can enriched with further markup to become valid. In this paper we introduce the notion of potential validity of XML documents, which allows us to distinguish between XML documents that are invalid because the encoding is simply incomplete and XML documents that are invalid because some of the DTD rules guiding the structure of the encoding were violated during the markup process. We give a linear-time algorithm for checking potential validity for documents.

We present a database framework for the efficient storage and manipulation of interval probability distributions and their associated information. Although work on interval probabilities and on probabilistic databases has appeared before, ours is the first to combine these into a coherent and mathematically sound framework including both standard relational queries and queries based on probability theory. In particular, our query algebra allows users not only to query existing interval probability distributions, but also to construct new ones by means of conditionalization and marginalization, as well as other more common database operations. © 2004 Wiley Periodicals, Inc. Int J Int Syst 19: 789–815, 2004.

We present an approach for improving requirements tracing based on framing it as an information retrieval (IR) problem. Specifically, we focus on improving recall and precision in order to reduce the number of missed traceability links as well as to reduce the number of irrelevant potential links that an analyst has to examine when performing requirements tracing. Several IR algorithms were adapted and implemented to address this problem. We evaluated our algorithms by comparing their results and performance to those of a senior analyst who traced manually as well as with an existing requirements tracing tool. Initial results suggest that we can retrieve a significantly higher percentage of the links than analysts, even when using existing tools, and do so in much less time while achieving comparable signal-to-noise levels.

We consider the complexity of determining whether differing probability distributions for the same Bayes net, result in different, policies, significantly different, policy outcomes or optimal value functions.

The article describes a novel theoretical framework for uniform storage and management of diverse probabilistic information. The semistructured data model has gained wide acceptance recently as a means of representing data which lacks a rigid structure of schema. In particular, the similarity of the semistructured data model and the underlying data model for eXtensible Markup Language (XML), the emerging open standard for data storage and transmission over the Internet, make our choice of this approach attractive. The authors present the formal model for semistructured probabilistic objects. They provide the theoretical foundations for storing and managing semistructured probabilistic objects. Previously (S. Hawkes and A. Dekhtyar, 2001), we started the process of translating this model into XML. We introduce the advising application and give formal definitions of semistructured probabilistic objects. Finally, we introduce the underlying algebra for semistructured probabilistic databases.

Dyreson and Snodgrass have drawn attention to the fact that, in many temporal database applications, there is often uncertainty about the start time of events, the end time of events, and the duration of events. When the granularity of time is small (e.g., milliseconds), a statement such as “Packet p was shipped sometime during the first 5 days of January, 1998” leads to a massive amount of uncertainty (5×24×60×60×1000) possibilities. As noted in Zaniolo et al. [1997], past attempts to deal with uncertainty in databases have been restricted to relatively small amounts of uncertainty in attributes. Dyreson and Snodgrass have taken an important first step towards solving this problem. In this article, we first introduce the syntax of Temporal-Probabilistic (TP) relations and then show how they can be converted to an explicit, significantly more space-consuming form, called Annotated Relations. We then present a theoretical annotated temporal algebra (TATA). Being explicit, TATA is convenient for specifying how the algebraic operations should behave, but is impractical to use because annotated relations are overwhelmingly large. Next, we present a temporal probabilistic algebra (TPA). We show that our definition of the TP-algebra provides a correct implementation of TATA despite the fact that it operates on implicit, succinct TP-relations instead of overwhemingly large annotated relations. Finally, we report on timings for an implementation of the TP-Algebra built on top of ODBC.