Cyber-Physical Systems (CPS) often leverage Reinforcement Learning (RL) techniques to adapt dynamically to changing environments and optimize performance. However, it is challenging to construct safety cases for RL components. We therefore propose the SAFE-RL (Safety and Accountability Framework for Evaluating Reinforcement Learning) for supporting the development, validation, and safe deployment of RL-based CPS. We adopt a design science approach to construct the framework and demonstrate its use in three RL applications in small Uncrewed Aerial systems (sUAS).

As autonomous systems increasingly demonstrate more sophisticated reasoning capabilities and make higher-level decisions, the need for interpretable runtime guidance becomes critical. Traditional Models@Runtime serve as abstractions that reflect system state to support adaptation and decision-making by external actors. We extend this paradigm by introducing Ambient Advisory Models, where model components such as classes, agents, or behavioral specifications are augmented with embedded reasoning capabilities that observe, interpret, and advise. Unlike conventional runtime models that provide passive structural or behavioral representations, each model component in our approach becomes an active advisory entity, continuously monitoring its domain of concern and generating contextual guidance. These advisory components operate without direct actuation authority, functioning as cognitive guardrails that provide guidance on safety, regulatory, ethical, and other relevant concerns, while enabling multi-perspective reasoning. Rather than a monolithic reasoning model, we distribute advisory intelligence across individual model components, each maintaining its own reasoning context and concern-specific knowledge. We demonstrate Ambient Advisory Models in autonomous multi-UAV emergency response operations. This approach transforms selected runtime models from reflective artifacts into proactive advisors, enabling a new form of human-AI collaboration where model components actively participate in system governance rather than merely representing system state.

Cyber-Physical Systems (CPS) increasingly leverage Reinforcement Learning (RL) to adapt dynamically to changing environments and optimize performance over time. While RL enhances efficiency and safety by enabling autonomous adjustments to unexpected conditions and hazard avoidance, it also introduces significant risks, as learned behaviors may lead to unpredictable or unsafe actions in real-world deployment. Therefore, integrating risk management into RL system design is essential. In this paper, we propose the QuestRL Framework, a question-driven approach that translates high-level safety guidelines into RL-specific considerations. This framework helps RL practitioners address key risks early in development, informing new or existing system requirements while ensuring traceability to risk management objectives. To evaluate its effectiveness, we conducted a study across two use cases, engaging six RL experts in developing system requirements with and without the framework. Our findings suggest that the framework promotes critical thinking and helps practitioners identify additional risk factors, ultimately supporting safer RL deployment.

The success of Emergency Response (ER) scenarios, such as search and rescue, is often dependent upon the prompt location of a lost or injured person. With the increasing use of small Unmanned Aerial Systems (sUAS) as “eyes in the sky” during ER scenarios, efficient detection of persons from aerial views plays a crucial role in achieving a successful mission outcome. Fatigue of human operators during prolonged ER missions, coupled with limited human resources, highlights the need for sUAS equipped with Computer Vision (CV) capabilities to aid in finding the person from aerial views. However, the performance of CV models onboard sUAS substantially degrades under real-life rigorous conditions of a typical ER scenario, where person search is hampered by occlusion and low target resolution. To address these challenges, we extracted images from the NOMAD dataset and performed a crowdsource experiment to collect behavioural measurements when humans were asked to “find the person in the picture”. We exemplify the use of our behavioral dataset, Psych-ER, by using its human accuracy data to adapt the loss function of a detection model. We tested our loss adaptation on a RetinaNet model evaluated on NOMAD against increasing distance and occlusion, with our psychophysical loss adaptation showing improvements over the baseline at higher distances across different levels of occlusion, without degrading performance at closer distances. To the best of our knowledge, our work is the first human-guided approach to address the location task of a detection model, while addressing real-world challenges of aerial search and rescue. All datasets and code can be found at: https://github.com/ArtRuss/NOMad.

Small uncrewed aerial systems (sUAS) provide an invaluable resource for performing a variety of surveillance, search, and delivery tasks in remote or hostile terrains which may not be accessible by other means. Due to the critical role sUAS play in these situations, it is vital that they are well configured in order to ensure a safe and stable flight. However, it is not uncommon for mistakes to occur in configuration and calibration, leading to failures or incomplete missions. To address this problem, we propose a set of self-adaptive mechanisms and implement them into a self-adaptive framework, Controller Instability-preventing Configuration-Aware Drone Adaptation (CICADA). CICADA dynamically detects unstable drone behavior during flight and adapts to mitigate this threat. We have built a prototype of CICADA using a popular open source sUAS flight control software and experimented with a large number of different configurations in simulation. We then performed a case study with physical drones to determine if our framework will work in practice. Experimental results show that CICADA ’s adaptations reduce controller instability and enable the sUAS to recover from up to 33.8% of poor configurations. In cases where we cannot complete the intended mission, invoking alternative adaptations may still help by allowing the vehicle to loiter or land safely in place, avoiding potentially catastrophic crashes. These safety-focused adaptations can mitigate unsafe behavior in 52.9% to 64.7% of dangerous configurations. We further show that rule-based approaches can be leveraged to automatically select an appropriate adaptation strategy based on the severity of instability encountered, with up to a 14.2% improvement over direct adaptation. Finally, we introduce a variation of our primary adaptation strategy designed to allow more cautious adaptation with limited configuration information, which gets within 6.7% of our primary adaptation strategy despite not requiring an optimal knowledge base.

Small Unmanned Aerial Systems (sUAS) must meet rigorous safety standards when deployed in high-stress emergency response scenarios; however many reported accidents have involved humans in the loop. In this paper, we, therefore, present the HiFuzz testing framework, which uses fuzz testing to identify system vulnerabilities associated with human interactions. HiFuzz includes three distinct levels that progress from a low-cost, limited-fidelity, large-scale, no-hazard environment, using fully simulated Proxy Human Agents, via an intermediate level, where proxy humans are replaced with real humans, to a high-stakes, high-cost, real-world environment. Through applying HiFuzz to an autonomous multi-sUAS system-under-test, we show that each test level serves a unique purpose in revealing vulnerabilities and making the system more robust with respect to human mistakes. While HiFuzz is designed for testing sUAS systems, we further discuss its potential for use in other Cyber-Physical Systems.

Software documentation supports a broad set of software maintenance tasks; however, creating and maintaining high-quality, multi-level software documentation can be incredibly time-consuming and therefore many code bases suffer from a lack of adequate documentation. We address this problem through presenting HGEN, a fully automated pipeline that leverages LLMs to transform source code through a series of six stages into a well-organized hierarchy of formatted documents. We evaluate HGEN both quantitatively and qualitatively. First, we use it to generate documentation for three diverse projects, and engage key developers in comparing the quality of the generated documentation against their own previously produced manually-crafted documentation. We then pilot HGEN in nine different industrial projects using diverse datasets provided by each project. We collect feedback from project stakeholders, and analyze it using an inductive approach to identify recurring themes. Results show that HGEN produces artifact hierarchies similar in quality to manually constructed documentation, with much higher coverage of the core concepts than the baseline approach. Stakeholder feedback highlights HGEN's commercial impact potential as a tool for accelerating code comprehension and maintenance tasks. Results and associated supplemental materials can be found at https://zenodo.org/records/11403244

Software engineering practices such as constructing requirements and establishing traceability help ensure systems are safe, reliable, and maintainable. However, they can be resource-intensive and are frequently underutilized. To alleviate the burden of these essential processes, we developed the Requirements Organization and Optimization Tool (ROOT). ROOT cen-tralizes project information and offers project visualizations and AI-based tools designed to streamline engineering processes. With ROOT's assistance, engineers benefit from improved oversight and early error detection, leading to the successful development of software systems. Link to screen cast: https://youtu.be/3rtMYRnsu24

With the increasing reliance on small Unmanned Aerial Systems (sUAS) for Emergency Response Scenarios, such as Search and Rescue, the integration of computer vision capabilities has become a key factor in mission success. Nevertheless, computer vision performance for detecting humans severely degrades when shifting from ground to aerial views. Several aerial datasets have been created to mitigate this problem, however, none of them has specifically addressed the issue of occlusion, a critical component in Emergency Response Scenarios. Natural Occluded Multi-scale Aerial Dataset (NOMAD) presents a benchmark for human detection under occluded aerial views, with five different aerial distances and rich imagery variance. NOMAD is composed of 100 different Actors, all performing sequences of walking, laying and hiding. It includes 42,825 frames, extracted from 5.4k resolution videos, and manually annotated with a bounding box and a label describing 10 different visibility levels, categorized according to the percentage of the human body visible inside the bounding box. This allows computer vision models to be evaluated on their detection performance across different ranges of occlusion. NOMAD is designed to improve the effectiveness of aerial search and rescue and to enhance collaboration between sUAS and humans, by providing a new benchmark dataset for human detection under occluded aerial views.

The Human Machine Teaming (HMT) paradigm focuses on supporting partnerships between humans and autonomous machines. HMT describes requirements for transparency, augmented cognition, and coordination that enable far richer partnerships than those found in typical human-on-the-loop and human-in-the-loop systems. Autonomous, self-adaptive systems in domains such as autonomous driving, robotics, and Cyber-Physical Systems, are often implemented using the MAPE-K feedback loop as the primary reference model. However, while MAPE-K enables fully autonomous behavior, it does not explicitly address the interactions that occur between humans and autonomous machines as intended by HMT. In this article, we, therefore, present the MAPE-K HMT framework, which utilizes runtime models to augment the monitoring, analysis, planning, and execution phases of the MAPE-K loop to support HMT despite the different operational cadences of humans and machines. We draw on examples from our own emergency response system of interactive, autonomous, small unmanned aerial systems to illustrate the application of MAPE-K HMT in both a simulated and physical environment, and we discuss how the various HMT models are connected and can be integrated into a MAPE-K solution.

Computer Vision (CV) is used in a broad range of Cyber-Physical Systems such as surgical and factory floor robots and autonomous vehicles including small Unmanned Aerial Systems (sUAS). It enables machines to perceive the world by detecting and classifying objects of interest, reconstructing 3D scenes, estimating motion, and maneuvering around objects. CV algorithms are developed using diverse machine learning and deep learning frameworks, which are often deployed on limited resource edge devices. As sUAS rely upon an accurate and timely perception of their environment to perform critical tasks, problems related to CV can create hazardous conditions leading to crashes or mission failure. In this paper, we perform a systematic literature review (SLR) of CV-related challenges associated with CV, hardware, and software engineering. We then group the reported challenges into five categories and fourteen sub-challenges and present existing solutions. As current literature focuses primarily on CV and hardware challenges, we close by discussing implications for Software Engineering, drawing examples from a CV-enhanced multi-sUAS system.

Missing person searches are typically initiated with a description of a person that includes their age, race, clothing, and gender, possibly supported by a photo. Unmanned Aerial Systems (sUAS) imbued with Computer Vision (CV) capabilities, can be deployed to quickly search an area to find the missing person; however, the search task is far more difficult when a crowd of people is present, and only the person described in the missing person report must be identified. It is particularly challenging to perform this task on the potentially limited resources of an sUAS. We therefore propose AirSight, as a new model that hierarchically combines multiple CV models, exploits both onboard and off-board computing capabilities, and engages humans interactively in the search. For illustrative purposes, we use AirSight to show how a person's image, extracted from an aerial video can be matched to a basic description of the person. Finally, as a work-in-progress paper, we describe ongoing efforts in building an aerial dataset of partially occluded people and physically deploying AirSight on our sUAS.

Flight-time failures of small Uncrewed Aerial Systems (sUAS) can have a severe impact on people or the environment. Therefore, sUAS applications must be thoroughly evaluated and tested to ensure their adherence to specified requirements, and safe behavior under real-world conditions, such as poor weather, wireless interference, and satellite failure. However, current simulation environments for autonomous vehicles, including sUAS, provide limited support for validating their behavior in diverse environmental contexts and moreover, lack a test harness to facilitate structured testing based on system-level requirements. We address these shortcomings by eliciting and specifying requirements for an sUAS testing and simulation platform, and developing and deploying it. The constructed platform, DroneReq Validator (DRV), allows sUAS developers to define the operating context, configure multi-sUAS mission requirements, specify safety properties, and deploy their own custom sUAS applications in a high-fidelity 3D environment. The DRV Monitoring system collects runtime data from sUAS and the environment, analyzes compliance with safety properties, and captures violations. We report on two case studies in which we used our platform prior to real-world sUAS deployments, in order to evaluate sUAS mission behavior in various environmental contexts. Furthermore, we conducted a study with developers and found that DRV simplifies the process of specifying requirements-driven test scenarios and analyzing acceptance test results.

Software systems are increasingly expected to address a broad range of stakeholder values representing both personal and societal values as well as values ensconced as laws and regulations. Whereas laws and regulations must be fully addressed, other human values need to be carefully analyzed and prioritized within the context of candidate architectural designs. The majority of prior work has investigated requirements engineering techniques for either regulatory compliance or for human-values, we take an integrated approach which simultaneously considers laws and regulations as well as societal and personal human values throughout the system analysis, specification, and design process. We illustrate our approach through detailed examples drawn from a multi-drone system regulated by the USA Federal Aviation Authority (FAA) and operating in a domain rich with human and societal values. We then discuss requirements engineering challenges and solutions unique to identifying analyzing, and prioritizing human, societal, and regulatory requirements, and ultimately for designing accountable software systems.

Small uncrewed aerial systems, sUAS, provide an invaluable resource for performing a variety of surveillance, search, and delivery tasks in remote or hostile terrains which may not be accessible by other means. Due to the critical role sUAS play in these situations, it is vital that they are well configured in order to ensure a safe and stable flight. However, it is not uncommon for mistakes to occur in configuration and calibration, leading to failures or incomplete missions. To address this problem, we propose a set of self-adaptive mechanisms and implement them into a self-adaptive framework, CICADA, for Controller Instability-preventing Configuration Aware Drone Adaptation. CICADA dynamically detects unstable drone behavior during flight and adapts to mitigate this threat. We have built a prototype of CICADA using a popular open source sUAS simulator and experimented with a large number of different configurations. Experimental results show that CICADA’s adaptations reduce controller instability and enable the sUAS to recover from a significant number of poor configurations. In cases where we cannot complete the intended mission, invoking alternative adaptations may still help by allowing the vehicle to loiter or land safely in place, avoiding potentially catastrophic crashes.

When dealing with safety–critical systems, various regulations, standards, and guidelines stipulate stringent requirements for certification and traceability of artifacts, but typically lack details with regards to the corresponding software engineering process. Given the industrial practice of only using semi-formal notations for describing engineering processes – with the lack of proper tool mapping – engineers and developers need to invest a significant amount of time and effort to ensure that all steps mandated by quality assurance are followed. The sheer size and complexity of systems and regulations make manual, timely feedback from Quality Assurance (QA) engineers infeasible. In order to address these issues, in this paper, we propose a novel framework for tracking, and “passively” executing processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to three case studies: a safety–critical open-source community system, a safety–critical system in the air-traffic control domain, and a non-safety–critical, web-based system. Results from our analysis confirm that trace links are often corrected or completed after the work step has been considered finished, and the engineer has already moved on to another step. Thus, support for timely and automated constraint checking has significant potential to reduce rework as the engineer receives continuous feedback already during their work step.

Runtime monitoring is critical for ensuring safe operation and for enabling self-adaptive behavior of Cyber-Physical Systems (CPS). Monitors are established by identifying runtime properties of interest, creating probes to instrument the system, and defining constraints to be checked at runtime. For many systems, implementing and setting up a monitoring platform can be tedious and time-consuming, as generic monitoring platforms do not adequately cover domain-specific monitoring requirements. This situation is exacerbated when the System under Monitoring (SuM) evolves, requiring changes in the monitoring platform. Most existing approaches lack support for the automated generation and setup of monitors for diverse technologies and do not provide adequate support for dealing with system evolution. In this paper, we present GRuM (Generating CPS Runtime Monitors), a framework that combines model-driven techniques and runtime monitoring, to automatically generate a customized monitoring platform for a given SuM. Relevant properties are captured in a Domain Model Fragment, and changes to the SuM can be easily accommodated by automatically regenerating the platform code. To demonstrate the feasibility and performance we evaluated GRuM against two different systems using TurtleBot robots and Unmanned Aerial Vehicles. Results show that GRuM facilitates the creation and evolution of a runtime monitoring platform with little effort and that the platform can handle a substantial amount of events and data.

Cyber–Physical Systems (CPS) are increasingly used in safety–critical scenarios where ensuring their correct behavior at runtime becomes a crucial task. Therefore, the behavior of the CPS needs to be monitored at runtime so that violations of requirements can be detected. With the inception of edge devices that facilitate runtime analysis at the edge and the increasingly diverse environments that CPS operate in, flexible monitoring approaches are needed that consider the data that needs to be monitored and the analyses performed on that data. In this paper, we propose AMon, a flexible adaptive monitoring framework that supports the specification and validation of monitoring adaptation rules, using a domain-specific language. Based on these rules, AMon automatically generates code for direct deployment onto devices. We evaluated AMon by applying it to TurtleBot Robots and a fleet of Unmanned Aerial Vehicles. Furthermore, we conducted a user study assessing the understandability and ease of use of our language. Results show that creating multiple adaptation rules with our DSL is feasible with minimal effort, and that adaptive monitoring can reduce the amount of runtime data transmitted from the edge device according to the current state of the system and its monitoring needs.

Recent breakthroughs in deep-learning (DL) approaches have resulted in the dynamic generation of trace links that are far more accurate than was previously possible. However, DL-generated links lack clear explanations, and therefore non-experts in the domain can find it difficult to understand the underlying semantics of the link, making it hard for them to evaluate the link's correctness or suitability for a specific software engineering task. In this paper we present a novel NLP pipeline for generating and visualizing trace link explanations. Our approach identifies domain-specific concepts, retrieves a corpus of concept-related sentences, mines concept definitions and usage examples, and identifies relations between cross-artifact concepts in order to explain the links. It applies a post-processing step to prioritize the most likely acronyms and definitions and to eliminate non-relevant ones. We evaluate our approach using project artifacts from three different domains of interstellar telescopes, positive train control, and electronic healthcare systems, and then report coverage, correctness, and potential utility of the generated definitions. We design and utilize an explanation interface which leverages concept definitions and relations to visualize and explain trace link rationales, and we report results from a user study that was conducted to evaluate the effectiveness of the explanation interface. Results show that the explanations presented in the interface helped non-experts to understand the underlying semantics of a trace link and improved their ability to vet the correctness of the link.

Many organizations seek to increase their agility in order to deliver more timely and competitive products. However, in safety-critical systems such as medical devices, autonomous vehicles, or factory floor robots, the release of new features has the potential to introduce hazards that potentially lead to run-time failures that impact software safety. As a result, many projects suffer from a phenomenon referred to as the big freeze. SAFA is designed to address this challenge. Through the use of cutting-edge deep-learning solutions, it generates trees of requirements, designs, code, tests, and other artifacts that visually depict how hazards are mitigated in the system, and it automatically warns the user when key artifacts are missing. It also uses a combination of colors, annotations, and recommendations to dynamically visualize change across software versions and augments safety cases with visual annotations to aid users in detecting and analyzing potentially adverse impacts of change upon system safety. A link to our tool demo can be found at https://www.youtube.com/watch?v=r-CwxerbSVA.

CyberPhysical systems (CPS) must be closely monitored to identify and potentially mitigate emergent problems that arise during their routine operations. However, the multivariate time-series data which they typically produce can be complex to understand and analyze. While formal product documentation often provides example data plots with diagnostic suggestions, the sheer diversity of attributes, critical thresholds, and data interactions can be overwhelming to non-experts who subsequently seek help from discussion forums to interpret their data logs. Deep learning models, such as Long Short-term memory (LSTM) networks can be used to automate these tasks and to provide clear explanations of diverse anomalies detected in real-time multivariate data-streams. In this paper we present RESAM, a requirements process that integrates knowledge from domain experts, discussion forums, and formal product documentation, to discover and specify requirements and design definitions in the form of time-series attributes that contribute to the construction of effective deep learning anomaly detectors. We present a case-study based on a flight control system for small Uncrewed Aerial Systems and demonstrate that its use guides the construction of effective anomaly detection models whilst also providing underlying support for explainability. RESAM is relevant to domains in which open or closed online forums provide discussion support for log analysis.

The MAPE-K feedback loop has been established as the primary reference model for self-adaptive and autonomous systems in domains such as autonomous driving, robotics, and Cyber-Physical Systems. At the same time, the Human Machine Teaming (HMT) paradigm is designed to promote partnerships between humans and autonomous machines. It goes far beyond the degree of collaboration expected in human-on-the-loop and human-in-the-loop systems and emphasizes interactions, partnership, and teamwork between humans and machines. However, while MAPE-K enables fully autonomous behavior, it does not explicitly address the interactions between humans and machines as intended by HMT. In this paper, we present the MAPE-KHMT framework which augments the traditional MAPE-K loop with support for HMT. We identify critical human-machine teaming factors and describe the infrastructure needed across the various phases of the MAPE-K loop in order to effectively support HMT. This includes runtime models that are constructed and populated dynamically across monitoring, analysis, planning, and execution phases to support human-machine partnerships. We illustrate MAPE-KHMT using examples from an autonomous multi-UAV emergency response system, and present guidelines for integrating HMT into MAPE-K.

Rapid advancements in Artificial Intelligence have shifted the focus from traditional human-directed robots to fully autonomous ones that do not require explicit human control. These are commonly referred to as Human-on-the-Loop (HotL) systems. Transparency of HotL systems necessitates clear explanations of autonomous behavior so that humans are aware of what is happening in the environment and can understand why robots behave in a certain way. However, in complex multi-robot environments, especially those in which the robots are autonomous and mobile, humans may struggle to maintain situational awareness. Presenting humans with rich explanations of autonomous behavior tends to overload them with lots of information and negatively affect their understanding of the situation. Therefore, explaining the autonomous behavior of multiple robots creates a design tension that demands careful investigation. This paper examines the User Interface (UI) design trade-offs associated with providing timely and detailed explanations of autonomous behavior for swarms of small Unmanned Aerial Systems (sUAS) or drones. We analyze the impact of UI design choices on human awareness of the situation. We conducted multiple user studies with both inexperienced and expert sUAS operators to present our design solution and initial guidelines for designing the HotL multi-sUAS interface.

Software traceability establishes and leverages associations between diverse development artifacts. Researchers have proposed the use of deep learning trace models to link natural language artifacts, such as requirements and issue descriptions, to source code; however, their effectiveness has been restricted by availability of labeled data and efficiency at runtime. In this study, we propose a novel framework called Trace BERT (T-BERT) to generate trace links between source code and natural language artifacts. To address data sparsity, we leverage a three-step training strategy to enable trace models to transfer knowledge from a closely related Software Engineering challenge, which has a rich dataset, to produce trace links with much higher accuracy than has previously been achieved. We then apply the T-BERT framework to recover links between issues and commits in Open Source Projects. We comparatively evaluated accuracy and efficiency of three BERT architectures. Results show that a Single-BERT architecture generated the most accurate links, while a Siamese-BERT architecture produced comparable results with significantly less execution time. Furthermore, by learning and transferring knowledge, all three models in the framework outperform classical IR trace models. On the three evaluated real-word OSS projects, the best T-BERT stably outperformed the VSM model with average improvements of 60.31% measured using Mean Average Precision (MAP). RNN severely underperformed on these projects due to insufficient training data, while T-BERT overcame this problem by using pretrained language models and transfer learning.

Regulations, standards, and guidelines for safety-critical systems stipulate stringent traceability but do not prescribe the corresponding, detailed software engineering process. Given the industrial practice of using only semi-formal notations to describe engineering processes, processes are rarely "executable" and developers have to spend significant manual effort in ensuring that they follow the steps mandated by quality assurance. The size and complexity of systems and regulations makes manual, timely feedback from Quality Assurance (QA) engineers infeasible. In this paper we propose a novel framework for tracking processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to two different case studies; one open source community system and a safety-critical system in the air-traffic control domain. Results from the analysis show that trace links are often corrected or completed after the fact and thus timely and automated constraint checking support has significant potential on reducing rework.

Software traceability establishes a network of connections between diverse artifacts such as requirements, design, and code. However, given the cost and effort of creating and maintaining trace links manually, researchers have proposed automated approaches using information retrieval techniques. Current approaches focus almost entirely upon generating links between pairs of artifacts and have not leveraged the broader network of interconnected artifacts. In this paper we investigate the use of intermediate artifacts to enhance the accuracy of the generated trace links - focusing on paths consisting of source, target, and intermediate artifacts. We propose and evaluate combinations of techniques for computing semantic similarity, scaling scores across multiple paths, and aggregating results from multiple paths. We report results from five projects, including one large industrial project. We find that leveraging intermediate artifacts improves the accuracy of end-to-end trace retrieval across all datasets and accuracy metrics. After further analysis, we discover that leveraging intermediate artifacts is only helpful when a project's artifacts share a common vocabulary, which tends to occur in refinement and decomposition hierarchies of artifacts. Given our hybrid approach that integrates both direct and transitive links, we observed little to no loss of accuracy when intermediate artifacts lacked a shared vocabulary with source or target artifacts.

Automatic construction of a taxonomy supports many applications in e-commerce, web search, and question answering. Existing taxonomy expansion or completion methods assume that new concepts have been accurately extracted and their embedding vectors learned from the text corpus. However, one critical and fundamental challenge in fixing the incompleteness of taxonomies is the incompleteness of the extracted concepts, especially for those whose names have multiple words and consequently low frequency in the corpus. To resolve the limitations of extraction-based methods, we propose GenTaxo to enhance taxonomy completion by identifying positions in existing taxonomies that need new concepts and then generating appropriate concept names. Instead of relying on the corpus for concept embeddings, GenTaxo learns the contextual embeddings from their surrounding graph-based and language-based relational information, and leverages the corpus for pre-training a concept name generator. Experimental results demonstrate that GenTaxo improves the completeness of taxonomies over existing methods.

With the rise of new AI technologies, autonomous systems are moving towards a paradigm in which increasing levels of responsibility are shifted from the human to the system, creating a transition from human-in-the-loop systems to human-on-the-loop (HoTL) systems. This has a significant impact on the safety analysis of such systems, as new types of errors occurring at the boundaries of human-machine interactions need to be taken into consideration. Traditional safety analysis typically focuses on system-level hazards with little focus on user-related or user-induced hazards that can cause critical system failures. To address this issue, we construct domain-level safety analysis assets for sUAS (small unmanned aerial systems) applications and describe the process we followed to explicitly, and systematically identify Human Interaction Points (HiPs), Hazard Factors and Mitigations from system hazards. We evaluate our approach by first investigating the extent to which recent sUAS incidents are covered by our hazard trees, and second by performing a study with six domain experts using our hazard trees to identify and document hazards for sUAS usage scenarios. Our study showed that our hazard trees provided effective coverage for a wide variety of sUAS application scenarios and were useful for stimulating safety thinking and helping users to identify and potentially mitigate human-interaction hazards.

The growing adoption of unmanned aerial vehicles (UAVs) for tasks such as eCommerce, aerial surveillance, and environmental monitoring introduces the need for new safety mechanisms in an increasingly cluttered airspace. In our work we thus emphasize safety issues that emerge at the intersection of infrastructures responsible for controlling the airspace, and the diverse UAVs operating in their space. We build on safety assurance cases (SAC) - a state-of-the-art solution for reasoning about safety - and propose a novel approach based on interlocking SACs. The infrastructure safety case (ISAC) specifies assumptions upon UAV behavior, while each UAV demonstrates compliance to the ISAC by presenting its own (pluggable) safety case (pSAC) which connects to the ISAC through a set of interlock points. To collect information on each UAV we enforce a “trust but monitor” policy, supported by runtime monitoring and an underlying reputation model. We evaluate our approach in three ways: first by developing ISACs for two UAV infrastructures, second by running simulations to evaluate end-to-end effectiveness, and finally via an outdoor field-study with physical UAVs. The results show that interlocking SACs can be effective for identifying, specifying, and monitoring safety-related constraints upon UAVs flying in a controlled airspace.

The use of semi-autonomous Unmanned Aerial Vehicles (UAV) to support emergency response scenarios, such as fire surveillance and search and rescue, offers the potential for huge societal benefits. However, designing an effective solution in this complex domain represents a "wicked design" problem, requiring a careful balance between trade-offs associated with drone autonomy versus human control, mission functionality versus safety, and the diverse needs of different stakeholders. This paper focuses on designing for situational awareness (SA) using a scenario-driven, participatory design process. We developed SA cards describing six common design-problems, known as SA demons, and three new demons of importance to our domain. We then used these SA cards to equip domain experts with SA knowledge so that they could more fully engage in the design process. We designed a potentially reusable solution for achieving SA in multi-stakeholder, multi-UAV, emergency response applications.

Software traceability provides support for various engineering activities including Program Comprehension; however, it can be challenging and arduous to complete in large industrial projects. Researchers have proposed automated traceability techniques to create, maintain and leverage trace links. Computationally intensive techniques, such as repository mining and deep learning, have showed the capability to deliver accurate trace links. The objective of achieving trusted, automated tracing techniques at industrial scale has not yet been successfully accomplished due to practical performance challenges. This paper evaluates high-performance solutions for deploying effective, computationally expensive trace-ability algorithms in large scale industrial projects and leverages generated trace links to answer Program Comprehension Queries. We comparatively evaluate four different platforms for supporting industrial-scale tracing solutions, capable of tackling software projects with millions of artifacts. We demonstrate that tracing solutions built using big data frameworks scale well for large projects and that our Spark implementation outperforms relational database, graph database (GraphDB), and plain Java implementations. These findings contradict earlier results which suggested that GraphDB solutions should be adopted for large-scale tracing problems.

Software traceability establishes associations between diverse software artifacts such as requirements, design, code, and test cases. Due to the non-trivial costs of manually creating and maintaining links, many researchers have proposed automated approaches based on information retrieval techniques. However, many globally distributed software projects produce software artifacts written in two or more languages. The use of intermingled languages reduces the efficacy of automated tracing solutions. In this paper, we first analyze and discuss patterns of intermingled language use across multiple projects, and then evaluate several different tracing algorithms including the Vector Space Model (VSM), Latent Semantic Indexing (LSI), Latent Dirichlet Allocation (LDA), and various models that combine mono-and cross-lingual word embeddings with the Generative Vector Space Model (GVSM). Based on an analysis of 14 Chinese-English projects, our results show that best performance is achieved using mono-lingual word embeddings integrated into GVSM with machine translation as a preprocessing step.

In many regulated domains, traceability is established across diverse artifacts such as requirements, design, code, test cases, and hazards - either manually or with the help of supporting tools, and the resulting trace links are used to support activities such as impact analysis, compliance verification, and safety inspections. Automated tracing techniques need to leverage the semantics of underlying artifacts in order to establish more accurate trace links and to provide explanations of links that have been created in either a manual or automated fashion. To support this, we propose an automated technique which leverages source code, project artifacts and an external domain corpus to generate a domain-specific concept model. We then use the generated concept model to improve traceability results and to provide explanations of the results. Our approach overcomes existing problems with deep-learning traceability algorithms, as it does not require a training set of existing trace links. Finally, as an initial proof-of-concept, we apply our semantically-guided approach to the Dronology project, and show that it improves over other tracing techniques that do not use a concept model.

We investigate the extent to which code commit summaries provide rationales and descriptions of code refactorings. We present a refactoring description detection tool CMMiner that detects code commit messages containing refactoring information and differentiates between twelve different refactoring types. We further explore whether refactoring information mined from commit messages using CMMiner, can be combined with refactoring descriptions mined from source code using the well-known RMiner tool. For six refactoring types covered by both CMMiner and RMiner, we observed 21.96% to 38.59% overlap in refactorings detected across four diverse open-source systems. RMiner identified approximately 49.13% to 60.29% of refactorings missed by CMMiner, primarily because developers often failed to describe code refactorings that occurred alongside other code changes. However, CMMiner identified 10.30% to 19.51% of refactorings missed by RMiner, primarily when refactorings occurred across multiple commits. Our results suggest that integrating both approaches can enhance the completeness of refactoring detection and provide refactoring rationales.

As new requirements are introduced and implemented in a software system, developers must identify the set of source code classes which need to be changed. Therefore, past effort has focused on predicting the set of classes impacted by a requirement. In this paper, we introduce and evaluate a new type of information based on the intuition that the set of requirements which are associated with historical changes to a specific class are likely to exhibit semantic similarity to new requirements which impact that class. This new Requirements to Requirements Set (R2RS) family of metrics captures the semantic similarity between a new requirement and the set of existing requirements previously associated with a class. The aim of this paper is to present and evaluate the usefulness of R2RS metrics in predicting the set of classes impacted by a requirement. We consider 18 different R2RS metrics by combining six natural language processing techniques to measure the semantic similarity among texts (e.g., VSM) and three distribution scores to compute overall similarity (e.g., average among similarity scores). We evaluate if R2RS is useful for predicting impacted classes in combination and against four other families of metrics that are based upon temporal locality of changes, direct similarity to code, complexity metrics, and code smells. Our evaluation features five classifiers and 78 releases belonging to four large open-source projects, which result in over 700,000 candidate impacted classes. Experimental results show that leveraging R2RS information increases the accuracy of predicting impacted classes practically by an average of more than 60 percent across the various classifiers and projects.

Safety Assurance Cases (SACs) are increasingly used to guide and evaluate the safety of software-intensive systems. They are used to construct a hierarchically organized set of claims, arguments, and evidence in order to provide a structured argument that a system is safe for use. However, as the system evolves and grows in size, a SAC can be difficult to maintain. In this paper we utilize design science to develop a novel solution for identifying areas of a SAC that are affected by changes to the system. Moreover, we generate actionable recommendations for updating the SAC, including its underlying artifacts and trace links, in order to evolve an existing safety case for use in a new version of the system. Our approach, Safety Artifact Forest Analysis (SAFA), leverages traceability to automatically compare software artifacts from a previously approved or certified version with a new version of the system. We identify, visualize, and explain changes in a Delta Tree. We evaluate our approach using the Dronology system for monitoring and coordinating the actions of cooperating, small Unmanned Aerial Vehicles. Results from a user study show that SAFA helped users to identify changes that potentially impacted system safety and provided information that could be used to help maintain and evolve a SAC.

Unmanned aerial vehicles (UAVs) are becoming increasingly pervasive in everyday life, supporting diverse use cases such as aerial photography, delivery of goods, or disaster reconnaissance and management. UAVs are cyber-physical systems (CPS): they integrate computation (embedded software and control systems) with physical components (the UAVs flying in the physical world). UAVs in particular and CPS in general require monitoring capabilities to detect and possibly mitigate erroneous and safety-critical behavior at runtime. Existing monitoring approaches mostly do not adequately address UAV CPS characteristics such as the high number of dynamically instantiated components, the tight int elements, and the massive amounts of data that need to be processed. In this paper we report results of a case study on monitoring in UAVs. We discuss CPS-specific monitoring challenges and present a prototype we implemented by extending \reminds, a framework for software monitoring so far mainly used in the domain of metallurgical plants. Additionally, we demonstrate the applicability and scalability of our approach by monitoring a real control and management system for UAVs in simulations with up to 30 drones flying in an urban area.

Software and systems traceability is widely accepted as an essential element for supporting many software development tasks. Today's version control systems provide inbuilt features that allow developers to tag each commit with one or more issue ID, thereby providing the building blocks from which project-wide traceability can be established between feature requests, bug fixes, commits, source code, and specific developers. However, our analysis of six open source projects showed that on average only 60% of the commits were linked to specific issues. Without these fundamental links the entire set of project-wide links will be incomplete, and therefore not trustworthy. In this paper we address the fundamental problem of missing links between commits and issues. Our approach leverages a combination of process and text-related features characterizing issues and code changes to train a classifier to identify missing issue tags in commit messages, thereby generating the missing links. We conducted a series of experiments to evaluate our approach against six open source projects and showed that it was able to effectively recommend links for tagging issues at an average of 96% recall and 33% precision. In a related task for augmenting a set of existing trace links, the classifier returned precision at levels greater than 89% in all projects and recall of 50%.

Requirements Engineering includes various activities aimed at discovering, analyzing, validating, evolving, and managing software and systems requirements. Many of these activities are human facing, effort intensive, and sometimes error prone. They could benefit greatly from cutting edge advances in automation. However, the software engineering community has primarily focused on automating other aspects of the development process such as testing, code analytics, and mining software respositories. As a result, advances in software analytics have had superficial impact upon advancing the state of art and practice in the field of requirements engineering. Two primary inhibitors are the lack of publicly available datasets and poorly publicized industry-relevant open requirements analytic challenges. To empower the Automated Software Engineering community to tackle open Requirements Engineering challenges, the talk will describe the rapidly evolving landscape of requirements engineering, clearly articulate open challenges, draw upon examples from an ongoing, agile, safety-critical project in the domain of Unmanned Aerial Vehicles, and introduce Dronology as a new community dataset.

This keynote addresses the challenges and opportunities introduced by disruptive change in the current requirements engineering landscape. Sea changes in the way practitioners develop software, along with advances in artificial intelligence algorithms and the ubiquity of social media environments have created a goldilocks opportunity for innovative creativity that potentially touches every aspect of requirements engineering research. Coupled with passion and vision, these advances revitalize our ability to address open requirements challenges in new and meaningful ways.

Traditionally, safety-critical projects have been developed using the waterfall process. However, this makes it costly and challenging to incrementally introduce new features and to certify the modified product for use. As a result, there has been increasing interest in adopting agile development paradigms within the safety-critical domain. This in turn introduces numerous challenges. In this paper we address the specific problems of discovering, analyzing, specifying, and managing safety requirements within the agile Scrum process. We propose SafetyScrum, a methodology that augments the Scrum lifecycle with incrementally applied safety-related activities and introduces the notion of "safety debt" for incrementally tracking the current safety status of a project. We demonstrate the viability of SafetyScrum for managing safety stories in an agile development environment by applying it to a project in which our existing Unmanned Aerial Vehicle system is enhanced to support a River-Rescue scenario.

Automated traceability has been investigated for over a decade with promising results. However, a human analyst is needed to vet the generated trace links to ensure their quality. The process of vetting trace links is not trivial and while previous studies have analyzed the performance of the human analyst, they have not focused on the analyst's information needs. The aim of this study is to investigate what context information the human analyst needs. We used design science research, in which we conducted interviews with ten practitioners in the traceability area to understand the information needed by human analysts. We then compared the information collected from the interviews with existing literature. We created a prototype tool that presents this information to the human analyst. To further understand the role of context information, we conducted a controlled experiment with 33 participants. Our interviews reveal that human analysts need information from three different sources: 1) from the artifacts connected by the link, 2) from the traceability information model, and 3) from the tracing algorithm. The experiment results show that the content of the connected artifacts is more useful to the analyst than the contextual information of the artifacts.

Industrial software systems are often systems of systems (SoS) whose full behavior only emerges during operation. They therefore require monitoring techniques to observe systems and detect deviations from their requirements. The focus of existing monitoring approaches, however, is mainly on detecting violations of expected behavior, while support for diagnosing violations is typically limited or even neglected. Diagnosis is particularly challenging in SoS due to their technological heterogeneity and the diversity of development tools in use. Uncovering the root cause of a violation typically requires developers to trace violations to artifacts such as source code or requirements documents, which is difficult without detailed domain knowledge. In this paper we describe our experiences of developing a tool-supported approach facilitating the diagnosis of requirements violations in SoS. We describe how we complemented a requirements monitoring model with a system artifact model relating SoS artifacts needed for diagnosis with monitored events. We customized our approach to an industrial SoS and conducted a scenario-based walkthrough with engineers developing the SoS and engineers and researchers unfamiliar with it. The results of our evaluation have shown that our approach can significantly ease diagnosing violations in a real-world SoS.

In most safety-critical domains the need for traceability is prescribed by certifying bodies. Trace links are generally created among requirements, design, source code, test cases and other artifacts, however, creating such links manually is time consuming and error prone. Automated solutions use information retrieval and machine learning techniques to generate trace links, however, current techniques fail to understand semantics of the software artifacts or to integrate domain knowledge into the tracing process and therefore tend to deliver imprecise and inaccurate results. In this paper, we present a solution that uses deep learning to incorporate requirements artifact semantics and domain knowledge into the tracing solution. We propose a tracing network architecture that utilizes Word Embedding and Recurrent Neural Network (RNN) models to generate trace links. Word embedding learns word vectors that represent knowledge of the domain corpus and RNN uses these word vectors to learn the sentence semantics of requirements artifacts. We trained 360 different configurations of the tracing network using existing trace links in the Positive Train Control domain and identified the Bidirectional Gated Recurrent Unit (BI-GRU) as the best model for the tracing task. BI-GRU significantly out-performed state-of-the-art tracing methods including the Vector Space Model and Latent Semantic Indexing.

Software projects produce large quantities of data such as feature requests, requirements, design artifacts, source code, tests, safety cases, release plans, and bug reports. If leveraged effectively, this data can be used to provide project intelligence that supports diverse software engineering activities such as release planning, impact analysis, and software analytics. However, project stakeholders often lack skills to formulate complex queries needed to retrieve, manipulate, and display the data in meaningful ways. To address these challenges we introduce TiQi, a natural language interface, which allows users to express software-related queries verbally or written in natural language. TiQi is a web-based tool. It visualizes available project data as a prompt to the user, accepts Natural Language (NL) queries, transforms those queries into SQL, and then executes the queries against a centralized or distributed database. Raw data is stored either directly in the database or retrieved dynamically at runtime from case tools and repositories such as Github and Jira. The transformed query is visualized back to the user as SQL and augmented UML, and raw data results are returned. Our tool demo can be found on YouTube at the following link:http://tinyurl.com/TIQIDemo.

Problems with the correctness and completeness of environmental assumptions contribute to many accidents in safety-critical systems. The problem is exacerbated when products are modified in new releases or in new products of a product line. In such cases existing sets of environmental assumptions are often carried forward without sufficiently rigorous analysis. This paper describes a new technique that exploits the traceability required by many certifying bodies to reason about the likelihood that environmental assumptions are omitted or incorrectly retained in new products. An analysis of over 150 examples of environmental assumptions in historical systems informs the approach. In an evaluation on three safety-related product lines the approach caught all but one of the assumption-related problems. It also provided clearly defined steps for mitigating the identified issues. The contribution of the work is to arm the safety analyst with useful information for assessing the validity of environmental assumptions for a new product.

Context and Motivation Natural language processing has been used since the 1980s to construct tools for performing natural language (NL) requirements engineering (RE) tasks. The RE field has often adopted information retrieval (IR) algorithms for use in implementing these NL RE tools. Problem Traditionally, the methods for evaluating an NL RE tool have been inherited from the IR field without adapting them to the requirements of the RE context in which the NL RE tool is used. Principal Ideas This panel discusses the problem and considers the evaluation of tools for a number of NL RE tasks in a number of contexts. Contribution The discussion is aimed at helping the RE field begin to consistently evaluate each of its tools according to the requirements of the tool's task.

Developers maintaining safety-critical systems need to assess the impact a proposed change would have upon existing safety controls. By leveraging the network of traceability links that are present in most safety-critical systems, we can push timely information about related hazards, environmental assumptions, and safety requirements to developers. In this work we take a design science approach to discover the informational needs of developers as they engage in software maintenance activities and then propose and evaluate techniques for presenting and visualizing this information. Through a human-centered study involving five safety-critical system practitioners and 14 experienced developers, we analyze the way in which developers use requirements knowledge while maintaining safety-critical code, identify their informational needs, and propose and evaluate a supporting visualization technique. The insights proposed as a result of this study can be used to design requirements-based knowledge tools for supporting developers' maintenance tasks.

The cost and effort of developing software systems in a new technical area can be extensive. An organization must perform a domain analysis to discover competing products, analyze their architectures and features, and ultimately discover and specify product requirements. However, delivering high quality products, depends not only on gaining an understanding of functional requirements, but also of qualities such as performance, reliability, security, and usability. Discovering such concerns early in the requirements process drives architectural design decisions. This paper extends our prior work on mining functional requirements from large collections of domain documents, by proposing and evaluating a new technique for discovering and specifying quality concerns related to specific functional components. We evaluate our approach against three domains of Positive Train Control, Electronic Health Records, and Medical Infusion Pumps, and show that it significantly outperforms a basic information retrieval approach. Finally we classified the forms of retrieved information, discussed the utility of different types, and conducted a small study with an experienced engineer to investigate the quality of requirements produced using our approach.

Requirements Engineering (RE) is comprised of various tasks related to discovering, documenting, and maintaining different kinds of requirements. To accomplish these tasks, a Requirements Engineer or Business Analyst needs to retrieve and combine information from multiple sources such as use case models, interview scripts, and business rules. However, collecting and analyzing all the required data can be tedious and the resulting data is often incomplete with inadequate trace links. Analyzing real-world queries can shed light on the questions requirements professionals would like to ask and the artifacts needed to support such questions. We therefore conducted an online survey with requirements professionals in the IT industry. Our analysis included 29 survey responses and a total of 159 natural language queries. Using open coding and grounded theory, we analyzed and grouped these queries into 9 different query purposes and 54 sub-purposes, and also identified frequently used artifacts. The results from the survey could help project-level planners identify important questions, proactively instrument their environments with supporting tools, and strategically collect data that is needed to answer the queries of interest to their project.

Security threats should be identified in the early phases of a project so that design solutions can be explored and mitigating requirements specified. In this paper, we present a crowd-sourcing approach for creating Personae non Gratae (PnGs), which model attack goals and techniques of unwanted, potentially malicious users. We present a proof of concept study that takes a diverse collection of potentially redundant PnGs and merges them into a single set. Our approach combines machine learning techniques and visualization. It is illustrated and evaluated using a collection of PnGs collected from undergraduate students for a drone-based rescue scenario. Lessons learned from the proof of concept study are discussed and lay the foundations for future work.

Software requirements specifications (SRSs) often lack the detail needed to make informed architectural decisions. Architects therefore either make assumptions, which can lead to incorrect decisions, or conduct additional stakeholder interviews, resulting in potential project delays. We previously observed that software architects ask Probing Questions (PQs) to gather information crucial to architectural decision-making. Our goal is to equip Business Analysts with appropriate PQs so that they can ask these questions themselves. We report a new study with over 40 experienced architects to identify reusable PQs for five areas of functionality and organize them into structured flows. These PQ-flows can be used by Business Analysts to elicit and specify architecturally relevant information. Additionally, we leverage machine learning techniques to determine when a PQ-flow is appropriate for use in a project, and to annotate individual PQs with relevant information extracted from the existing SRS. We trained and evaluated our approach on over 8,000 individual requirements from 114 requirements specifications and also conducted a pilot study to validate its usefulness.

To facilitate research into trace link evolution we present 27 versions of Cassandra source code, feature descriptions for each version, deltas between versions, structured descriptions of each version, and trace links between a subset of 48 features and source code for the starting and ending versions.

Trace links provide critical support for numerous software engineering activities including safety analysis, compliance verification, test-case selection, and impact prediction. However, as the system evolves over time, there is a tendency for the quality of trace links to degrade into a tangle of inaccurate and untrusted links. This is especially true with the links between source-code and upstream artifacts such as requirements - because developers frequently refactor and change code without updating the links. We present TLE (Trace Link Evolver), a solution for automating the evolution of trace links as changes are introduced to source code. We use a set of heuristics, open source tools, and information retrieval methods to detect common change scenarios across different versions of software. Each change scenario is then associated with a set of link evolution heuristics which are used to evolve trace links. We evaluate our approach through a controlled experiment and also through applying it across 27 releases of the Cassandra Database System. Results show that the trace links evolved using our approach are significantly more accurate than those generated using information retrieval alone.

Software project artifacts such as source code, requirements, and change logs represent a gold-mine of actionable information. As a result, software analytic solutions have been developed to mine repositories and answer questions such as "who is the expert?," "which classes are fault prone?," or even "who are the domain experts for these fault-prone classes?" Analytics often require training and configuring in order to maximize performance within the context of each project. A cold-start problem exists when a function is applied within a project context without first configuring the analytic functions on project-specific data. This scenario exists because of the non-trivial effort necessary to instrument a project environment with candidate tools and algorithms and to empirically evaluate alternate configurations. We address the cold-start problem by comparatively evaluating 'best-of-breed' and 'profile-driven' solutions, both of which reuse known configurations in new project contexts. We describe and evaluate our approach against 20 project datasets for the three analytic areas of artifact connectivity, fault-prediction, and finding the expert, and show that the best-of-breed approach outperformed the profile-driven approach in all three areas; however, while it delivered acceptable results for artifact connectivity and find the expert, both techniques underperformed for cold-start fault prediction.

When organizations enter domains that are entirely new to them, they need to invest significant time and effort to acquire domain knowledge. This typically involves searching through a broad set of domain documents, retrieving relevant ones, and analyzing the textual content in order to discover and specify pertinent requirements. Depending on the nature of the domain and the availability of documentation, this task can be extremely time-consuming and may require non-trivial human effort. Furthermore, the task must often be performed repeatedly throughout early phases of the project. In this paper we first explore the effort needed to manually build a high-level domain model capturing the functional components. We then present MaRK (Mining Requirements Knowledge), which identifies and retrieves the documents containing descriptions of functional components in the domain model. Domain analysts can use this information to to specify requirements. We introduce and evaluate an algorithm which ranks domain documents according to their relevance to a component and then highlights sections of text which are likely to contain requirements-related information. We describe our process within the context of the Positive Train Control (PTC) domain with a repository of of 523 documents, representing 852MB of data. We empirically evaluate the MaRK relevance algorithm and its ability to retrieve relevant requirements knowledge for requirements related to PTC's On-Board Unit.

Software architectures are often constructed through a series of design decisions. In particular, architectural tactics are selected to satisfy specific quality concerns such as reliability, performance, and security. However, the knowledge of these tactical decisions is often lost, resulting in a gradual degradation of architectural quality as developers modify the code without fully understanding the underlying architectural decisions. In this paper we present a machine learning approach for discovering and visualizing architectural tactics in code, mapping these code segments to tactic traceability patterns, and monitoring sensitive areas of the code for modification events in order to provide users with up-to-date information about underlying architectural concerns. Our approach utilizes a customized classifier which is trained using code extracted from fifty performance-centric and safety-critical open source software systems. Its performance is compared against seven off-the-shelf classifiers. In a controlled experiment all classifiers performed well; however our tactic detector outperformed the other classifiers when used within the larger context of the Hadoop Distributed File System. We further demonstrate the viability of our approach for using the automatically detected tactics to generate viable and informative messages in a simulation of maintenance events mined from Hadoop's change management system.

Architectural qualities such as reliability, performance, and security, are often realized in a software system through the adoption of tactical design decisions such as the decision to use redundant processes, a heartbeat monitor, or a specific authentication mechanism. Such decisions are critical for delivering a system that meets its quality requirements. Despite the stability of high-level decisions, our analysis has shown that tactic-related classes tend to be modified more frequently than other classes and are therefore stronger predictors of change than traditional Object-Oriented coupling and cohesion metrics. In this paper we present the results from this initial study, including an analysis of why tactic-related classes are changed, and a discussion of the implications of these findings for maintaining architectural quality over the lifetime of a software system.

Software Traceability is a critical element in all safety critical software systems. Trace links are created across diverse artifacts such as requirements, design, code, test cases, and hazards - either manually or with the help of supporting tools. The links are then used to support a range of software engineering activities including impact analysis, compliance verification, and safety inspections. For traceability to effectively support these activities it is important for the meaning and rationale of each link to be clearly communicated. It is often insuficient to know that one artifact satisfies, realizes, or complies to another. Instead, it is important to know why and how it does so. Terms and phrases used to describe artifacts are connected through composition, synonymic, and generalization relationships which often can only be interpreted by domain experts. In this RE:Next! paper we propose a novel approach for utilizing domain-specific knowledge bases to generate trace link rationales. We illustrate our approach with examples of automatically generated rationales taken from the domain of Communication and Control of a Transportation system, and from a Medical Infusion pump domain.

Software architects are responsible for designing an architectural solution that satisfies the functional and non-functional requirements of the system to the fullest extent possible. However, the details they need to make informed architectural decisions are often missing from the requirements specification. An earlier study we conducted indicated that architects intuitively recognize architecturally significant requirements in a project, and often seek out relevant stakeholders in order to ask Probing Questions (PQs) that help them acquire the information they need. This paper presents results from a qualitative interview study aimed at identifying architecturally significant functional requirements' categories from various business domains, exploring relevant PQs for each category, and then grouping PQs by type. Using interview data from 14 software architects in three countries, we identified 15 categories of architecturally significant functional requirements and 6 types of PQs. We found that the domain knowledge of the architect and her experience influence the choice of PQs significantly. A preliminary quantitative evaluation of the results against real-life software requirements specification documents indicated that software specifications in our sample largely do not contain the crucial architectural differentiators that may impact architectural choices and that PQs are a necessary mechanism to unearth them. Further, our findings provide the initial list of PQs which could be used to prompt business analysts to elicit architecturally significant functional requirements that the architects need.

Research projects tend to evolve through multiple phases of incubation and experimentation, before maturing to levels of full industry adoption. Practice has shown that successful research solutions often take over 20 years to achieve full technology transfer. However, many projects never leave the incubation phase either because the new technique fails to perform well, or because researchers lack the knowledge, skills, or time to transition the idea to practice. A healthy research community could be expected to produce a steady stream of innovative solutions that positively impact industrial practice. To achieve these goals, we need ongoing, rigorous, and mutually beneficial conversations between academics and practitioners. Such exchanges are a desirable part of the research process, and will help the requirements engineering community to integrate technology transfer plans into the ongoing research plans. In this interactive panel, teams of researchers, representing different requirements engineering research areas, will present their research solutions to a panel of seasoned industrial practitioners. The practitioners provide insightful feedback that can help with the transition to practice. While Ready-Set-Transfer is presented as an interactive game-show, it has the serious goal of fostering collaboration and conversations between practitioners and researchers in the requirements engineering community.

Bernard of Chartres (via Sir Isaac Newton) reminded us that all progress is achieved “on the shoulders of giants” - that our greatest discoveries and innovations build upon the inspirations, triumphs, and foundational truths established by those who have gone before us. However, in our field of Software Engineering, as new ideas are transmitted at the speed of light, rather than the speed of Bernard's horse, innovations are typically achieved as we, the ordinary people, exchange ideas, deliver incremental improvements, and offer the occasional truly novel idea to advance our field. In this fast-paced environment it is particularly important for us to take the time to build a strong foundation for our knowledge - keeping audit trails of our experiments, sharing our datasets, releasing the code we used to run our experiments, and generally making our work transparent and reproducible, so that we no longer depend on giants to further the field. Instead our successes are a collective effort from our community. Unfortunately, this degree of openness comes with its own challenges. In this talk, Dr. Cleland-Huang will explore some of the success stories in our field and discuss ways to deal with the psychological, philosophical, and practical barriers that impede open collaboration.

Many guidelines for safety-critical industries such as aeronautics, medical devices, and railway communications, specify that traceability must be used to demonstrate that a rigorous process has been followed and to provide evidence that the system is safe for use. In practice, there is a gap between what is prescribed by guidelines and what is implemented in practice, making it difficult for organizations and certifiers to fully evaluate the safety of the software system. In this paper we present an approach, which parses a guideline to extract a Traceability Model depicting software artifact types and their prescribed traces. It then analyzes the traceability data within a project to identify areas of traceability failure. Missing traceability paths, redundant and/or inconsistent data, and other problems are highlighted. We used our approach to evaluate the traceability of seven safety-critical software systems and found that none of the evaluated projects contained traceability that fully conformed to its relevant guidelines.

State-of-the-art software trace retrieval techniques are unable to perform the complex reasoning that a human analyst follows in order to create accurate trace links between artifacts such as regulatory codes and requirements. As a result, current algorithms often generate imprecise links. To address this problem, we present the Domain-Contextualized Intelligent Traceability Solution (DoCIT), designed to mimic some of the higher level reasoning that a human trace analyst performs. We focus our efforts on the complex domain of communication and control in a transportation system. DoCIT includes rules for extracting ``action units'' from software artifacts, a domain-specific knowledge base for relating semantically similar concepts across action units, and a set of link-creation heuristics which utilize the action units to establish meaningful trace links across pairs of artifacts. Our approach significantly improves the quality of the generated trace links. We illustrate and evaluate DoCIT with examples and experiments from the control and communication sector of a transportation domain.

Many software systems utilize forums to allow a broad set of stakeholders to request features. However the resulting mass of ideas and comments can make prioritization and management of feature requests challenging. In this paper we propose a novel approach for partially automating the creation of personas from a set of feature requests in open forums. Our approach utilizes topic clustering, classification, and association rules to identify meaningful groupings of feature requests and then uses them to guide the construction of personas. Once created, these personas are leveraged to coordinate feature requests, track changes, and to provide stakeholder communication mechanisms. We illustrate our approach with examples taken from the health-insurance domain and then evaluate it against feature requests in the SugarCRM project.

One of the surprising observations of traceability in practice is the under-utilization of existing trace links. Organizations often create links in order to meet compliance requirements, but then fail to capitalize on the potential benefits of those links to provide support for activities such as impact analysis, test regression selection, and coverage analysis. One of the major adoption barriers is caused by the lack of accessibility to the underlying trace data and the lack of skills many project stakeholders have for formulating complex trace queries. To address these challenges we introduce TiQi, a natural language approach, which allows users to write or speak trace queries in their own words. TiQi includes a vocabulary and associated grammar learned from analyzing NL queries collected from trace practitioners. It is evaluated against trace queries gathered from trace practitioners for two different project environments.

Software requirements specifications often focus on functionality and fail to adequately capture quality concerns such as security, performance, and usability. In many projects, quality-related requirements are either entirely lacking from the specification or intermingled with functional concerns. This makes it difficult for stakeholders to fully understand the quality concerns of the system and to evaluate their scope of impact. In this paper we present a data mining approach for automating the extraction and subsequent modeling of quality concerns from requirements, feature requests, and online forums. We extend our prior work in mining quality concerns from textual documents and apply a sequence of machine learning steps to detect quality-related requirements, generate goal graphs contextualized by project-level information, and ultimately to visualize the results. We illustrate and evaluate our approach against two industrial health-care related systems.

Despite the fact that traceability is a required element of almost all safety-critical software development processes, the trace data is often incomplete, inaccurate, redundant, conflicting, and outdated. As a result, it is neither trusted nor trustworthy. In this vision paper we propose a philosophical change in the traceability landscape which transforms traceability from a heavy-weight process producing untrusted trace links, to a light-weight results-oriented trustworthy solution. Current traceability practices which retard agility are cast away and replaced with a disciplined, just-in-time approach. The novelty of our solution lies in a clear separation of trusted trace links from untrusted ones, the change in perspective from `living-with' inacurate traces toward rigorous and ongoing debridement of stale links from the trusted pool, and the notion of synthesizing available `project exhaust' as evidence to systematically construct or reconstruct purposed, highly-focused trace links.

The quality of a software architecture is largely dependent upon the underlying architectural decisions at the framework, tactic, and pattern levels. Decisions to adopt certain solutions determine the extent to which desired qualities such as security, availability, and performance are achieved in the delivered system. In this tool demo, we present our Eclipse plug-in named Archie as a solution for maintaining architectural qualities in the design and code despite long-term maintenance and evolution activities. Archie detects architectural tactics such as heartbeat, resource pooling, and role-based access control (RBAC) in the source code of a project; constructs traceability links between the tactics, design models, rationales and source code; and then uses these to monitor the environment for architecturally significant changes and to keep developers informed of underlying design decisions and their associated rationales.

The disciplines of requirements engineering (RE) and software architecture (SA) are fundamental to the success of software projects. Even though RE and SA are often considered separately, it has been argued that drawing a line between RE and SA is neither feasible nor reasonable as requirements and architectural design processes impact each other. Requirements are constrained by what is feasible technically and also by time and budget restrictions. On the other hand, feedback from the architecture leads to renegotiating architecturally significant requirements with stakeholders. The topic of bridging RE and SA has been discussed in both the RE and SA communities, but mostly independently. Therefore, the motivation for this ICSE workshop is to bring both communities together in order to identify key issues, explore the state of the art in research and practice, identify emerging trends, and define challenges related to the transition and the relationship between RE and SA.

In automated requirements traceability, significant improvements can be realized through incorporating user feedback into the trace retrieval process. However, existing feedback techniques are designed to improve results for individual queries. In this paper we present a novel technique designed to extend the benefits of user feedback across multiple trace queries. Our approach, named Trace Query Transformation (TQT), utilizes a novel form of Association Rule Mining to learn a set of query transformation rules which are used to improve the efficacy of future trace queries. We evaluate TQT using two different kinds of training sets. The first represents an initial set of queries directly modified by human analysts, while the second represents a set of queries generated by applying a query optimization process based on initial relevance feedback for trace links between a set of source and target documents. Both techniques are evaluated using requirements from theWorldVista Healthcare system, traced against certification requirements for the Commission for Healthcare Information Technology. Results show that the TQT technique returns significant improvements in the quality of generated trace links.

Attempts to utilize information retrieval techniques to fully automate the creation of traceability links have been hindered by terminology mismatches between source and target artifacts. Therefore, current trace retrieval algorithms tend to produce imprecise and incomplete results. In this paper we address this mismatch by proposing an expert system which integrates a knowledge base of domain concepts and their relationships, a set of logic rules for defining relationships between artifacts based on these rules, and a process for mapping artifacts into a structure against which the rules can be applied. This paper lays down the core foundations needed to integrate an expert system into the automated tracing process. We construct a knowledge base and inference rules for part of a large industrial project in the transportation domain and empirically show that our approach significantly improves precision and recall of the generated trace links.

As Requirements Engineering research continues to grow into a mature and rigorous discipline, an increasing focus is placed on the need for sound evaluation techniques that compare the benefits of a new solution against existing ones. In this tool demonstration we introduce TraceLab, an instrumented environment for modeling, executing, and comparatively evaluating experimental results. While initially developed for the Software Traceability domain, TraceLab provides a framework which can be populated with experiments, datasets, and reusable components for almost any empirical software engineering domain. In this demo we present examples from the Requirements Engineering domain.

Requirements engineering research is undertaken to propose innovative solutions, to develop concepts, algorithms, processes, and technologies, to validate effective solutions for important requirements-related problems, and ultimately to support the transition of important findings to practice. However prior studies have shown that successful projects often take from 20-25 years to reach the stage of full industry adoption, while many other projects fizzle out and never advance beyond the initial research phase. This panel provides the opportunity for practitioners and academics to engage in a meaningful discussion around the topic of technology transfer. In this third offering of the Ready-Set-Transfer panel, three research groups will present products that they believe to be industry-ready to a panel of industrial practitioners. Each team will receive feedback from the panelists. The long-term goal of the panel is to increase technology transfer in the requirements engineering domain.

Feature Models (FMs) are used extensively in software product line engineering to help generate and validate individual product configurations and to provide support for domain analysis. As FM construction can be tedious and time-consuming, researchers have previously developed techniques for extracting FMs from sets of formally specified individual configurations, or from software requirements specifications for families of existing products. However, such artifacts are often not available. In this paper we present a novel, automated approach for constructing FMs from publicly available product descriptions found in online product repositories and marketing websites such as SoftPedia and CNET. While each individual product description provides only a partial view of features in the domain, a large set of descriptions can provide fairly comprehensive coverage. Our approach utilizes hundreds of partial product descriptions to construct an FM and is described and evaluated against antivirus product descriptions mined from SoftPedia.

This panel will discuss what characterizes the publication process in the software engineering community and debate how it serves the needs of the community, whether it is fair - e.g. valuable work gets published and mediocre work rejected - and highlight the obstacles for young scientists. The panel will conclude with a discussion on suggested next steps.

Software traceability is a sought-after, yet often elusive quality in large software-intensive systems primarily because the cost and effort of tracing can be overwhelming. State-of-the art solutions address this problem through utilizing trace retrieval techniques to automate the process of creating and maintaining trace links. However, there is no simple one- size-fits all solution to trace retrieval. As this paper will show, finding the right combination of tracing techniques can lead to significant improvements in the quality of generated links. We present a novel approach to trace retrieval in which the underlying infrastructure is configured at runtime to optimize trace quality. We utilize a machine-learning approach to search for the best configuration given an initial training set of validated trace links, a set of available tracing techniques specified in a feature model, and an architecture capable of instantiating all valid configurations of features. We evaluate our approach through a series of experiments using project data from the transportation, healthcare, and space exploration domains, and discuss its implementation in an industrial environment. Finally, we show how our approach can create a robust baseline against which new tracing techniques can be evaluated.

Domain analysis is a labor-intensive task in which related software systems are analyzed to discover their common and variable parts. Many software projects include extensive domain analysis activities, intended to jumpstart the requirements process through identifying potential features. In this paper, we present a recommender system that is designed to reduce the human effort of performing domain analysis. Our approach relies on data mining techniques to discover common features across products as well as relationships among those features. We use a novel incremental diffusive algorithm to extract features from online product descriptions, and then employ association rule mining and the (k)-nearest neighbor machine learning method to make feature recommendations during the domain analysis process. Our feature mining and feature recommendation algorithms are quantitatively evaluated and the results are presented. Also, the performance of the recommender system is illustrated and evaluated within the context of a case study for an enterprise-level collaborative software suite. The results clearly highlight the benefits and limitations of our approach, as well as the necessary preconditions for its success.

Software Engineering challenges and contests are becoming increasingly popular for focusing researchers' efforts on particular problems. Such contests tend to follow either an exploratory model, in which the contest holders provide data and ask the contestants to discover “interesting things” they can do with it, or task-oriented contests in which contestants must perform a specific task on a provided dataset. Only occasionally do contests provide more rigorous evaluation mechanisms that precisely specify the task to be performed and the metrics that will be used to evaluate the results. In this paper, we propose actionable and crowd-sourced contests: actionable because the contest describes a precise task, datasets, and evaluation metrics, and also provides a downloadable operating environment for the contest; and crowd-sourced because providing these features creates accessibility to Information Technology hobbyists and students who are attracted by the challenge. Our proposed approach is illustrated using research challenges from the software traceability area as well as an experimental workbench named TraceLab.

TraceLab is designed to empower future traceability research, through facilitating innovation and creativity, increasing collaboration between researchers, decreasing the startup costs and effort of new traceability research projects, and fostering technology transfer. To this end, it provides an experimental environment in which researchers can design and execute experiments in TraceLab's visual modeling environment using a library of reusable and user-defined components. TraceLab fosters research competitions by allowing researchers or industrial sponsors to launch research contests intended to focus attention on compelling traceability challenges. Contests are centered around specific traceability tasks, performed on publicly available datasets, and are evaluated using standard metrics incorporated into reusable TraceLab components. TraceLab has been released in beta-test mode to researchers at seven universities, and will be publicly released via CoEST.org in the summer of 2012. Furthermore, by late 2012 TraceLab's source code will be released as open source software, licensed under GPL. TraceLab currently runs on Windows but is designed with cross platforming issues in mind to allow easy ports to Unix and Mac environments.

Rapid prototypes are often developed early in the software development process in order to help project stakeholders explore ideas for possible features, and to discover, analyze, and specify requirements for the project. As prototypes are typically thrown-away following the initial analysis phase, it is imperative for them to be created quickly with little cost and effort. Tool support for finding and reusing components from open-source repositories offers a major opportunity to reduce this manual effort. In this paper, we present a system for rapid prototyping that facilitates software reuse by mining feature descriptions and source code from open-source repositories. Our system identifies and recommends features and associated source code modules that are relevant to the software product under development. The modules are selected such that they implement as many of the desired features as possible while exhibiting the lowest possible levels of external coupling. We conducted a user study to evaluate our approach and the results indicated that our proposed system returned packages that implemented more features and were considered more relevant than the state-of-the-art approach.

The software architectures of business, mission, or safety critical systems must be carefully designed to balance an exacting set of quality concerns describing characteristics such as security, reliability, and performance. Unfortunately, software architectures tend to degrade over time as maintainers modify the system without understanding the underlying architectural decisions. Although this problem can be mitigated by manually tracing architectural decisions into the code, the cost and effort required to do this can be prohibitively expensive. In this paper we therefore present a novel approach for automating the construction of traceability links for architectural tactics. Our approach utilizes machine learning methods and lightweight structural analysis to detect tactic-related classes. The detected tactic-related classes are then mapped to a Tactic Traceability Information Model. We train our trace algorithm using code extracted from fifteen performance-centric and safety-critical open source software systems and then evaluate it against the Apache Hadoop framework. Our results show that automatically generated traceability links can support software maintenance activities while helping to preserve architectural qualities.

In many software intensive systems traceability is used to support a variety of software engineering activities such as impact analysis, compliance verification, and requirements validation. However, in practice, traceability links are often created towards the end of the project specifically for approval or certification purposes. This practice can result in inaccurate and incomplete traces, and also means that traceability links are not available to support early development efforts. We address these problems by presenting a trace recommender system which pushes recommendations to project stakeholders as they create or modify traceable artifacts. We also introduce the novel concept of a trace obligation, which is used to track satisfaction relations between a target artifact and a set of source artifacts. We model traceability events and subsequent actions, including user recommendations, using the Business Process Modeling Notation (BPMN). We demonstrate and evaluate the efficacy of our approach through an illustrative example and a simulation conducted using the software engineering artifacts of a robotic system for supporting arm rehabilitation. Our results show that tracking trace obligations and generating trace recommendations throughout the active phases of a project can lead to early construction of traceability knowledge.

Traceability underlies many important software and systems engineering activities, such as change impact analysis and regression testing. Despite important research advances, as in the automated creation and maintenance of trace links, traceability implementation and use is still not pervasive in industry. A community of traceability researchers and practitioners has been collaborating to understand the hurdles to making traceability ubiquitous. Over a series of years, workshops have been held to elicit and enhance research challenges and related tasks to address these shortcomings. A continuing discussion of the community has resulted in the research roadmap of this paper. We present a brief view of the state of the art in traceability, the grand challenge for traceability and future directions for the field.

In current project environments, requirements often evolve throughout the project and are worked on by stakeholders in large and distributed teams. Such teams often use online tools such as mailing lists, bug tracking systems or online discussion forums to communicate, clarify or coordinate work on requirements. In this kind of environment, the expected evolution from initial idea, through clarification, to a stable requirement, often stagnates. When project managers are not aware of underlying problems, development may proceed before requirements are fully understood and stabilized, leading to numerous implementation issues and often resulting in the need for early redesign and modification. In this paper, we present an approach to analyzing online requirements communication and a method for the detection and classification of clarification events in requirement discussions. We used our approach to analyze online requirements communication in the IBM®Rational Team Concert®(RTC) project and identified a set of six clarification patterns. Since a predominant amount of clarifications through the lifetime of a requirement is often indicative of problematic requirements, our approach lends support to project managers to assess, in real-time, the state of discussions around a requirement and promptly react to requirements problems.

Architectural tactics are important building blocks of software architecture. Tactics come in many shapes and sizes, describe solutions for addressing specific quality concerns, and are prevalent across high-performance fault-tolerant systems. Once a decision is made to utilize a tactic, the developer must generate a concrete plan for realizing the tactic in the design and code. Unfortunately, the variability points found in individual tactics can make this a challenging task. To address this knowledge gap, we conducted a study to investigate how design patterns were used to implement various tactics. Data mining techniques were used to identify potential pattern instances within tactic implementations. Our manual analysis of the retrieved data identified a distinct set of variability points for each tactic, as well as corresponding design patterns used to address them. From these observations we construct tactic-level decision trees depicting variability points of a tactic and generate a reference model which provides implementation guidance.

We present a recommender system that models and recommends product features for a given domain. Our approach mines product descriptions from publicly available online specifications, utilizes text mining and a novel incremental diffusive clustering algorithm to discover domain-specific features, generates a probabilistic feature model that represents commonalities, variants, and cross-category features, and then uses association rule mining and the k-Nearest-Neighbor machine learning strategy to generate product specific feature recommendations. Our recommender system supports the relatively labor-intensive task of domain analysis, potentially increasing opportunities for re-use, reducing time-to-market, and delivering more competitive software products. The approach is empirically validated against 20 different product categories using thousands of product descriptions mined from a repository of free software applications.

Software architecture is shaped by a diverse set of interacting and competing quality concerns, each of which may have broad-reaching impacts across multiple architectural views. Without traceability support, it is easy for developers to inadvertently change critical architectural elements during ongoing system maintenance and evolution, leading to architectural erosion. Unfortunately, existing traceability practices, tend to result in the proliferation of traceability links, which can be difficult to create, maintain, and understand. We therefore present a decision-centric approach that focuses traceability links around the architectural decisions that have shaped the delivered system. Our approach, which is informed through an extensive investigation of architectural decisions made in real-world safety-critical and performance-critical applications, provides enhanced support for advanced software engineering tasks.

The software architectures of safety and mission-critical systems are designed to satisfy and balance an exacting set of quality concerns describing characteristics such as performance, reliability, and safety. Unfortunately, practice has shown that long-term maintenance activities can erode these architectural qualities. In this paper we present a novel solution for preserving architectural qualities through the use of Tactic Traceability Information Models (tTIMs). A tTIM provides a reusable infrastructure of traceability links focused around a commonly implemented architectural tactic, as well as a set of mapping points for tracing the tactic into the architectural design and the implemented code. The use of tTIMs significantly reduces the effort needed to create and maintain traceability links, provides support for visualizing the rationale behind various architectural components, and delivers timely information to maintainers so that they can preserve critical architectural qualities while implementing modifications. Our approach is described and evaluated within the context of a mission-critical software-intensive system.

A software architecture is carefully designed to satisfy the quality concerns of its stakeholders, and as such, represents a systematic and intricately balanced set of design decisions which deliver required qualities such as performance, reliability, and safety. In practice, architectural degradation tends to occur over the lifetime of the software system, as developers make ongoing and incremental maintenance changes to the system without knowledge of its underlying design decisions. Fortunately, this problem can be alleviated by establishing traceability between concrete elements in the architecture and their associated design decisions, and then using these traceability links to keep developers informed of relevant architectural tactics, styles, and design patterns throughout the development process. This paper focuses on the task of creating and using such traceability links. We present six trace creation patterns describing techniques and supporting structures for creating architecturally significant traceability links, and two usage patterns describing techniques for using the created links to help preserve qualities in the architectural design. The patterns described in this paper emerged from our experiences and observations of tracing architectural concerns in safety critical systems.

The primary goal of requirements engineering research is to propose, develop, and validate effective solutions for important practical problems. However practice has shown that successful projects often take from 20-25 years to reach the stage of full industry adoption, while many other projects fizzle out and never advance beyond the initial research phase. In this interactive panel, teams of researchers representing several different requirements engineering research areas, bring ideas for technology transfer to a panel of industrial and government practitioners. The teams proceed through a series of interactive presentations and receive feedback from panelists. Underlying the game-show genre of the panel is the more serious goal to foster conversation between practitioners and researchers in order to improve the effectiveness of technology transfer in the requirements engineering community.

Service Oriented Architecture (SOA) has been recognized as an important paradigm for software engineering. Several organizations are in the process of adopting and evolving SOA deployments. In this paper we present IntelliTrace, an intelligent traceability framework to support impact analysis across different modeling layers of a SOA based system. The framework uses traceability links among different SOA artifacts to analyze the impact that changes in SOA-based systems can have in key performance indicators. The change impact analysis is triggered by different situations such as changes at the service level, business process level, goal level, key performance indicators, and SOA infrastructure. A prototype tool has been implemented in order to illustrate and evaluate the framework. An extensive case study built around an online airline reservation system is used to evaluate the framework.

This paper presents a visual modeling notation for use in planning globally distributed requirements engineering projects. An underlying meta-model defines the elements of the modeling language, including site locations, stakeholder roles, communication flows, critical documents, and supporting tools and repositories. The modeling notation is motivated through the findings of eight in-depth interviews with requirements analysts who had worked on requirements elicitation, analysis, and specification tasks in globally distributed projects. We illustrate the modeling notation with examples drawn from telecommunications, video gaming, retail, and consulting projects. Based on a set of recurring problems and best practices identified in our interviews, the models are then analyzed, and specific recommendations are made to mitigate the identified risks.

Regulatory standards, designed to protect the safety, security, and privacy of the public, govern numerous areas of software intensive systems. Project personnel must therefore demonstrate that an as-built system meets all relevant regulatory codes. Current methods for demonstrating compliance rely either on after-the-fact audits, which can lead to significant refactoring when regulations are not met, or else require analysts to construct and use traceability matrices to demonstrate compliance. Manual tracing can be prohibitively time-consuming; however automated trace retrieval methods are not very effective due to the vocabulary mismatches that often occur between regulatory codes and product level requirements. This paper introduces and evaluates two machine-learning methods, designed to improve the quality of traces generated between regulatory codes and product level requirements. The first approach uses manually created traceability matrices to train a trace classifier, while the second approach uses web-mining techniques to reconstruct the original trace query. The techniques were evaluated against security regulations from the USA government's Health Insurance Privacy and Portability Act (HIPAA) traced against ten healthcare related requirements specifications. Results demonstrated improvements for the subset of HIPAA regulations that exhibited high fan-out behavior across the requirements datasets.

Automated trace retrieval methods can significantly reduce the cost and effort needed to create and maintain requirements traces. However, the set of generated traces is generally quite imprecise and must be manually evaluated by analysts. In applied settings when the retrieval algorithm is unable to find the relevant links for a given query, a human user can improve the trace results by manually adding additional search terms and filtering out unhelpful ones. However, the effectiveness of this approach is largely dependent upon the knowledge of the user. In this paper we present an automated technique for replacing the original query with a new set of query terms. These query terms are learned through seeding a web-based search with the original query and then processing the results to identify a set of domain-specific terms. The query-mining algorithm was evaluated and fine-tuned using security regulations from the USA government's Health Insurance Privacy and Portability Act (HIPAA) traced against ten healthcare related requirements specifications.

Requirements elicitation has long been recognized as a crucial activity in any software development project. Unfortunately, the traditional elicitation practices do not scale well when applied to larger projects, where knowledge is distributed across numerous geographically dispersed stakeholders. As a result, new distributed requirements elicitation tools have started to surface, such as online forums and wiki pages. In our previous work, we introduced a framework for supporting distributed elicitation by utilizing data mining and machine learning techniques to automatically group stakeholder ideas into forums, and by using recommender system technologies to help promote these forums to potentially interested stakeholders. The framework is designed to create an open and more inclusive environment where points of view, conflicts, interests and tradeoffs are identified as early as possible. In this paper, we present two substantial enhancements to the Recommender System component of this framework, and demonstrate through experiments how they improve the quality of the recommendations.

This panel builds upon a growing wave of reality television shows and proposes to go in search of requirements engineering's very own next top model. Through a series of tasks, some pre-prepared and some assigned on the fly, a number of small teams will compete for this prestigious title. Teams will be challenged to illustrate the power and flexibility of their favorite approach for requirements modeling, while conference participants will act as the ultimate judges as they vote approaches off the panel round-by-round. Does the requirements engineering community have any brave modelers? Is the requirements engineering community ready for a model showdown? Come to the panel session and find out!

Recommender systems can be used in online forums to recommend discussion topics to users; however as these forums are characterized by a constant influx of new users and new posts, it is important to consider the performance of the recommender system under a scenario in which the internal composition of the items to be recommended, i.e., discussion threads, and the user preferences are constantly changing. In this paper we describe and evaluate a forum recommender designed to handle the challenges of dynamically evolving internet forums used to gather and discuss feature requests for various software products. In particular, we empirically show that two proposed enhancements to the representations of user profiles will result in improved recommendation effectiveness in dynamic environments.

Managing large-scale software projects involves a number of activities such as viewpoint extraction, feature detection, and requirements management, all of which require a human analyst to perform the arduous task of organizing requirements into meaningful topics and themes. Automating these tasks through the use of data mining techniques such as clustering could potentially increase both the efficiency of performing the tasks and the reliability of the results. Unfortunately, the unique characteristics of this domain, such as high dimensional, sparse, noisy data sets, resulting from short and ambiguous expressions of need, as well as the need for the interactive engagement of stakeholders at various stages of the process, present difficult challenges for standard clustering algorithms. In this paper, we propose a semi-supervised clustering framework, based on a combination of consensus-based and constrained clustering techniques, which can effectively handle these challenges. Specifically, we provide a probabilistic analysis for informative constraint generation based on a co-association matrix, and utilize consensus clustering to combine multiple constrained partitions in order to generate high-quality, robust clusters. Our approach is validated through a series of experiments on six well-studied TREC data sets and on two sets of user requirements.

Requirements related problems, especially those originating from inadequacies in the human-intensive task of eliciting stakeholderspsila needs and desires, have contributed to many failed and challenged software projects. This is especially true for large and complex projects in which requirements knowledge is distributed across thousands of stakeholders. This short paper introduces a new process and related framework that utilizes data mining and recommender technologies to create an open, scalable, and inclusive requirements elicitation process capable of supporting projects with thousands of stakeholders. The approach is illustrated and evaluated using feature requests mined from an open source software product.

This panel presents and discusses effective techniques for teaching requirements engineering principles and practices, in ways which actively engage students in the learning process.

Successful software development involves the elicitation, implementation, and management of critical systemic requirements related to qualities such as security, usability, and performance. Unfortunately, even when such qualities are carefully incorporated into the initial design and implemented code, there are no guarantees that they will be consistently maintained throughout the lifetime of the software system. Even though it is well known that system qualities tend to erode as functional and environmental changes are introduced, existing regression testing techniques are primarily designed to test the impact of change upon system functionality rather than to evaluate how it might affect more global qualities. The concept of using goal-centric traceability to establish relationships between a set of strategically placed assessment models and system goals is introduced. This paper describes the process, algorithms, and techniques for utilizing goal models to establish executable traces between goals and assessment models, detect change impact points through the use of automated traceability techniques, propagate impact events, and assess the impact of change upon systemic qualities. The approach is illustrated through two case studies.

Automated trace tools dynamically generate links between various software artifacts such as requirements, design elements, code, test cases, and other less structured supplemental documents. Trace algorithms typically utilize information retrieval methods to compute similarity scores between pairs of artifacts. Results are returned to the user as a ranked set of candidate links, and the user is then required to evaluate the results through performing a top-down search through the list. Although clustering methods have previously been shown to improve the performance of information retrieval algorithms by increasing understandability of the results and minimizing human analysis effort, their usefulness in automated traceability tools has not yet been explored. This paper evaluates and compares the effectiveness of several existing clustering methods to support traceability; describes a technique for incorporating them into the automated traceability process; and proposes new techniques based on the concepts of theme cohesion and coupling to dynamically identify optimal clustering granularity and to detect cross-cutting concerns that would otherwise remain undetected by standard clustering algorithms. The benefits of utilizing clustering in automated trace retrieval are then evaluated through a case study

This panel will discuss the role of quality requirements in bringing products successfully to market. Techniques for eliciting, modeling, balancing, specifying, and measuring quality requirements will be explored.

Budgetary restrictions and time-to-market deadlines often require stakeholders to prioritize requirements and decide which ones to include in a given product release. Lack of an effective prioritization and triage process can lead to problems such as missed deadlines, disorganized development efforts, and late discovery of architecturally significant requirements. Existing prioritization techniques do not provide sufficient automation for large projects with hundreds of stakeholders and thousands of potentially conflicting requests and requirements. This paper therefore proposes an approach for automating a significant part of the prioritization process. The proposed method utilizes a probabilistic traceability model combined with a standard hierarchical clustering algorithm to cluster incoming stakeholder requests into hierarchical feature sets. Additional cross-cutting clusters are then generated to represent factors such as architecturally significant requirements or impacted business goals. Prioritization decisions are initially made at the feature level and then more critical requirements are promoted according to their relationships with the identified cross-cutting concerns. The approach is illustrated and evaluated through a case study applied to the requirements of the ice breaker system.

Even though traceability is legally required in most safety critical software applications and is a recognized component of many software process improvement initiatives, organizations continue to struggle to implement it in a cost-effective manner. This paper addresses the problems and challenges of requirements traceability and asks questions such as "How much traceability is enough?" and "What kinds of traceability provide cost effective solutions?" Traditional, automated, and lean traceability methods are all discussed

Dynamic trace retrieval provides an alternate option to traditional traceability methods such as matrices, hyperlinks, and manual link construction. Instead of relying upon manually constructed and maintained traces, links are generated dynamically on an 'as-needed' basis using information retrieval techniques. Prior work in this area has indicated that in order to retrieve between 90% to 95% of the correct traces, only low precision levels can be obtained, which means that analysts must spend time filtering out unwanted links. This paper describes a method for improving the precision of trace results through incorporating the use of phrases detected and constructed from requirements using a part-of-speech tagger. A project glossary is also used to find additional phrases and weight the contributions of key phrases and terms. The approach is implemented in a probabilistic trace retrieval tool and evaluated through a series of experiments. The results show that phrasing can significantly increase the accuracy of the dynamic trace retrieval tool by generally increasing precision, and also by moving good trace links towards the top of the candidate links list

Goal oriented methods help software engineers to model high-level systemic goals, propose and evaluate architectural solutions, and detect and resolve conflicts that occur. This paper describes a new technique, known as softgoal traceability patterns, for enabling reusable class mechanisms such as design patterns to be applied within a goal-oriented framework. Softgoal traceability patterns increase the reliability of a design in respect to its goals through the automated generation of design elements and the establishment of bidirectional traces between goals and design. These traces are used to monitor the integrity of the design in respect to architectural quality goals, and to support impact analysis when design changes are proposed. Softgoal traceability patterns are described using the well-known Observer pattern and then expanded with a more complex pattern that incorporates authentication

Finding the right traceability process that delivers effective and efficient traceability can be difficult. This panel explores traceability challenges and solutions for finding the right techniques and process to deliver costeffective traceability within an organization.

This paper introduces an information retrieval based approach for automating the detection and classification of non-functional requirements (NFRs). Early detection of NFRs is useful because it enables system level constraints to be considered and incorporated into early architectural designs as opposed to being refactored in at a later time. Candidate NFRs can be detected in both structured and unstructured documents, including requirements specifications that contain scattered and non-categorized NFRs, and freeform documents such as meeting minutes, interview notes, and memos containing stakeholder comments documenting their NFR related needs. This paper describes the classification algorithm and then evaluates its effectiveness in an experiment based on fifteen requirements specifications developed as term projects by MS students at DePaul University. An additional case study is also described in which the approach is used to classifying NFRs from a large free form requirements document obtained from Siemens Logistics and Automotive Organization

Poirot is a Web-based tool supporting traceability of distributed heterogeneous software artifacts. A probabilistic network model is used to generate traces between requirements, design elements, code and other artifacts stored in distributed 3rdparty case tools such as DOORS, rational rose, and source code repositories. The tool is designed with extensibility in mind, so that additional artifact types and 3rdparty case tools can be easily added. Trace results are displayed in both a textual and visual format. This paper briefly describes the underlying probabilistic model, and the user interface of the tool, and then discusses Poirot's deployment and use in an industrial setting

This tutorial introduces a financially responsible approach to requirements prioritization that enhances the value creating potential of a software development project. The approach, known as the Incremental Funding Method (IFM), is described in the book "Software by Numbers: Low-risk, High-Return Development" [2,3]. Tutorial attendees will learn how to group requirements into "chunks" of revenue-generating functionality known as Minimal Marketable Features (MMFs), and how to carefully sequence those MMFs in order to maximize the overall value of the project, reduce initial funding investments, and manipulate other project metrics such as the time needed for a project to reach break-even status. A gentle introduction to financial analysis will also equip participants to analyze and understand the impact of other requirements prioritization decisions upon the financial returns of a project. This process is applicable within any iterative development approach.

This paper describes a Goal Centric approach for effectively maintaining critical system qualities such as security, performance, and usability throughout the lifetime of a software system. In Goal Centric Traceability (GCT) non-functional requirements and their interdependencies are modeled as softgoals in a Softgoal Interdependency Graph (SIG). A probabilistic network model is then used to dynamically retrieve links between classes affected by a functional change and elements within the SIG. These links enable developers to identify potentially impacted goals; to analyze the level of impact on those goals; to make informed decisions concerning the implementation of the proposed change; and finally to develop appropriate risk mitigating strategies. This paper also reports experimental results for the link retrieval and illustrates the GCT process through an example of a change applied to a road management system.

Establishing and maintaining traceability links and consistency between software artifacts produced or modified in the software life-cycle are costly and tedious activities that are crucial but frequently neglected in practice. Traceability between the free text documentation associated with the development and maintenance cycle of a software system and its source code are crucial in a number of tasks such as program comprehension, software maintenance, and software verification & validation. Finally, maintaining traceability links between subsequent releases of a software system is important for evaluating relative source code deltas, highlighting effort/code variation inconsistencies, and assessing the change history. The main theme of the workshop is focused on understanding and defining the foundations for consistency and change management of software systems within the scope of artifact-to-artifact (model-to-model) traceability.The workshop will address the following issues:A formal definition of model to model traceabilityTraceability between artifacts and processesThe semantics of traceability linksRecovery of traceability linksVisualization of traceability linksInteroperable approaches to support traceabilityTraceability in emerging forms of software engineering including production lines, frameworks, components, etc..The goals of the workshop are to:Broaden awareness within the software engineering community of the potential for the application of traceabilityFacilitate the exchange of ideas and interaction between international researchersDefine open research problems faced in realizing usable approaches for traceabilityConstruct a foundation of materials for future research on traceability .For more information please visit the workshop web site is: http://re.cs.depaul.edu/tefse05/. The workshop proceedings are available through the ACM digital library.

Requirements traceability provides critical support throughout all phases of a software development project. However practice has repeatedly shown the difficulties involved in long term maintenance of traditional traceability matrices. Dynamic retrieval methods minimize the need for creating and maintaining explicit links and can significantly reduce the effort required to perform a manual trace. Unfortunately they suffer from recall and precision problems. This paper introduces three strategies for incorporating supporting information into a probabilistic retrieval algorithm in order to improve the performance of dynamic requirements traceability. The strategies include hierarchical modeling, logical clustering of artifacts, and semi-automated pruning of the probabilistic network. Experimental results indicate that enhancement strategies can be used effectively to improve trace retrieval results thereby increasing the practicality of utilizing dynamic trace retrieval methods.

Although the benefits of requirements traceability are widely recognized, the actual practice of maintaining a traceability scheme is not always entirely successful. The traceability infrastructure underlying a software system tends to erode over its lifetime, as time-pressured practitioners fail to consistently maintain links and update impacted artifacts each time a change occurs, even with the support of automated systems. This paper proposes a new method of traceability based upon event-notification and is applicable even in a heterogeneous and globally distributed development environment. Traceable artifacts are no longer tightly coupled but are linked through an event service, which creates an environment in which change is handled more efficiently, and artifacts and their related links are maintained in a restorable state. The method also supports enhanced project management for the process of updating and maintaining the system artifacts.

Although requirements traceability is crucial in both the development and maintenance of a software system, traceability links and related artifacts tend to deteriorate, as time-pressured practitioners fail to systematically update them in response to change. Event-based traceability addresses this issue by establishing links through a loosely coupled publisher/subscriber scheme. Dependent entities subscribe to requirements and receive event notifications as changes occur. This paper focuses upon the role played by the requirements specification as a publisher of events. A set of standard change events is defined and a method for monitoring a user's actions within a requirements management environment and the subsequent recognition and publication of the change events is proposed. Early results obtained from testing this approach are reported.

Posing speculative questions about a software system is an important yet often unsupported activity. Current impact analysis techniques tend to focus upon the functionality of the system, whilst the effects of change upon performance requirements are largely ignored until after implementation. This tendency can lead to costly and time-consuming mistakes. Event-based traceability provides a robust method for handling both long-term evolutionary change as well as the short-term speculative change needed to support performance related impact analysis. By establishing dynamic links, capable of propagating data values and commands between requirements and performance models, it becomes possible to automate a wide range of speculative queries and to enhance the overall ability to predict the impact of change upon the performance of the system.

Software metrics increase our ability to understand the behavior of software systems. An accurate measurement provides us with solid understanding of the entity we are measuring. In Object Oriented software, most current metrics quantify a class's coupling complexity by simply counting the number of connections with other classes but such metrics are unable to capture the underlying complexity or tension of individual connections. In this paper we propose a technique for measuring the strength of interclass relationships that takes into account both the number of statements participating in the connection, as well as the complexity of those statements. Our approach introduces a new concept for measuring Object Oriented coupling complexity and provides a more sensitive measurement than traditional approaches. We present examples in C++ to support our method.

Because early design decisions can have a major long term impact on the performance of a system, early evaluation of the high-level architecture can be an important risk mitigation technique. This paper proposes a technique for predicting the volume of data that will flow across a network in a distributed system. The prediction is based upon anticipated execution of scenarios and can be applied at an extremely early stage of the design. It is driven by requirements specifications and captures dynamic metrics by defining typical usage patterns in terms of scenarios. Scenarios are then mapped to architectural components, and dataflow across inter-partition links is estimated. The feasibility of the approach is demonstrated through an experiment in which predicted metrics are compared to runtime measurements.

Function-Class Decomposition is a hybrid method that integrates structured analysis with an object oriented approach to decompose a system. The task of class identification is performed in parallel to the decomposition of the system into a hierarchy of functional modules. This hierarchy provides the infrastructure for a systematic approach to partitioning components for distribution and for evaluating key attributes of the resulting architecture. Complexity is reduced by the fact that partitioning decisions are made along the boundaries of previously identified groupings. Early evaluation of the resulting component distribution is also a key factor in mitigating the risks associated with developing distributed applications.