Testing of dependable event-based systems is very important to ensure that all requirements (including nonfunctional requirements such as reliability, availability, safety and security) are met and the relevant standards are considered. Siemens Program and Systems Engineering is a company that builds dependable event-based systems in multiple domains. A special unit at PSE, the Support Center Test, focuses on testing issues. In this paper we provide an overview of the state-of-the-practice in testing dependable event-based systems and identify the challenges that have to be addressed in the future. We illustrate our findings by two case studies, one for a transportation system and one for telecommunications. The top three topics for improvement are: testability and requirements tracing, generation of test cases, and verification and validation

Software development standards demand requirements traceability without being explicit about the appropriate level of quality of trace links. Unfortunately, long-term trace utilizations are typically unknown at the time of trace acquisition which represents a dilemma for many companies. This paper suggests ways to balance the cost and benefits of requirements traceability. We present data from 3 case studies. Lessons learned suggest a traceability strategy that (1) provides trace links more quickly, (2) refines trace links according to user-definable value considerations, and (3) supports the later refinement of trace links in case the initial value considerations change.

Major software development standards mandate the establishment of trace links among software artifacts such as requirements, architectural elements, or source code without explicitly stating the required level of detail of these links. However, the level of detail vastly affects the cost and quality of trace link generation and important applications of trace analysis such as conflict analysis, consistency checking, or change impact analysis. In this paper, we explore these cost-quality trade-offs with three case study systems from different contexts - the open-source ArgoUML modeling tool, an industrial route-planning system, and a movie player. We report the cost-quality trade-off of automated trace generation with the Trace Analyzer approach and discuss its expected impact onto several applications that consume its trace information. In the study we explore simple techniques to predict and manipulate the cost-benefit trade-off with threshold-based filtering. We found that (a) 80% of the benefit comes from only 20% of the cost and (b) weak trace links are predominantly false trace links and can be efficiently eliminated through thresholds.

Project managers aim at keeping track of interdependencies between various artifacts of the software development lifecycle, to find out potential requirements conflicts, to better understand the impact of change requests, and to fulfill process quality standards, such as CMMI requirements. While there are many methods and techniques on how to technically store requirements traces, the economic issues of dealing with requirements tracing complexity remain open. In practice tracing is typically not an explicit systematic process, but occurs rather ad hoc with considerable hidden tracing-related quality costs. This paper reports a case study on value-based requirements tracing (VBRT) that systematically supports project managers in tailoring requirements tracing precision and effort based on the parameters stakeholder value, requirements risk/volatility, and tracing costs. Main results of the case study were: (a) VBRT took around 35% effort of full requirements tracing; (b) more risky or volatile requirements warranted more detailed tracing because of their higher change probability.