Abstract Automotive test case specifications document test cases to be performed for a specific test object at a defined test level. They are a fundamental part of a structured automotive testing process, as required by the ISO 26262. The aim of our research is to identify challenges from a practitioner’s point of view that lead to poor quality of test case specifications and thus negatively impact time, cost, and probability of defect detection. We designed an exploratory case study to systematically identify challenges focusing on (C) creation, (P) processing, and (Q) quality assurance related aspects of test case specifications. We conducted 17 semi-structured interviews covering a German OEM as well as three of its automotive suppliers and analyzed them qualitatively. We investigated causes and consequences arising from the challenges. Additionally, we conducted a descriptive survey to assess frequency and criticality. The identified challenges were summarized in a taxonomy consisting of nine main categories: (1) availability and (2) content-related problems with input artifacts, problems related to (3) a lack of knowledge, (4) the test case description, (5) the test case specification content, (6) processes, (7) communication, (8) quality assurance, and (9) tools. The challenges were assessed by 26 internal and 10 external employees. Hence, we identified differences between these groups in terms of access to documents, incomplete requirements, scope of model series, process, and tool-related aspects. Overall, the study results underline the necessity of quality assurance measures for test case specifications. Based on the assessments, our research indicates a broad range of test case description related challenges that are promising candidates for improving test case specification quality.

Test case specifications are a fundamental part of a structured test process and required by standards like ISO 26262 in the automotive domain. Test cases are usually derived from requirements and provide a necessary basis for test implementation and execution. The aim of this work is to gain insights into the creation and processing of test case specifications from a practitioner's point of view. We decided on an explorative case study to identify challenges related to three problem areas: creation, processing and quality assessment related aspects of test case specifications. We conducted 17 semi-structured interviews covering a German OEM and three automotive suppliers. Afterwards, we analyzed the collected data qualitatively to identify challenges as well as causes and consequences arising from them. The identified challenges were categorized in a taxonomy consisting of nine main categories: (1) availability and (2) content-related problems with input artifacts, problems related to (3) a lack of knowledge, (4) the test case description, (5) the content of a test case specification, (6) quality assurance, (7) processes, (8) communication, and (9) tools. Overall, we observed a present awareness of the identified challenges, which is evident from different solutions to handle these challenges. Nevertheless, some of the current solutions are not efficient and still require a lot of manual work, for instance phrasing-based problems in test cases.

In requirements engineering literature dealing with natural language specifications, we usually find writing rules like `avoid passive voice' or `do not use weak words'. Adhering to such rules should result in understandable and unambiguous requirements. Passive voice, especially when used without an explicit actor, is considered to result in incomplete requirements. The usage of weak words is considered to result in imprecise requirements that are hardly testable. But is the inversion of the claim correct, i.e. does the violation of the writing rules result in problematic specifications? At least in our environment (the passenger car development of Mercedes-Benz) we observe that authors often use passive voice, and there are many requirements containing weak words. To answer this question, we conducted an empirical investigation whose results we report in this paper. The results of this investigation are quite surprising: The use of passive voice, even when the actor is missing, is almost never problematic, as the missing information (the actor) can in most cases easily derived from the context (i.e. surrounding requirements or the general project context). The usage of weak words may be considered problematic in approximately 12% of all occurrences. For an automatic analysis on weak words linguistic patterns can be defined to detect these problematic occurences.

Reuse of requirements leads to reduction in time spent for specification of new products. Variant management of requirement documents is an essential prerequisite in terms of a successful reuse of requirements. It supports the decisions if available requirements can be reused or not. One possibility to document the variability is feature modelling. One main challenge while introducing feature modelling in a grown environment is to extract product features from large natural language specifications. The current practice is a manual review of specifications conducted by domain experts. This procedure is very costly in terms of time. A promising approach to optimize feature identification is a semi-automatic identification of features in natural language specifications based on lexical analysis. This paper presents the current approaches used for handling variability in automotive specifications at Daimler passenger car development along with first experiences gained in using the optimized approach for feature identification using a lexical analysis.

Many (technical) systems are not developed from scratch but as an evolution of existing systems. Consequently, a large portion of the system requirements employed can be recycled when building the next version of the product. Usually, this recycling step is performed unsystematically, i.e. simply by copying and modifying complete requirements documents. In this paper, we present in a case study a lightweight requirements recycling approach which evolved from observations and concrete needs of projects at DaimlerChrysler Passenger Car Development. The basic idea of the approach is separation of model-dependent from model-independent requirements on the same level of abstraction. This notion is supported by document structures, criteria for identifying reusable requirements and tool support. The paper presents the core elements of the approach and provides observations and valuable experiences we made in the projects.

Ad-hoc recycling between requirements documents of product variants is a major source of requirements defects. We present an approach for systematic requirements recycling based on a combination of abstraction (in terms of a template) and traceability (between requirements). The main features of our approach are the use of conceptual models to determine relationships necessary for correct recycling and the focus on minimizing link selling. This approach can also be used to develop abstraction and traceability guidelines tailored to other application domains and traceability goals, such as change or project management.

The deployment of executable specifications has increased significantly in the last few years. Just as with any other specification documents, these specifications must be examined to ensure the necessary degree of quality. A common and successful technique used for examining traditional specifications is inspection. Now the question has arisen whether inspections on executable specification are the best choice, or if other techniques which use the execution capabilities of the models perform better. In this paper, we empirically compare several defect detection techniques for executable specifications. In particular, we examine inspections, testing, and ad-hoc simulation. Here, we use the specification languages Statemate and Matlab/Simulink. Also, we take a closer look at the inspection process itself and try to quantify the benefits of an inspection meeting for executable specifications.

The experience factory concept enables systematic learning and continuous improvement in software development. As with most learning initiatives, it is hard to establish. In our experience, there is a great deal of uncertainty and skepticism about the mission and contents of an experience factory. The starting phase is especially endangered through pitfalls or unexpected delays. As expectations vary and there is pressure to demonstrate success within only a few months, tension arises which may jeopardize the entire enterprise. In the course of a large-scale software improvement program, we have established three experience factories in different environments of the Daimler-Benz AG within two years. At each site, several application projects are involved. We describe how we approached the task, what actions we took, and the lessons we learned.