This paper investigates the development of self-aware mechanisms for automated vehicles, introducing the notion of an automation state estimation system. This system is capable to understand its capabilities in a given context, and can leverage that knowledge to estimate the current and near-future automation performance based on internal metrics, as well as external, static (e.g. lane geometry) and dynamic environmental elements (e.g. traffic and weather information). From an application perspective, we consider automation state estimation in the scope of automation mediation, as part of a broader and holistic mediation system, with the goal to tackle challenging aspects related to transitions of control, mode confusion, and driver engagement. We used real-world data for system design, and implemented the proposed automation estimation system in a prototype vehicle. Based on 70 hours of real-world driving, we also validated the performance of the automation state estimation for automation mediation purposes.

The integration of human factors (HF) knowledge is crucial when developing safety-critical systems, such as automated vehicles (AVs). Ensuring that HF knowledge is considered continuously throughout the AV development process is essential for several reasons, including efficacy, safety, and acceptance of these advanced systems. However, it is challenging to include HF as requirements in agile development. Recently, Requirements Strategies have been suggested to address requirements engineering challenges in agile development. By applying the concept of Requirements Strategies as a lens to the investigation of HF requirements in agile development of AVs, this paper arrives at three areas for investigation: a) ownership and responsibility for HF requirements, b) structure of HF requirements and information models, and c) definition of work and feature flows related to HF requirements. Based on 13 semi-structured interviews with professionals from the global automotive industry, we provide qualitative insights in these three areas. The diverse perspectives and experiences shared by the interviewees provide insightful views and helped to reason about the potential solution spaces in each area for integrating HF within the industry, highlighting the real-world practices and strategies used.

Abstract Driving automation systems, including autonomous driving and advanced driver assistance, are an important safety-critical domain. Such systems often incorporate perception systems that use machine learning to analyze the vehicle environment. We explore new or differing topics and challenges experienced by practitioners in this domain, which relate to requirements engineering (RE), quality, and systems and software engineering. We have conducted a semi-structured interview study with 19 participants across five companies and performed thematic analysis of the transcriptions. Practitioners have difficulty specifying upfront requirements and often rely on scenarios and operational design domains (ODDs) as RE artifacts. RE challenges relate to ODD detection and ODD exit detection, realistic scenarios, edge case specification, breaking down requirements, traceability, creating specifications for data and annotations, and quantifying quality requirements. Practitioners consider performance, reliability, robustness, user comfort, and—most importantly—safety as important quality attributes. Quality is assessed using statistical analysis of key metrics, and quality assurance is complicated by the addition of ML, simulation realism, and evolving standards. Systems are developed using a mix of methods, but these methods may not be sufficient for the needs of ML. Data quality methods must be a part of development methods. ML also requires a data-intensive verification and validation process, introducing data, analysis, and simulation challenges. Our findings contribute to understanding RE, safety engineering, and development methodologies for perception systems. This understanding and the collected challenges can drive future research for driving automation and other ML systems.

Software that contains machine learning algorithms is an integral part of automotive perception, for example, in driving automation systems. The development of such software, specifically the training and validation of the machine learning components, requires large annotated datasets. An industry of data and annotation services has emerged to serve the development of such data-intensive automotive software components. Wide-spread difficulties to specify data and annotation needs challenge collaborations between OEMs (Original Equipment Manufacturers) and their suppliers of software components, data, and annotations.This paper investigates the reasons for these difficulties for practitioners in the Swedish automotive industry to arrive at clear specifications for data and annotations. The results from an interview study show that a lack of effective metrics for data quality aspects, ambiguities in the way of working, unclear definitions of annotation quality, and deficits in the business ecosystems are causes for the difficulty in deriving the specifications. We provide a list of recommendations that can mitigate challenges when deriving specifications and we propose future research opportunities to overcome these challenges. Our work contributes towards the on-going research on accountability of machine learning as applied to complex software systems, especially for high-stake applications such as automated driving.

Due to the technical complexity and social impact, automated vehicle (AV) development challenges the current state of automotive engineering practice. Research shows that it is important to consider human factors (HF) knowledge when developing AVs to make them safe and accepted. This study explores the current practices and challenges of the automotive industries for incorporating HF requirements during agile AV development. We interviewed ten industry professionals from several Swedish automotive companies, including HF experts and AV engineers. Based on our qualitative analysis of the semi-structured interviews, a number of current approaches for communicating and incorporating HF knowledge into agile AV development and associated challenges are discussed. Our findings may help to focus future research on issues that are critical to effectively incorporate HF knowledge into agile AV development.

Software engineering practice has shifted from the development of products in closed environments toward more open and collaborative efforts. Software development has become significantly interdependent with other systems (e.g. services, apps) and typically takes place within large ecosystems of networked communities of stakeholder organizations. Such software ecosystems promise increased innovation power and support for consumer-oriented software services at scale and are characterized by a certain openness of their information flows. While such openness supports project and reputation management, it also brings requirements engineering-related challenges within the ecosystem, such as managing dynamic, emergent contributions from the ecosystem stakeholders, as well as collecting their input while protecting their IP. In this paper, we report from a study of requirements communication and management practices within IBM ® ’s Collaborative Lifecycle Management ® product development ecosystem. Our research used multiple methods for data collection, including interviews within several ecosystem actors, on-site participatory observation, and analysis of online project repositories. We chart and describe the flow of product requirements information through the ecosystem, how the open communication paradigm in software ecosystems provides opportunities for “just-in-time” RE—and which relies on emergent contributions from the ecosystem stakeholders—, as well as some of the challenges faced when traditional requirements engineering approaches are applied within such an ecosystem. More importantly, we discuss two tradeoffs brought about by the openness in software ecosystems: (1) allowing open, transparent communication while keeping intellectual property confidential within the ecosystem and (2) having the ability to act globally on a long-term strategy while empowering product teams to act locally to answer end users’ context-specific needs in a timely manner. A sufficient level of openness facilitates contributions of emergent stakeholders. The ability to include important emergent contributors early in requirements elicitation appears to be a crucial asset in software ecosystems.

The size of a software artifact influences the software quality and impacts the development process. In industry, when software size exceeds certain thresholds, memory errors accumulate and development tools might not be able to cope anymore, resulting in a lengthy program start up times, failing builds, or memory problems at unpredictable times. Thus, foreseeing critical growth in software modules meets a high demand in industrial practice. Predicting the time when the size grows to the level where maintenance is needed prevents unexpected efforts and helps to spot problematic artifacts before they become critical.Although the amount of prediction approaches in literature is vast, it is unclear how well they fit with prerequisites and expectations from practice. In this paper, we perform an industrial case study at an automotive manufacturer to explore applicability and usability of prediction approaches in practice. In a first step, we collect the most relevant prediction approaches from literature, including both, approaches using statistics and machine learning. Furthermore, we elicit expectations towards predictions from practitioners using a survey and stakeholder workshops. At the same time, we measure software size of 48 software artifacts by mining four years of revision history, resulting in 4,547 data points. In the last step, we assess the applicability of state-of-the-art prediction approaches using the collected data by systematically analyzing how well they fulfill the practitioners' expectations.Our main contribution is a comparison of commonly used prediction approaches in a real world industrial setting while considering stakeholder expectations. We show that the approaches provide significantly different results regarding prediction accuracy and that the statistical approaches fit our data best.

The technology in the area of automated vehicles is gaining speed and promises many advantages. However, with the recent introduction of conditionally automated driving, we have also seen accidents. Test protocols for both, conditionally automated (e.g., on highways) and automated vehicles do not exist yet and leave researchers and practitioners with different challenges. For instance, current test procedures do not suffice for fully automated vehicles, which are supposed to be completely in charge for the driving task and have no driver as a back up. This paper presents current challenges of testing the functionality and safety of automated vehicles derived from conducting focus groups and interviews with 26 participants from five countries having a background related to testing automotive safety-related topics. We provide an overview of the state-of-practice of testing active safety features as well as challenges that needs to be addressed in the future to ensure safety for automated vehicles. The major challenges identified through the interviews and focus groups, enriched by literature on this topic are related to 1) virtual testing and simulation, 2) safety, reliability, and quality, 3) sensors and sensor models, 4)required scenario complexity and amount of test cases, and 5)handover of responsibility between the driver and the vehicle.

The validation of simulation models (e.g., of electronic control units for vehicles) in industry is becoming increasingly challenging due to their growing complexity. To systematically assess the quality of such models, software metrics seem to be promising. In this paper we explore the use of software metrics and outlier analysis as a means to assess the quality of model-based software. More specifically, we investigate how results from regression analysis applied to measurement data received from size and complexity metrics can be mapped to software quality. Using the moving averages approach, models were fit to data received from over 65,000 software revisions for 71 simulation models that represent different electronic control units of real premium vehicles. Consecutive investigations using studentized deleted residuals and Cook’s Distance revealed outliers among the measurements. From these outliers we identified a subset, which provides meaningful information (anomalies) by comparing outlier scores with expert opinions. Eight engineers were interviewed separately for outlier impact on software quality. Findings were validated in consecutive workshops. The results show correlations between outliers and their impact on four of the considered quality characteristics. They also demonstrate the applicability of this approach in industry.

Self-adaptive systems are capable of dealing with uncertainty at runtime handling complex issues as resource variability, changing user needs, and system intrusions or faults. If the requirements depend on context, runtime uncertainty will affect the execution of these contextual requirements. This work presents SACRE, a proof-of-concept implementation of an existing approach, ACon, developed by researchers of the Univ. of Victoria (Canada) in collaboration with the UPC (Spain). ACon uses a feedback loop to detect contextual requirements affected by uncertainty and data mining techniques to determine the best operationalization of contexts on top of sensed data. The implementation is placed in the domain of smart vehicles and the contextual requirements provide functionality for drowsy drivers.

A growing number of software systems is characterized by continuous evolution as well as by significant interdependence with other systems (e.g. services, apps). Such software ecosystems promise increased innovation power and support for consumer oriented software services at scale, and are characterized by a certain openness of their information flows. While such openness supports project and reputation management, it also brings some challenges to Requirements Engineering (RE) within the ecosystem. We report from a mixed-method study of IBM®'s CLM®ecosystem that uses an open commercial development model. We analyzed data from from interviews within several ecosystem actors, participatory observation, and software repositories, to describe the flow of product requirements information through the ecosystem, how the open communication paradigm in software ecosystems provides opportunities for `just-in-time' RE, as well as some of the challenges faced when traditional requirements engineering approaches are applied within such an ecosystem. More importantly, we discuss two tradeoffs brought about the openness in software ecosystems: i) allowing open, transparent communication while keeping intellectual property confidential within the ecosystem, and ii) having the ability to act globally on a long-term strategy while empowering product teams to act locally to answer end-users' context specific needs in a timely manner.

Today's systems are faced with the need of constant evolution to remain competitive, especially when looking at IT Ecosystems and their growing number of subsystems. As a prerequisite for these to stay competitive, system providers need a clear understanding of their stakeholder's needs. As systems tend to be increasingly complex nowadays, support an increasingly number of stakeholders, have a shorter release cycles to evolve and need to adapt to the environment and the users, some of the standard requirements elicitation techniques tend not to be suitable any more. Especially when adaptivity is necessary, system providers need to understand the context, in which the systems are used, but also the context of users for the adaptation. In this paper I concentrate on the largest stakeholder group, namely the end-users for requirements elicitation. Evaluation criteria include (i) support of context, (ii) scalability to large numbers of end-users, and (iii) scalability to large numbers of end-user's needs and problems that lead to new requirements. My literature review suggests that this important field is currently underrepresented in Requirements Engineering research. This research proposes to develop a framework that explains the different context types and their role for requirements elicitation. The framework is then used to investigate existing requirements elicitation techniques and their potential for considering context. It is also used to show how emerging techniques can further support requirements elicitation with context.