• The SAFT-GT toolchain enables semi-automatic Attack-Fault Tree generation for enhanced safety and security assessment in self-adaptive systems. • The toolchain efficiently integrates into the feedback loop of self-adaptive systems, allowing for dynamic updates based on security assessments. • A user study with domain experts confirms the toolchain’s relevance and practical applicability in real-world scenarios. • Performance experiments demonstrate that the Attack-Fault Tree generation pipeline operates within feasible time constraints, supporting real-time applications. • The complete toolchain and resources are provided for download, fostering further research and collaboration in the field. In the rapidly evolving landscape of software engineering, the demand for robust and secure systems has become increasingly critical. This is especially true for self-adaptive systems due to their complexity and the dynamic environments in which they operate. To address this issue, we designed and developed the SAFT-GT toolchain that tackles the multifaceted challenges associated with ensuring both safety and security. This paper provides a comprehensive description of the toolchain’s architecture and functionalities, including the Attack-Fault Trees generation and model combination approaches. We emphasize the toolchain’s ability to integrate seamlessly with existing systems, allowing for enhanced safety and security analyses without requiring extensive modifications and domain knowledge. Our proposed approach can address evolving security threats, including both known vulnerabilities and emerging attack vectors that could compromise the system. As a use case for the toolchain, we integrate it into the feedback loop of self-adaptive systems. Finally, to validate the practical applicability of the toolchain, we conducted an extensive user study involving domain experts, whose insights and feedback underscore the toolchain’s relevance and usability in real-world scenarios. Our findings demonstrate the toolchain’s effectiveness in real-world applications while highlighting areas for future improvements. The toolchain and associated resources are available in an open-source repository to promote reproducibility and encourage further research in this field.

Machine learning (ML) systems in finance raise concerns about fairness, bias, and regulatory compliance, especially in high-stakes areas like creditworthiness, lending, and risk assessment. Bias in ML can lead to systemic economic exclusion, disproportionately affecting marginalized communities. This paper explores fairness adequacy testing in financial ML models, addressing regulatory, technical, and ethical challenges while proposing bias mitigation strategies. Financial ML models face unique constraints, including strict privacy regulations such as the General Data Protection Regulation (GDPR), the Equal Credit Opportunity Act (ECOA) in the U.S., and Basel Committee fairness standards, along with reliance on proprietary data. While this work provides an overview of fairness challenges, the detailed scoping review will be conducted in a subsequent study to further analyze regulatory frameworks and industry practices. This paper aims to highlight key gaps and considerations that must be addressed to balance fairness and accuracy in ML-driven financial decision-making.

Machine Learning (ML)-enabled systems that run in safety-critical settings expose humans to risks. Hence, it is important to build such systems with strong assurances for domain-specific safety requirements. Simulation as well as metaheuristic optimizing search have proven to be valuable tools for online testing of ML-enabled systems for early detection of hazards. However, the efficient generation of effective test cases remains a challenging issue. In particular, the testing process shall produce as many failures as possible but also unveil diverse sets of failure scenarios.To study this phenomenon, we introduce a risk-driven test case generation and diversity analysis method tailored to ML-enabled systems. Our approach uses an online testing technique based on metaheuristic optimizing search to falsify domain-specific safety requirements. All test cases leading to hazards are then analyzed to assess their diversity by using clustering and interpretable ML. We evaluated our approach in a collaborative robotics case study showing that generating tests considering risk metrics represents an effective strategy. Furthermore, we compare alternative optimizing search algorithms and rank them based on the overall diversity of the test cases, ultimately showing that selecting the testing strategy based on the number of failures only may be misleading.

Cyber–physical systems (CPS) are increasingly self-adaptive, i.e. they have the ability to introspect and change their behavior. This self-adaptation process must be considered when modeling the safety and security aspects of the system. This study collects and compares security attacks and safety hazards on self-adaptive systems (SAS) described in the literature. In addition, mitigation and treatment strategies, as well as the modeling and analysis approaches, are investigated. We conducted a systematic literature review on 21 selected papers. The selection process included a database search on four scientific databases using a common search string (1430 papers), forward and backward snowballing (1402 papers), and filtering the results based on predefined inclusion and exclusion criteria. The coding scheme to analyze the content of the papers was obtained through research questions, existing domain-specific taxonomies, and open coding. Safety and security are not jointly modeled in the context of self-adaptive systems. The adaptation process is often not considered in the attack and hazard analysis due to naïve assumptions and modeling. The proposed approaches are mostly verified and validated through simulation often using simple use cases and scenarios. A thorough and joint modeling approach for safety and security in self-adaptive systems is still an open challenge that needs to be addressed. Further work is needed to address the gap between safety and security modeling in self-adaptive systems. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Metamorphic testing has proven to be effective for test case generation and fault detection in many domains. It is a software testing strategy that uses certain relations between input-output pairs of a program, referred to as metamorphic relations. This approach is relevant in the autonomous systems domain since it helps in cases where the outcome of a given test input may be difficult to determine. In this paper therefore, we provide an overview of metamorphic testing as well as an implementation in the autonomous systems domain. We implement an obstacle detection and avoidance task in autonomous drones utilising the GNC API alongside a simulation in Gazebo. Particularly, we describe properties and best practices that are crucial for the development of effective metamorphic relations. We also demonstrate two metamorphic relations for metamorphic testing of single and more than one drones, respectively. Our relations reveal several properties and some weak spots of both the implementation and the avoidance algorithm in the light of metamorphic testing. The results indicate that metamorphic testing has great potential in the autonomous systems domain and should be considered for quality assurance in this field.