Ensuring safety and providing rigorous behavioral guarantees are critical for robotic systems operating in high-stakes environments such as autonomous driving. Field testing is common, but costly and risky. Simulation-based testing offers a safer and lower-cost alternative for automatically generating traces for analysis and performance assessment. An oracle is essential for evaluating each trace, assessing whether a robot behavior fulfills key criteria such as task completion, safety, efficiency, and reliability. Supervised learning for oracle learning is accurate but costly and time-consuming due to manual labeling, whereas unsupervised learning requires no labels but often sacrifices accuracy. To overcome these limitations, we propose human-in-the-loop oracle learning as a new approach to develop and refine oracles that are capable of distinguishing good from bad behaviors with reduced manual effort. We illustrate this approach through a conceptual framework for integrating human-in-the-loop learning into robotic system evaluation.

As automotive systems become more intelligent than ever, they need to handle many functional tasks, resulting in more and more software programs running in automotive systems. However, whether a software program should be executed depends on the environmental conditions (surrounding conditions). For example, a deraining algorithm supporting object detection and image recognition should only be executed when it is raining. Supported by the advance of over-the-air (OTA) updates and plug-and-play systems, adaptive automotive systems, where the software programs are updated, activated, and deactivated before driving and during driving, can be realized. In this paper, we consider the upcoming environmental conditions of an automotive system and target the corresponding software selection problem during runtime. We formulate the problem as a set cover problem with timing constraints and then propose a heuristic approach to solve the problem. The approach is very efficient so that it can be applied during runtime, and it is a preliminary step towards the broad realization of adaptive automotive systems.