The advent of strong generative AI has a considerable impact on various software engineering tasks such as code repair, test generation, or language translation. While tools like GitHub Copilot are already in widespread use in interactive settings, automated approaches require a higher level of reliability before being usable in industrial practice. In this paper, we focus on three aspects that directly influence the quality of the results: a) the effect of automated feedback loops, b) the choice of Large Language Model (LLM), and c) the influence of behavior-preserving code changes. We study the effect of these three variables on an automated C-to-Rust translation system. Code translation from C to Rust is an attractive use case in industry due to Rust's safety guarantees. The translation system is based on a generate-and-check pattern, in which Rust code generated by the LLM is automatically checked for compilability and behavioral equivalence with the original C code. For negative checking results, the LLM is re-prompted in a feedback loop to repair its output. These checks also allow us to evaluate and compare the respective success rates of the translation system when varying the three variables. Our results show that without feedback loops LLM selection has a large effect on translation success. However, when the translation system uses feedback loops the differences across models diminish. We observe this for the average performance of the system as well as its robustness under code perturbations. Finally, we also identify that diversity provided by code perturbations can even result in improved system performance.

In this paper, we present a framework to assess the quality of a pedestrian detector in an autonomous driving scenario. To do this, we exploit performance metrics from the domain of computer vision on one side and so-called threat metrics from the motion planning domain on the other side. Based on a reachability analysis that accounts for the uncertainty in future motions of other traffic participants, we can determine the worst-case threat from the planning domain and relate it to the corresponding detection from the visual input. Our evaluation results for a RetinaNet on the Argoverse 1.1 [1] dataset show that already a rather simple threat metric such as time-to-collision (TTC) allows to select potentially dangerous interactions between the ego vehicle and a pedestrian when purely vision-based detections fail, even if they are passed to a subsequent object tracker. In addition, our results show that two different DNNs (Deep Neural Networks) with comparable performance differ significantly in the number of critical scenarios that we can identify with our method.

Due to the impressive performance of deep neural networks (DNNs) for visual perception, there is an increased demand for their use in automated systems. However, to use deep neural networks in practice, novel approaches are needed, e.g., for testing. In this work, we focus on the question of how to test deep learning-based visual perception functions for automated driving. Classical approaches for testing are not sufficient: A purely statistical approach based on a dataset split is not enough, as testing needs to address various purposes and not only average case performance. Additionally, a complete specification is elusive due to the complexity of the perception task in the open context of automated driving. In this article, we review and discuss existing work on testing DNNs for visual perception with a special focus on automated driving for test input and test oracle generation as well as test adequacy. We conclude that testing of DNNs in this domain requires several diverse test sets. We show how such tests sets can be constructed based on the presented approaches addressing different purposes based on the presented methods and identify open research questions.

Scenario-based approaches have gained popularity in the context of automated vehicles. As in any model-based approach, validation to real data needs to be considered. This work studies concrete scenarios based on real data from a large-scale and open road user trajectory dataset. In particular, we detail on cut-ins and (hard) braking maneuvers and study them based on existing threat metrics. In this work, we present the complete workflow starting from the pre-processing and validation of the data, the definition of concrete scenarios based on how we extract them from the dataset and their evaluation based on several threat metrics. We identify peculiarities of the dataset itself, as well as of the driving behavior of vehicles therein. As this work relies on an open dataset, results can be reproduced and readily compared.

Model-based development is an important paradigm for developing cyber-physical systems (CPS). The underlying assumption is that the functional behavior of a model is related to the behavior of a more concretized model or the real system. A formal definition of such a relation is called conformance relation. There are a variety of conformance relations, and the question arises of how to select a conformance relation for the development of CPS. The contribution of this article is a survey of the definitions and algorithms of conformance relations for CPS. Additionally, the article compares several conformance relations and provides guidance on which relation to select for specific problems. Finally, we discuss how to select inputs for testing conformance.

Industrial-sized hybrid systems are typically not amenable to formal verification techniques. For this reason, a common approach is to formally verify abstractions of (parts of) the original system. However, we need to show that this abstraction conforms to the actual system implementation including its physical dynamics. In particular, verified properties of the abstract system need to transfer to the implementation. To this end, we introduce a formal conformance relation, called reachset conformance, which guarantees transference of safety properties, while being a weaker relation than the existing trace inclusion conformance. Based on this formal relation, we present a conformance testing method which allows us to tune the trade-off between accuracy and computational load. Additionally, we present a test selection algorithm that uses a coverage measure to reduce the number of test cases for conformance testing. We experimentally show the benefits of our novel techniques based on an example from autonomous driving.

The theories underlying control engineering and real-time systems engineering use idealized models that mutually abstract from central aspects of the other discipline. Control theory usually assumes jitter-free sampling and negligible (constant) input-output latencies, disregarding complex real-world timing effects. Real-time systems theory uses abstract performance models that neglect the functional behavior and derives worst-case situations with limited expressiveness for control functions, e.g., In physically dominated automotive systems. In this paper, we propose an approach that integrates state-of-the art timing models into functional analysis. We combine physical, control and timing models by representing them as a network of hybrid automata. Closed-loop properties can then be verified on this hybrid automata network by using standard model checkers for hybrid systems. Since the computational complexity is critical for model checking, we discuss abstract models of timing behavior that seem particularly suited for this type of analysis. The approach facilitates systematic co-engineering between both control and real-time disciplines, increasing design efficiency and confidence in the system. The approach is illustrated by analyzing an industrial example, the control software of an electro-mechanical braking system, with the hybrid model checker Space Ex.

In large-scale resource-constrained systems, such as wireless sensor networks, global objectives should be ideally achieved through inexpensive local interactions. A technique satisfying these requirements is information potentials, in which distributed functions disseminate information about the process monitored by the network. Information potentials are usually computed through local aggregation or gossiping. These methods however, do not consider the topological properties of the network, such as node density, which could be exploited to enhance the performance of the system. This paper proposes a novel aggregation method with which a potential becomes sensitive to the network topology. Our method introduces the notion of affinity spaces, which allow us to uncover the deep connections between the aggregation scope (the radius of the extended neighborhood whose information is aggregated) and the network's Laplacian (which captures the topology of the connectivity graph). Our study provides two additional contributions: (i) It characterizes the convergence of information potentials for static and dynamic networks. Our analysis captures the impact of key parameters, such as node density, time-varying information, as well as of the addition (or removal) of links and nodes. (ii) It shows that information potentials are decomposed into wave-like eigenfunctions that depend on the aggregation scope. This result has important implications, for example it prevents greedy routing techniques from getting stuck by eliminating local-maxima. Simulations and experimental evaluation show that our main findings hold under realistic conditions, with unstable links and message loss.

Fair rate allocation deals with the fundamental problem of sharing the channel efficiently and fairly. In wireless networks, several notable works have proposed optimal solutions to this problem. These approaches work well for static networks, but rely on an assumption that renders them sub-optimal when nodes are mobile: at each computation step, nodes must collect the state of all their neighbors (1-hop knowledge assumption). In large-scale mobile networks, nodes need to continuously adapt to changing network conditions. Under these circumstances, it is hard to gather complete 1-hop information accurately and promptly. The key to any efficient solution in mobile networks is fast convergence with limited information. In this paper, we propose a simple decentralized algorithm for fair rate allocation that works well even with partial 1-hop information. The algorithm converges linearly and can be tuned to approach a wide range of trade-offs (from proportional to harmonic fairness). Our evaluation, using real-world mobility traces from 400 taxi cabs, shows that even in the challenging case of highly dynamic and dense networks, the algorithm assigns rate efficiently (mean error of 2.5% from the optimum), while using on average 37% of the 1-hop information.

In spite of using unreliable resource-constrained devices, sensor networks can nowadays deliver 99.9% of their data with duty cycles well below 1%. This remarkable performance is, however, dependent on one or more of the following assumptions: low traffic rates, medium size densities and static nodes.

The operation of wireless network protocol stacks is heavily dependent on the actual deployment of the system and especially on the corresponding network topology, e.g. due to channel contention. The nature of wireless communication does not allow for a-priori determination of network topology, network-defining metrics such as neighbor density and routing span may drastically differ for various deployments. Therefore, it is a difficult problem to foresee and consider the large number of possible topologies that a system may run on during protocol stack development. We propose to use an automated approach for searching topologies for which a protocol stack exhibits particularly poor quantitative performance. We formulate stress testing of protocol stacks on specific topologies as a multi-objective optimization problem and use an evolutionary algorithm for finding a set of small topologies that particularly stress the protocol stack of a wireless network. For searching the topology space, we present novel problem-specific variation operators and show their improvements on search performance in case studies. We showcase our results on stress testing using two protocol stacks for wireless sensor networks.

Cyber-Physical Systems (CPS) require a high degree of reliability and robustness. Hence it is important to assert their correctness with respect to extra-functional properties, like power consumption, temperature, etc. In turn the physical quantities may be exploited for assessing system implementations. This article develops a methodology for utilizing measurements of physical quantities for testing the conformance of a running CPS with respect to a formal description of its required behavior allowing to uncover defects. We present foundations and implementations of this approach and demonstrate its usefulness by conformance testing power measurements of a wireless sensor node with a formal model of its power consumption.

In this paper, we present a performance analysis of the communication stack of the PermaDozer application. Our offline analysis is based on long-term performance data from three diverse real-world wireless sensor network deployments. Two of the deployments considered are located in the Swiss mountains at 3.500 meters a.s.l., the third deployment is located along a river in the Swiss midlands. Apart from climatic differences, the three deployments also vary in network size, network density, node placement, and type and quantity of RF interference. From September 2008 to May 2011 more than 99.6 million WSN packets have been collected serving as the dataset. All three deployments are based on the same software and hardware. This allows us to comparatively study the performance of the PermaDozer protocol under different deployment settings and environmental conditions. For the period between June 2010 and May 2011, our networks achieved a data yield of >; 99.5%. But, we can also clearly notice that achieving this performance requires varying effort in terms of radio duty cycle resulting in different power consumption and network lifetime.

When a wireless sensor network is deployed indoors, sensor nodes often need to compete for channel usage with other, more powerful devices such as 802.11 access points. To mitigate the effects of such external interference we propose Chrysso, a protocol extension specifically designed for data collection applications that leverages the channel diversity of sensor node radios. In the face of external interference Chrysso switches only the directly affected set of nodes onto a new channel. In this paper, we present the design of Chrysso as well as its implementation in Contiki. Evaluation of Chrysso on two different testbeds, and its comparison with a state-of-the-art channel hopping protocol stack show that Chrysso is effective in maintaining a stable data flow at the sink, even under extreme interference.

Communication stacks are composed of distinct layers that, in principle, operate independently and interact through well-defined interfaces. However, resource constraints in sensor networks typically necessitate optimizations, leading to implicit assumptions and dependencies among layers (e.g., a collection protocol assumes the MAC protocol provides sufficient bandwidth). These dependencies are often tracked manually, yet become extremely complex as protocols evolve and requirements change. We propose to model assumptions and dependencies explicitly, as constraints on protocol parameters. This allows for using standard tools to generate feasible protocol configurations. We demonstrate the effectiveness of our approach using the example of FTSP running on top of a low-power listening MAC protocol.

We analyze push and pull for data collection in wireless sensor networks. Most applications to date use the traditional push approach, where nodes transmit sensed data immediately to the sink. Using a pull approach, nodes store the data in their local flash memory, and only engage in communication during dedicated collection phases. We show how one can transform an existing push-based collection protocol into a pull-based one, and compare the power consumption of both approaches on a 35-node testbed. Our results show that substantial energy gains are possible with pull, provided that the application can tolerate a long latency.

Wireless sensor networks (WSNs) are used for long-term observation and monitoring. Such long-lasting deployments require different maintenance tasks, such as the replacement of nodes and the most critical initial installation of the sensor nodes. During maintenance, the actual node placement is modified resulting in temporary topology fluctuations, which are very expensive in terms of energy. We propose the NoSE protocol stack enhancement for WSNs to target maintenance tasks. NoSE provides the functionality for switching the network between an operational state and a deep sleep state. The deep sleep state allows for switching the network to energy savings, while performing maintenance. The network may be woken up at any given time. During the time bounded start-up, a comprehensive neighborhood assessment provides a solid basis for the subsequent network topology setup. Thus the success of a maintenance task, e.g., the initial deployment of the nodes, can be instantly validated. We present NoSE on a case study focusing on the initialization of a fire-detector WSN validated on a testbed and in simulation.

A vital factor for a successful deployment of sensor nodes is testing of all system aspects in a realistic setup. This work presents a testbed architecture which allows for detailed monitoring and stimulation of a wireless sensor node. In particular, time-accurate state extraction and power measurements are provided in a distributed, yet synchronized context. The FlockLab testbed architecture provides a distributed lab instrument, where detailed observations of every sensor node enable thorough testing. Software services allow for formulating testcases and reliable test data collection.