Embedded systems play a crucial role in today’s Internet-of-Things (IoT) ecosystems. These systems can range from simple sensors to edge Artificial Intelligence (AI) solutions. However, their complex Hardware (HW)/Software (SW) interactions demand new analytical methodologies which encompass both the HW and the SW execution.In this work, we present a novel approach for early visualization of complex HW/SW interactions during SW development for embedded systems. Our approach traces the lifetime of HW and SW functions during the simulation of a Virtual Prototype (VP), which represents the HW while executing the SW. We dynamically instrument the execution of the VP at runtime such that neither the VP binary file nor the SW binary file has to be modified for tracing. The results are presented as a Function Lifetime Diagram (FLD) by storing the data into the Fast Transaction Recording (FTR) file format, which can be visualized, e.g. by the Surfer waveform viewer.To demonstrate the effectiveness of our approach, we first analyze the HW and SW interactions of a Micro-Electro-Mechanical System (MEMS) sensor. More specifically, the root causes of two already identified HW/SW interaction issues are analyzed. Second, the application flow of an edge AI application for recognizing handwritten digits on a touch display utilizing a pretrained Neural Network (NN) is analyzed. These experiments demonstrate that FLDs provide an effective abstraction to foster a deeper understanding of the embedded system behavior. An additional runtime evaluation reveals an approximately 1.9-fold runtime overhead, demonstrating that our instrumentation approach remains runtime-efficient even for larger IoT applications.

Modern applications increasingly rely on embedded systems that incorporate visual interfaces developed utilizing so-called embedded graphics libraries. Verifying these embedded graphics libraries is challenging due to hardware dependencies and the lack of reference outputs. The lack of reference outputs is tackled in Metamorphic Testing (MT) by constructing two Firmware (FW) versions with distinct implementations that maintain the same input-output relationships. These relations are known as Metamorphic Relations (MRs). However, the development of these MRs remains a tedious and challenging task.In this paper, we present a novel approach for generating MRs for MT of embedded graphics libraries using Large Language Models (LLMs). Because directly creating MRs with simple prompts is too complex for the LLM, we employ proven prompting strategies to develop our LLM-assisted MR pipeline. Strategies include role prompting, least-to-most prompting, zero-shot prompting, constraint-based prompting, and style prompting. In our experiments, we verify a widely used embedded graphics library. We compare our results with an existing manual approach and demonstrate that LLM-assisted MRs nearly doubles coverage and identifies additional bugs.

Embedded graphics libraries are part of the firmware of embedded systems and provide complex functionalities optimized for specific hardware. After unit testing of embedded graphics libraries, integration testing is a significant challenge, in particular since the hardware is needed to obtain the output image as well as the inherent difficulty in defining the reference result. In this paper, we present a novel approach focusing on integration testing of embedded graphic libraries. We leverage Virtual Prototypes (VPs) and integrate them with Metamorphic Testing (MT). MT is a software testing technique that uncovers faults or issues in a system by exploring how its outputs change under predefined input transformations, without relying on explicit oracles or predetermined results. In combination with virtualizing the displays in VPs, we even eliminate the need for physical hardware. This allows us to develop a MT framework automating the verification process. In our evaluation, we demonstrate the effectiveness of our MT framework. On an extended RISC-V VP for the GD32V platform we found 15 distinct bugs for the widely used TFT eSPI embedded graphics library, confirming the strength our approach.

While the Hardware (HW) domain and the Software (SW) domain use the concept of coverage to measure the thoroughness of tests, there isn’t an established common metric that applies to both worlds. In this paper we make two major contributions: First, leveraging the abstraction of Virtual Prototypes (VPs), we unify HW/SW coverage by viewing the HW/SW system as a single model. This enables the measurement of structural HW/SW metrics like line, function, and branch coverage via a novel non-intrusive approach, where neither the VP (representing the HW) nor the SW requires any modification. Second, based on the unified HW/SW coverage, we introduce relation coverage. The innovation is that the user can define a relation between the frequency of executing lines in the SW and the execution count of corresponding lines of the HW model. This relation expresses expected behavior to be covered during testing. As a case study, we consider HW/SW testing of a Gyroscope sensor controlled by SW running on a RISC-V VP.