Healthcare Internet of Things (IoT) applications often integrate various third-party healthcare applications and medical devices through REST APIs, resulting in complex and interdependent networks of REST APIs. Oslo City’s healthcare department collaborates with various industry partners to develop these applications, enriched with diverse REST APIs that evolve during the DevOps process to accommodate evolving needs such as new features, services, and devices. Oslo City’s primary goal is to utilize automated solutions for continuous testing of REST APIs at each evolution stage to ensure dependability. Although the literature offers various automated REST API testing tools, their effectiveness in regression testing of the evolving REST APIs of healthcare IoT applications within a DevOps context remains undetermined. This article evaluates state-of-the-art and well-established REST API testing tools—specifically, RESTest, EvoMaster, Schemathesis, RESTler, and RestTestGen—for the regression testing of a real-world healthcare IoT application, considering failures, faults, coverage, regressions, and cost. We conducted experiments using all accessible REST APIs (17 APIs with 120 endpoints), and 14 releases evolved during DevOps. Overall, all tools generated tests leading to several failures, 18 potential faults, up to 84% coverage, and 23 regressions. Over 70% of tests generated by all tools fail to detect failures, resulting in significant overhead.

Testing healthcare Internet of Things (IoT) applications at system and integration levels necessitates integrating numerous medical devices. Challenges of incorporating medical devices are: (i) their continuous evolution, making it infeasible to include all device variants and (ii) rigorous testing at scale requires multiple devices and their variants, which is time-intensive, costly, and impractical. Our collaborator, Oslo City’s health department, faced these challenges in developing automated test infrastructure, which our research aims to address. In this context, we propose a meta-learning-based approach ( MeDeT ) to generate digital twins (DTs) of medical devices and adapt DTs to evolving devices. We evaluate MeDeT in Oslo City’s context using five widely used medical devices integrated with a real-world healthcare IoT application. Our evaluation assesses MeDeT ’s ability to generate and adapt DTs across various devices and versions using different few-shot methods, the fidelity of these DTs, the scalability of operating 1,000 DTs concurrently, and the associated time costs. Results show that MeDeT can generate DTs with over 96% fidelity, adapt DTs to different devices and newer versions with reduced time cost (around one minute), and operate 1,000 DTs in a scalable manner while maintaining the fidelity level, thus serving in place of physical devices for testing.

Healthcare applications using the Internet of Things (IoT) architecture are connected with various medical devices designed to serve patients. Rigorous system testing of health care IoT applications requires integrating multiple medical devices to ensure the dependability of these applications. The integration of numerous physical medical devices with varying versions is a costly and time-consuming process. In this regard, our previous work introduced the concept of employing digital twins (DTs) as substitutes for physical devices for testing purposes. Specifically, we presented a model-based approach to generate DTs of medicine dispensers. The evaluation of our approach with a Karie medicine dispenser demonstrated 92% fidelity of Karie DTs. From our experiences, we observed that the real operating environment of medical devices involves several non-deterministic factors, essential for DTs to reflect devices' behavior precisely. Therefore, we plan to devise a methodology to model and simulate the environment of medical devices DTs, taking into account environmental uncertainties. We intend to empirically evaluate our methodology in the real-world context to analyze the simulation of behavioral models of the environment and uncertain events generated for DTs.