ABSTRACT Objective Antipatterns (APs) represent potential issues in software systems stemming from poor design choices, coding practices, and undisciplined development. This systematic literature review analyzes 97 primary studies (PSs) from 2005 to 2024, exploring the impact of APs on Object‐Oriented (OO), Service‐Oriented (SO), and Mobile‐Oriented (MO) systems across various quality attributes. Methods PSs are classified by techniques, datasets, evaluation measures, and tool support. Result Findings highlight the association of APs with increased maintenance costs (27.8%), fault‐proneness (26.8%), change‐proneness (12.3%), and evolution challenges (25.7%). Most studies employ descriptive statistics, regression analysis, and Pearson correlation, with limited datasets and tool support for SO and MO systems compared to OO systems. Intermediate source code representations and program comprehension strategies are commonly used for analysis. Conclusion These findings emphasize the need for further research on the impact of APs, particularly in MO systems, and their negative effects on software quality attributes.

IoT systems are increasingly developed and deployed across various domains, where End-to-End (E2E) testing is critical to ensure reliability and expected behavior. However, generating comprehensive tests remains challenging due to the heterogeneity, distributed nature, and unique characteristics of IoT systems, which limit the effectiveness of generic test generation approaches. Recent studies demonstrated the effectiveness of Large Language Models (LLM) for test generation in traditional software systems. Building on this foundation, this study explores and evaluates four distinct approaches for generating E2E tests from use case specifications (UCSs) tailored to IoT systems. These include (1) a custom, (2) a single-stage LLM, (3) a multi-stage LLM, and (4) a hybrid approach combining custom and LLM capabilities. We evaluated these approaches on an IoT system, focusing on correctness and scenario coverage criteria. Experimental results indicate that all approaches perform well, with notable variations in specific aspects of test generation. The custom and hybrid approaches are more reliable in producing correctly structured and complete tests, with the hybrid approach slightly outperforming others. This study is a work in progress, requiring further investigation to fully realize its potential.

Internet of Things (IoT) systems refer to interconnected systems of devices that collect, process, and exchange data. As IoT adoption continues to grow, ensuring effective testing is of paramount importance. However, testing IoT systems remains a challenge, particularly for software engineers, due to the need to test aspects beyond their primary area of expertise (e.g., security, sensor calibration, and connectivity). Testing aspects refer to any concept or concern that should be considered when testing a given system. While several frameworks for testing exist that focus on generic aspects of IoT systems, there is no dedicated framework for testing technical software engineering (SE) aspects of IoT systems. To address this gap, we propose and evaluateTISSEA, a framework to guide software engineers to test the technical software engineering (SE) aspects of IoT systems. We constructed TISSEA by identifying all possible technical software-engineering aspects from published taxonomies for IoT systems testing. Further, we mapped each aspect to the granularity of testing at each layer of the IoT system. We finally mapped each aspect with test orchestration strategies, test input artifacts, and execution strategies. We evaluated the TISSEA by surveying 22 professionals and conducting two case studies: (1) event logging and handling testing and (2) data integrity testing. The survey results show that professionals agreed with the proposed technical SE aspects for testing the device and application layers. However, the aspects proposed for testing the gateway and cloud layers still require further investigation. Results of the case studies indicate a gap between expected and captured log events. Regarding event handling, we found that some of the events reported by the system as successfully handled may include unhandled events that cannot be identified when relying on a single orchestration strategy. Regarding data integrity testing, we found that data can be altered at any node at any layer of the IoT system. However, accessing the original data allows the detection of modifications made to it at each node. Overall evaluation of TISSEA shows strong agreement with practitioners, and it could usefulness to test technical software engineering aspects of IoT systems.

The Internet of Things (IoT) is reshaping our lives, increasing the need for thorough pre-deployment testing. However, traditional software testing may not address the testing requirements of IoT systems, leading to quality challenges. A specific testing taxonomy is crucial, yet no widely recognized taxonomy exists for IoT system testing. We introduced an IoT-specific testing taxonomy that categorizes aspects of IoT systems testing into seven distinct categories. We mined testing aspects from 83 primary studies in IoT systems testing and built an initial taxonomy. This taxonomy was refined and validated through two rounds of surveys involving 16 and then 204 IoT industry practitioners. We assessed its effectiveness by conducting an empirical evaluation on two separate IoT systems, each involving 12 testers. Our findings categorize seven testing aspects: (1) testing objectives, (2) testing tools and artifacts, (3) testers, (4) testing stage, (5) testing environment, (6) Object Under Test (OUT) and metrics, and (7) testing approaches. The evaluation showed that testers equipped with the taxonomy could more effectively identify diverse test cases and scenarios. Additionally, we recommend new research opportunities to enhance the testing of IoT systems. • Conducted a literature review of 83 primary studies to develop an initial taxonomy for testing IoT systems, comprising seven key aspects: testing objectives, tools and artifacts, testers, stages, environments, Object Under Test (OUT), and testing approaches. • Refined and validated the proposed taxonomy through surveys involving 16 and 204 IoT industry practitioners. • Conducted an empirical evaluation using two case studies and 12 practitioners for each to assess the taxonomy’s effectiveness. • Provided structured guidance for practitioners to navigate and apply the taxonomy effectively. • Discussed insights from the empirical evaluation and offered recommendations for practitioners and researchers. • Set up two public access points for professionals to continuously access and stay updated with our IoT systems testing taxonomy. The first is hosted on the Ptidej website, while the second is available in a GitHub repository, ensuring that the latest version, incorporating newly identified aspects, is always accessible.

Scalability and maintainability challenges in monolithic systems have led to the adoption of microservices, which divide systems into smaller, independent services. However, migrating existing monolithic systems to microservices is a complex and resource-intensive task, which can benefit from machine learning (ML) to automate some of its phases. Choosing the right ML approach for migration remains challenging for practitioners. Previous works studied separately the objectives, artifacts, techniques, tools, and benefits and challenges of migrating monolithic systems to microservices. No work has yet investigated systematically existing ML approaches for this migration to understand the automated migration phases, inputs used, ML techniques applied, evaluation processes followed, and challenges encountered.We present a systematic literature review (SLR) that aggregates, synthesises, and discusses the approaches and results of 81 primary studies (PSs) published between 2015 and 2024. We followed the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) statement to report our findings and answer our research questions (RQs).We extract and analyse data from these PSs to answer our RQs. We synthesise the findings in the form of a classification that shows the usage of ML techniques in migrating monolithic systems to microservices. The findings reveal that some phases of the migration process, such as monitoring and service identification, are well-studied, while others, like packaging microservices, remain unexplored. Additionally, the findings highlight key challenges, including limited data availability, scalability and complexity constraints, insufficient tool support, and the absence of standardized bench-marking, emphasizing the need for more holistic solutions.

The Internet of Things (IoT) is increasingly prevalent, with systems developed across various domains. Choosing the right IoT architectural style is challenging due to the diversity of devices, dynamic environments, and real-time data needs. This choice significantly impacts system quality, requiring a careful balance of quality requirements and tradeoffs. Previous studies have not adequately identified the most suitable architectural styles for specific IoT quality needs. This study presents a systematic literature review of 103 primary studies (PSs) on IoT system quality requirements and architectural styles, assessing how each architectural style satisfies specific requirements. We followed the preferred reporting items for systematic review and meta-analysis (PRISMA) protocol to report our findings and answer three research questions (RQs). We selected PSs by applying inclusion and exclusion criteria to relevant papers published until the end of 2023. We analyzed data from PSs to understand IoT system quality requirements and architectural styles, assessing their alignment. The research revealed ten essential quality requirements for IoT systems and identified ten distinct architectural styles. Notably, each architectural style varies in its capacity to fulfill specific quality requirements, particularly regarding security, scalability, and performance. SOA, client-server, and REST architectural styles best fulfill many quality requirements. However, various architectural styles, such as Layered, Microservices, and Peer-to-Peer, show limited support for privacy requirements. Our findings can guide IoT systems practitioners in selecting an architectural style that aligns with their desired quality standards. Additionally, we recommend new research opportunities to deepen understanding of key architectural styles based on specific quality requirements.

As the Internet of Things (IoT) grows, its failures may have dramatic consequences on the lives of people who depend on it. Yet, it is hard to test IoT systems before they are deployed. Several researchers have provided state-of-the-art approaches for testing IoT systems. However, many of those approaches are based on academia rather than industry. Therefore, we conducted a multimethod study of IoT systems testing in the industry with IoT practitioners. We used three methods: 1) an industry survey; 2) practitioners interviews; and 3) analysis of Eclipse IoT surveys. This study focuses on testing IoT systems by industry practitioners. The findings show the following. 1) Testing focuses more on the device, network, and application layers. IoT testing gives more importance to integration testing than acceptance testing. Test coverage is the most important metric, but metrics may vary depending on the project. 2) IoT system testing mainly uses the model-based approach and is often manual or semi-automated, with low adoption of white box testing. Node-RED is commonly used in testing IoT systems, while Amazon AWS IoT is popular for cloud platform testing of IoT devices. 3) Log analysis is the main approach to analyzing the root cause of bugs. 4) The main challenges in IoT testing include the lack of standards, security, connectivity, and reference architecture. Generating test cases and establishing a standard test approach are recommended for further research. This study’s findings can help IoT practitioners and researchers to identify and tackle challenges in IoT system testing, leading to future research opportunities.

Internet of Things (IoT) systems are becoming prevalent in various domains, from healthcare to smart homes. Testing IoT systems is critical in ensuring their reliability. Previous papers studied separately the objectives, approaches, tools, and challenges of IoT systems testing. However, despite the rapid evolution of the IoT domain, no review has been undertaken to investigate all four aspects collectively. This paper presents a systematic literature review that aggregates, synthesizes, and discusses the results of 83 primary studies (PSs) concerning IoT testing objectives, approaches, tools, and challenges. We followed the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) protocol to report our findings and answer research questions (RQs). To select PSs, we applied inclusion and exclusion criteria to relevant studies published between 2012 and 2022. We extracted and analyzed the data from PSs to understand IoT systems testing. The results reveal that IoT systems testing embraces traditional software quality attributes but also introduces new ones like connectivity, energy efficiency, device lifespan, distributivity, and dynamicity. They also show that existing IoT systems testing approaches are limited to specific aspects and should be expanded for more comprehensive testing. They also show 19 testing tools and 15 testbeds for testing IoT systems with their limitations, necessitating the development or enhancement for wider coverage. The large number of heterogeneous devices generating data in different formats, along with the need for testing in real-world scenarios, poses a challenge. Thus, our study offers insights into the testing objectives, approaches, tools, and challenges associated with IoT systems. Based on the results, we also provide practical guidance for IoT practitioners by cataloging existing tools and approaches, while also identifying new research opportunities for interested researchers.