Internet of Things (IoT) devices are widely used and pervasive, commonly collecting daily life data and often involving personal information. The sensitivity of the data brings security concerns when handling it. Blockchain ensures decentralization and data security through immutable service records and tamper-proof query execution. The single-chain architecture is constrained in supporting multi-agency IoT environments for isolated data structures and interoperability bottlenecks. On the other hand, cross-chain service queries enable secure interoperability and dynamic resource integration across heterogeneous blockchain systems, but are amenable to privacy breaches, query latency, and consistency in multi-chain environments. To address these challenges, we propose a verifiable and low-latency cross-chain IoT service query scheme with the collaborative indexing. To encapsulate IoT service attributes, we design a service metadata model based on partial homomorphic encryption, allowing service providers to upload services without revealing the private service information. To achieve fine-grained metadata queries, the collaborative indexing mechanism filters irrelevant chains through a global index to narrow the query scope and then utilizes localized sub-indexes, optimizing search efficiency. Finally, we construct data structures based on fingerprint verification to ensure consistency between the query layer and blockchain data. We provide a security analysis of the robustness and privacy, and develop a supporting tool named ServChain. Experimental results demonstrate that, even with consistency verification, our scheme outperforms state-of-the-art solutions by significantly reducing query time (by at least 68.18%), communication overhead (by 45.9%), and computational cost (by 46.5% ).

Service composition has become a crucial mechanism for integrating heterogeneous Internet of Things (IoT) services to meet users' complex and dynamic requirements. In decentralized IoT environments, the absence of a trusted third party presents significant challenges in coordinating service providers. Blockchain offers a promising solution by maintaining tamperproof records to address the inherent multi-party trust problem. However, as IoT services are deployed across diverse platforms, cross-chain service composition introduces new challenges, particularly in establishing trust among multiple stakeholders and ensuring adaptive Quality of Service (QoS) optimization. To address these challenges, we propose a cross-chain IoT service composition approach named CSCIoT, which constructs adaptive QoS constraints while ensuring trustworthiness throughout the entire composition process. Our scheme establishes a Trustaware Practical Byzantine Fault Tolerance (T-PBFT) mechanism to evaluate IoT services using a QoS reputation model. A constraint graph is then constructed to capture dependencies among services, devices, and user requirements, enabling adaptive service selection in response to environmental changes. Smart contracts are further employed for continuous QoS monitoring. We demonstrate the adaptability and effectiveness of CSCIoT through a case study on autonomous vehicle leasing, and show its efficiency and QoS performance on real-world datasets compared to state-of-the-art approaches.

Kubernetes is a popular containerized application orchestration platform that is widely adopted for the development of large-scale service-oriented systems. Such a system often involves the integration of third-party applications, which may introduce security risks, such as excessive permission configurations and privilege escalations, resulting in the leakage of sensitive resources and unauthorized operations. Existing risk detection techniques mainly examine the permission configurations with some predefined rules, which may not be adaptive and precise due to the dynamic nature of Kubernetes environments. To overcome the limitations, we propose a systematic permission-oriented risk detection approach called KubeGuard. First, KubeGuard identifies a minimal permission set and uses it to check whether the existence of excessive permissions in the permission configurations. Second, KubeGuard detects various privilege escalation risks through dynamic rule-based auditing. Third, KubeGuard prevents sensitive resource leakages by monitoring the messages transferred between pods in Kubernetes and alerting in case sensitive resources are involved through the keyword matching. We further developed a supporting prototype called Kube-Guarder. Experiments were conducted on a suite of open-source Kubernetes applications and simulated scenarios to evaluate the effectiveness of KubeGuard. Experimental results have shown that KubeGuard can detect excessive permissions more effectively and precisely compared with the static rule-based baseline technique, and in the meanwhile, KubeGuard can detect various types of privilege escalation and sensitive resource leakage. As a result, this study delivered a promising technique for improving the security of Kubernetes applications.

The abstract syntax tree differencing (ASTDiff) technique aims to capture syntactic code changes through comparing the differences between a pair of ASTs of a program, which has been widely used in various program analysis or testing tasks, such as code review, clone detection, and regression testing. A key issue for ASTDiff lies in the accurate mappings between nodes of two ASTs. However, most existing approaches often fail to generate such perfect diffs due to the gap between diverse code changes and unsound node matching heuristics. Our in-depth investigation reveals that most inaccurate mappings are caused by the ignorance of context- and/or type-specific constraints. Accordingly, we propose an AST diff fixing approach DiffFix that leverages both the node’s context and type constraints to iteratively and incrementally fix imperfect diffs. Comprehensive experiments have been conducted to evaluate the effectiveness of DiffFix through its application to fix diffs generated by five state-of-the-art ASTDiff techniques. The experimental results demonstrate that DiffFix can improve the perfect diff rate of these baseline techniques by 5.25% to 51.12% with negligible time overhead.

Software testing is a mainstream approach for software quality assurance. One fundamental challenge for testing is that in many practical situations, it is very difficult to verify the correctness of test results given inputs for Software Under Test (SUT), which is known as the oracle problem. Metamorphic Testing (MT) is a software testing technique that can effectively alleviate the oracle problem. The core component of MT is a set of Metamorphic Relations (MRs), which are basically the necessary properties of SUT, represented in the form of relationship among multiple inputs and their corresponding expected outputs. Different methods have been proposed to support the systematic MR identification. However, most of them still rely heavily on test engineers' understanding of the SUT and involve massive manual work. Although a few preliminary studies have shown LLMs' viability in generating MRs, there does not exist a thorough and in-depth investigation on their capability in MR identification. We are thus motivated to conduct a comprehensive and large-scale empirical study to systematically evaluate the performance of LLMs in identifying appropriate MRs for a wide variety of software systems. This study makes use of 37 SUTs collected from previous MT studies. Prompts are constructed for two LLMs, gpt-3.5-turbo-1106 and gpt-4-1106-preview, to perform the MR identification for each SUT. The empirical results demonstrate that both LLMs can generate a large amount of MR candidates (MRCs). Among them, 29.86% and 43.79% of all MRCs are identified as the MRs valid for the corresponding SUT, respectively. In addition, 24.59% and 38.63% of all MRCs are MRs that had never been identified in previous studies. Our study not only reinforces LLM-based MR identification as a promising research direction for MT, but also provides some practical guidelines for how to further improve LLMs' performance in generating good MRs.

Metamorphic testing has become one mainstream technique to address the notorious oracle problem in software testing, thanks to its great successes in revealing real-life bugs in a wide variety of software systems. Metamorphic relations, the core component of metamorphic testing, have continuously attracted research interests from both academia and industry. In the last decade, a rapidly increasing number of studies have been conducted to systematically generate metamorphic relations from various sources and for different application domains. In this article, based on the systematic review on the state of the art for metamorphic relations’ generation, we summarize and highlight visions for further advancing the theory and techniques for identifying and constructing metamorphic relations and discuss promising research directions in related areas.

In recent years, deep neural networks have been applied in machine translation systems, resulting in the so-called neural machine translation (NMT) models that can improve translation quality significantly. However, due to the brittleness of deep neural network, machine translation systems could return erroneous translations that lead to misunderstandings or even cause serious losses. To detect translation errors, various testing techniques have been proposed. As a popularly used technique, metamorphic testing mainly relies on text or syntactic structure of translations while ignoring the meaning of sentences (i.e., semantic information). Compared with text and syntactic information, semantic information of sentences is more stable when dealing with languages that have rich vocabulary and flexible word order. Motivated by this observation, we propose semantic structure invariance-based metamorphic testing (SSIMT) for machine translation systems. The key insight is that contextually similar sentences should typically have translations of similar semantic structures. Experiments have been conducted to evaluate SSIMT on two widely used machine translation systems, Microsoft Bing Translator and Google Translate with 600 seed sentences crawled from well-known news websites covering six different corpus topics. The experimental results show that SSIMT is able to find thousands of erroneous translations in both translation systems with high accuracy (over 70%). Translation errors reported by SSIMT covers a wide variety of common error types.

Smart contracts are essential applications for blockchains, which have been used in a wide variety of fields and are handling large amounts of valuable assets. Once deployed on the blockchain network, smart contracts cannot be altered, thus making the pre-deployment testing of them extremely critical. Nevertheless, the features of smart contracts, especially their transaction-driven nature, pose huge challenges to testing. In particular, since their inputs are not static data but dynamic sequences of user behavior, it is very difficult to obtain a feasible oracle for testing, which refers to the systematic mechanism to verify the correctness of test results given any test input. This is the notorious oracle problem in the context of software testing, for which the metamorphic testing technique has been widely recognized as a simple yet effective solution. In this paper, we develop a comprehensive framework, namely MT4SC, for implementing metamorphic testing on smart contracts. Specifically, we propose a systematic way to construct metamorphic relations, the core component of metamorphic testing, based on the user behavior sequences. A series of experiments have been conducted to evaluate the performance of MT4SC on eight different smart contract scenarios. The experimental results demonstrate MT4SC’s high effectiveness in detecting potential faults in smart contracts, even without the need for test oracles. This study bolsters the research on the testing of smart contracts, thereby improving their quality and ultimately advancing the reliability of blockchains.

There are long-term concerns about the risk of privacy leakage when data is acquired and transmitted via Internet of Things (IoT) devices, which too often hinders the development of data-driven applications and services. To tackle such obstacle, we present a novel blockchain-based data service provisioning architecture based on federated learning, named BP-DSP. The system enables data owners to contribute to data-driven services without exposing their private data. To enhance the model accuracy and reduce the communication overhead, we build a data preference and data summary model, by which the model selects data providers whose data preferences align with those of the customers, serving as participating nodes in federated learning training. By leveraging lightweight homomorphic encryption algorithms, BP-DSP encrypts the model gradients to prevent the potential privacy leakage of data owners’ sensitive information from the inference attack. We construct a reputation-based incentive mechanism to motivate computational nodes to perform model parameter validation and punish participating nodes that upload false parameters or low-quality models in order to constrain their self-interest. Through a detailed security analysis, we show that parameter and model privacy are well preserved during the whole process. Finally, we conduct extensive experiments on real-world datasets to demonstrate the efficiency and reliability of BP-DSP, that the BP-DSP speeds up operational efficiency by over 40%.

Abstract Distributed systems have been widely used in many safety‐critical areas. Any abnormalities (e.g., service interruption or service quality degradation) could lead to application crashes or decrease user satisfaction. These things may cause serious economic losses. Among the various quality assurance approaches for distributed systems, log‐based anomaly detection (LAD) has become a popular research topic. Its popularity relates to system logs being able to record and reveal important run‐time information. This paper presents a general LAD framework for distributed systems. Log grouping and feature‐pattern mining are two crucial LAD components that impact on the anomaly‐detection effectiveness. We also present a systematic survey of techniques in these two directions; propose classification frameworks for log grouping and feature patterns; and summarize four log‐grouping techniques and five feature patterns (which refer to invariant relationships among logs that can be used for anomaly detection). To evaluate their applicability, we report on the findings when applying existing techniques to Ray, a popular industrial distributed system. Based on these findings, several open issues are identified, which provide potential guidance for future research and development.

Summary Metamorphic testing (MT) is an effective technique to alleviate the test oracle problem. The principle of MT is to detect failures by checking whether some necessary properties, commonly known as metamorphic relations (MRs), of software under test (SUT) hold among multiple executions of source and follow‐up test cases. Since both the generation of follow‐up test cases and test result verification depend on MRs, the identification of MRs plays a key role in MT, which is an important yet difficult task requiring deep domain knowledge of the SUT. Accordingly, techniques that can direct a tester to identify MRs effectively are desirable. In this paper, we propose MT, a data mutation directed approach to identifying MRs. MT guides a tester to identify MRs by providing a set of data mutation operators and template‐style mapping rules, which not only alleviates the difficulties faced in the process of MR identification but also improves the identification effectiveness. We have further developed a tool to implement the proposed approach and conducted an empirical study to evaluate the MR identification effectiveness of MT and the performance of MRs identified by MT with respect to fault detection capability and statement coverage. The empirical results show that MT is able to identify MRs for numeric programs effectively, and the identified MRs have high fault detection capability and statement coverage. The work presented in this paper advances the field of MT by providing a simple yet practical approach to the MR identification problem.

Concurrent programs are normally composed of multiple concurrent threads sharing memory space. These threads are often interleaved, which may lead to some non-determinism in execution results, even for the same program input. This poses huge challenges to the testing of concurrent programs, especially on the test result verification—that is, the prevalent existence of the oracle problem. In this article, we investigate the application of metamorphic testing (MT), a mainstream technique to address the oracle problem, into the testing of concurrent programs. Based on the unique features of interleaved executions in concurrent programming, we propose an extended notion of metamorphic relations, the core part of MT, which are particularly designed for the testing of concurrent programs. A comprehensive testing approach, namely ConMT , is thus developed and a tool is built to automate its implementation on concurrent programs written in Java. Empirical studies have been conducted to evaluate the performance of ConMT, and the experimental results show that in addition to addressing the oracle problem, ConMT outperforms the baseline traditional testing techniques with respect to a higher degree of automation, better bug detection capability, and shorter testing time. It is clear that ConMT can significantly improve the cost-effectiveness for the testing of concurrent programs and thus advances the state of the art in the field. The study also brings novelty into MT, hence promoting the fundamental research of software testing.

Background. Software testing is a critical activity for ensuring the quality and reliability of software systems. To evaluate the effectiveness of different test suites, researchers have developed a variety of metrics. Problem. However, comparing these metrics is challenging due to the lack of a standardized evaluation framework including comprehensive factors. As a result, researchers often focus on single factors (e.g., size), which finally leads to different or even contradictory conclusions. After comparing dozens of pieces of work in detail, we have found two main problems most troubling to our community: (1) researchers tend to oversimplify the description of the ground truth they use, and (2) data involving real defects is not suitable for analysis using traditional statistical indicators. Objective. We aim at scrutinizing the whole process of comparing test suites for our community. Method. To hit this aim, we propose a framework ASSENT (ev A luating te S t S uite E ffective N ess me T rics) to guide the follow-up research for evaluating a test suite effectiveness metric. ASSENT consists of three fundamental components: ground truth, benchmark test suites, and agreement indicator. Its functioning is as follows: first, users clarify the ground truth for determining the real order in effectiveness among test suites. Second, users generate a set of benchmark test suites and derive their ground truth order in effectiveness. Third, users use the metric to derive the order in effectiveness for the same test suites. Finally, users calculate the agreement indicator between the two orders derived by two metrics. Result. With ASSENT, we are able to compare the accuracy of different test suite effectiveness metrics. We apply ASSENT to evaluate representative test suite effectiveness metrics, including mutation score and code coverage metrics. Our results show that, based on the real faults, mutation score, and subsuming mutation score are the best metrics to quantify test suite effectiveness. Meanwhile, by using mutants instead of real faults, test effectiveness will be overestimated by more than 20% in values. Conclusion. We recommend that the standardized evaluation framework ASSENT should be used for evaluating and comparing test effectiveness metrics in the future work.

In light of high-performance computer processing, massive datasets, and mighty algorithms, we are rapidly entering an age where the advanced deep learning (DL) capabilities are integrated into the contemporary software systems to fulfill critical tasks. Like “traditional” software, DL systems are not immune to faults, some of which may even cause catastrophic disasters. As a mainstream testing technique for DL systems, fuzzing attempts to generate a large amount of semirandom yet syntactically valid test cases, from which the so-called adversarial inputs can be found, indicating the detection of faults. Test cases in fuzzing are generated based on a seed queue, which is constructed by randomly selecting seeds from the existing test suite (that is, the set of test cases). In this article, we propose a diversity-driven approach, namely DFuzzer, for constructing seed queues in fuzzing. We particularly develop two algorithms, namely DFuzzer-IB and DFuzzer-FB, based on the information theory and deep features, respectively, to improve the diversity of seed queues. Experimental studies have been conducted to evaluate the proposed techniques based on five fuzzers, three datasets, and seven DL models. The experimental results show that both strategies can significantly improve the performance of the state-of-the-art fuzzers for DL, including DeepXplore, DLFuzz, Tensorfuzz, DeepHunter, and DeepSmartFuzzer, not only in terms of finding more adversarial inputs for triggering faults but also achieving higher coverage. Our article demonstrates that the improved diversity of seed queues and the resultant test cases can help achieve a high testing effectiveness of fuzzing.

Microservice architecture (MSA) has been widely adopted to develop various large-scale distributed systems. Microservice-based systems (MBSs) comprise a number of independently deployed microservices fulfilling the specific functionalities. Unique characteristics of microservices, such as independent and parallel development, rapid iteration, and distributed deployment, result in low observability and reliability of MBSs. A typical solution is to regulate system behavior in specifications of MBSs, and then develop and test MBSs based on these specifications. However, current microservice specifications focus on describing the APIs of microservices without describing the behavior expectation for an MBS. In this article, we propose an annotation-assisted and scenario-oriented approach, called MSA_Sighter, to detect behavior inconsistencies in MBSs. In MSA_Sighter, the details of an MBS are captured in a description model (MSDM), which can be extracted automatically from the functional services through annotation-assisted runtime component instance analysis and static program analysis. Given a specific business scenario, inconsistency detection is conducted by analyzing the actual behavior's conformance to the expected behavior, where the former is collected through distributed tracing while the latter is derived from the MSDM. We have developed a supporting tool called ConsChecker and evaluated MSA_Sighter's effectiveness on three open-source MBSs in GitHub. The experimental results have shown that MSA_Sighter can effectively detect inconsistencies in MBSs during system development and evolution.

Microservice architecture has been widely used for the development of large-scale distributed applications. Microservice systems normally have high complexity and loose coupling nature, which make it challenging to localize faults in them. Automated fault localization is particularly difficult for microservice systems, due to their unique features, such as frequent updates, complex dependencies, and multiple microservice instances. In this paper, we propose a fault localization approach for microservice systems based on trace log clusterings, called TLCluster. TLCluster first derives trace logs by collecting and combining communication messages and logs of microservice systems, then clusters trace logs for different business process categories, calculates similarities between normal and abnormal trace logs, and finally evaluates and ranks the suspiciousness scores of microservice instances. We conducted a series of experiments to evaluate the effectiveness of TLCluster using a large-scale microservice system. Experimental results show that our approach is able to effectively localize faults of microservice systems and demonstrates a better fault localization accuracy and precision compared with state-of-the-art baseline techniques.

Over the past decade, metamorphic testing has gained rapidly increasing attention from both academia and industry, particularly thanks to its high efficacy on revealing real-life software faults in a wide variety of application domains. On the basis of a set of metamorphic relations among multiple software inputs and their expected outputs, metamorphic testing not only provides a test case generation strategy by constructing new (or follow-up) test cases from some original (or source) test cases, but also a test result verification mechanism through checking the relationship between the outputs of source and follow-up test cases. Many efforts have been made to further improve the cost-effectiveness of metamorphic testing from different perspectives. Some studies attempted to identify “good” metamorphic relations, while other studies were focused on applying effective test case generation strategies especially for source test cases. In this article, we propose improving the cost-effectiveness of metamorphic testing by leveraging the feedback information obtained in the test execution process. Consequently, we develop a new approach, namely feedback-directed metamorphic testing, which makes use of test execution information to dynamically adjust the selection of metamorphic relations and selection of source test cases. We conduct an empirical study to evaluate the proposed approach based on four laboratory programs, one GNU program, and one industry program. The empirical results show that feedback-directed metamorphic testing can use fewer test cases and take less time than the traditional metamorphic testing for detecting the same number of faults. It is clearly demonstrated that the use of feedback information about test execution does help enhance the cost-effectiveness of metamorphic testing. Our work provides a new perspective to improve the efficacy and applicability of metamorphic testing as well as many other software testing techniques.

Web services have been widely used to develop complex distributed software systems in the context of Service Oriented Architecture (SOA). As a standard for describing Web services, the Web Service Description Language (WSDL) provides a universal mechanism to describe the service’s functionalities for the service consumers. However, the current WSDL only provides the description of the interfaces to a Web Service without any restrictions or assumptions on how to properly invoke the service, resulting in divergent understanding of the Web service’s behavior between the service developer and service consumer. A particular challenge is how to make explicit the various behavior assumptions and restrictions of a service (for the user), and make sure that the service implementation conforms to them (for the developer). In this article, we propose a constraint-based model-driven approach to improving the behavior conformance of Web services. In our approach, constraints are introduced in an extended WSDL, called CxWSDL, to formally and explicitly express the implicit restrictions and assumptions on the behavior of a Web service, and then the predefined constraints are used to derive test cases in a model-driven manner to test the service implementation’s conformance to its behavior constraints from the user’s perspective. An empirical study involving four real-life Web services was conducted to evaluate the effectiveness of our approach, and four actual inconsistencies were discovered.

The advantages of Service-Oriented Architecture (SOA) combined with the emerging and diffusion of the Internet of Things (IoT) instances have given birth to a new paradigm for IoT components integration, i.e., Service-Oriented IoT. Microservices especially have been widely used to deliver IoT services due to their lightweight implementation and distributed nature. With the continuous increase in IoT services available on the Internet, the selection of services becomes difficult. Furthermore, IoT services are often featured with rich contexts and OpenAPI descriptions, which impede service recommendation approaches that are designed for WSDL-based Web services or mashup services. To address this challenging issue, we propose a context-aware IoT service recommendation approach called DFORM for proactive service provision. DFORM considers both functional features of OpenAPI descriptions and contextual features of IoT environments, and leverages a deep collaborative filtering-based recommendation model to learn the feature representations and capture the interactions between users and services. We conduct a series of experiments to evaluate the recommendation performance of DFORM and the experimental results show that DFORM is effective in IoT service recommendation and outperforms state-of-the-art techniques.

Deep learning (DL) systems are increasingly adopted in various fields, while fatal failures are still inevitable in them.One mainstream testing approach for DL is fuzzing, which can generate a large amount of semi-random yet syntactically valid test cases.Previous studies on fuzzing are mainly focused on selecting "quality" seeds or using "good" mutation strategies.In this paper, we attempt to improve the performance of fuzzing from a different perspective.A new fuzzer, namely DeepController, is accordingly developed, which makes use of the feedback information obtained in the test execution process to dynamically select seeds and mutation strategies.DeepController is evaluated through empirical studies on three datasets and eight DL models.The experimental results show that, with the same number of seeds, DeepController can generate more adversarial inputs and achieve higher neuron coverage than the state-of-the-art testing techniques for DL systems.

[J1C2 Presentation Abstract at IEEE SERVICES 2022 for IEEE Transactions on Services Computing, 2022, 15(2):736-751. DOI: 10.1109/TSC.2019.2960496].

Mutation testing is basically a fault-based software testing technique, which has been proposed to measure the fault detection effectiveness of a test suite using programs with simulated faults (namely mutants). However, mutation testing is time consuming and computationally expensive because of the normal use of a large amount of mutants. Thus, reducing the mutants is of great significance. To address this problem, various mutant reduction techniques have been proposed. Among them, the identification of redundant mutants aims at removing mutants whose test results can be inferred by other mutants. This article proposes a redundant mutant identification technique based on selective symbolic execution called ReMuSSE for weak mutation testing. Redundant mutants could be revealed by identifying those with similar program execution state changes within a program block involving mutated statements. An empirical study was conducted using 13 C programs from different application domains with varying sizes. The empirical results showed that ReMuSSE could identify up to 31.4% redundant mutants and consequentially save up to 35.2% time cost of weak mutation testing. The results demonstrated that ReMuSSE could effectively identify redundant mutants and thus could significantly improve the efficiency of weak mutation testing.

In recent years, service oriented architecture (SOA) has been increasingly adopted to develop distributed applications in the context of the Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of web services. In this article, we propose a dynamic random testing (DRT) technique for web services, which is an improvement over the widely-practiced random testing (RT) and partition testing (PT) approaches. We examine key issues when adapting DRT to the context of SOA, including a framework, guidelines for parameter settings, and a prototype for such an adaptation. Empirical studies are reported where DRT is used to test three real-life web services, and mutation analysis is employed to measure the effectiveness. Our experimental results show that, compared with the three baseline techniques, RT, Adaptive Testing (AT) and Random Partition Testing (RPT), DRT demonstrates higher fault-detection effectiveness with a lower test case selection overhead. Furthermore, the theoretical guidelines of parameter setting for DRT are confirmed to be effective. The proposed DRT and the prototype provide an effective and efficient approach for testing web services.

A microservice-based system is composed of numerous independently deployed and executed microservices, among which normally exist the complex dependencies. Traditional service composition approaches usually expect the business process predefined at design time. As a result, it is difficult for the microservice-based system to quickly adapt to the frequently changing operation environments and business requirements. To address the above limitations, we propose a variability-enabling and model-driven approach to developing adaptive microservice-based systems. Our approach first models the business process with variability using VxBPMN4MS, an extension of Business Process Model and Notation (BPMN) with support for variability, then transforms the business process model to variability supported microservice composition frameworks, and finally derives business process instances at run-time according to the predefined process configuration. We have developed a platform to automate the proposed approach as much as possible, and conducted a case study to evaluate the effectiveness of the proposed approach and platform.

Service-Oriented Architecture (SOA) provides a scalable framework for heterogeneous, pervasive Internet of Things (IoT) devices. For such open and dynamic infrastructure, a proper service description schema is fundamental to unify the self-description of services. However, existing approaches mainly rely on verbose service protocols which produce unnecessary service overhead for resource limited devices. In addition, these approaches usually focus on describing service functions while largely ignoring context characteristics of IoT services. A lightweight and context-aware service description approach is yet to be proposed. In this paper, we propose a context-aware IoT service description approach based on the microservice architecture. Our approach not only provides a service description schema with lightweight protocols, but also comprehensively describes the context, service, and interface characteristics of IoT services. We demonstrate the applicability and effectiveness of our approach for the case of a smart elderly care system. Furthermore, we provide a comparison with related approaches to elaborate on pros and cons of our proposal with respect to the state of the art in the field.

Metamorphic testing is well known for its ability to alleviate the oracle problem in software testing. The main idea ofmetamorphic testing is to test a software system by checking whether each identified metamorphic relation (MR) holds among severalexecutions. In this regard, identifying MRs is an essential task in metamorphic testing. In view of the importance of this identificationtask, METRIC (METamorphic Relation Identification based on Category-choice framework) was developed to help software testersidentify MRs from a given set of complete test frames. However, during MR identification, METRIC primarily focuses on the inputdomain without sufficient attention given to the output domain, thereby hindering the effectiveness of METRIC. Inspired by this problem,we have extended METRIC into METRIC+by incorporating the information derived from the output domain for MR identification. A toolimplementing METRIC+has also been developed. Two rounds of experiments, involving four real-life specifications, have beenconducted to evaluate the effectiveness and efficiency of METRIC+. The results have confirmed that METRIC+is highly effective andefficient in MR identification. Additional experiments have been performed to compare the fault detection capability of the MRsgenerated by METRIC+and those bymMT (another MR identification technique). The comparison results have confirmed that the MRsgenerated by METRIC+are highly effective in fault detection.

Random testing and partition testing are two major families of software testing techniques. They have been compared both theoretically and empirically in numerous studies for decades, and it has been widely acknowledged that they have their own advantages and disadvantages and that their innate characteristics are fairly complementary to each other. Some work has been conducted to develop advanced testing techniques through the integration of random testing and partition testing, attempting to preserve the advantages of both while minimizing their disadvantages. In this paper, we propose a new testing approach, adaptive partition testing, where test cases are randomly selected from some partition whose probability of being selected is adaptively adjusted along the testing process. We particularly develop two algorithms, Markov-chain based adaptive partition testing and reward-punishment based adaptive partition testing, to implement the proposed approach. The former algorithm makes use of Markov matrix to dynamically adjust the probability of a partition to be selected for conducting tests; while the latter is based on a reward and punishment mechanism. We conduct empirical studies to evaluate the performance of the proposed algorithms using ten faulty versions of three large-scale open source programs. Our experimental results show that, compared with two baseline techniques, namely random partition testing (RPT) and dynamic random testing (DRT), our algorithms deliver higher fault-detection effectiveness with lower test case selection overhead. It is demonstrated that the proposed adaptive partition testing is an effective testing approach, taking advantages of both random testing and partition testing.

Mutation analysis is a classical software testing approach which attempts to imitate faults using a set of mutants. It has been advocated to be an appropriate technique for evaluating the quality of test suites as well as the effectiveness of a testing method. However, the applicability of mutation analysis, especially in many practical situations, has been hindered due to the high computation cost and the long execution time, which are mainly caused by the large number of mutants. Numerous studies, particularly those based on parallel computing, have been conducted to reduce the overhead of mutation analysis. In this paper, we aim to improve the efficiency of mutation analysis from a different perspective. We make use of lightweight program analysis techniques to identify a group of mutants that share the common execution traces before the mutation location, and then merge them into a synthesized program with the concurrent mechanism, on which mutation analysis can be efficiently executed without the duplicate execution of common traces. Our empirical study demonstrates that our approach can significantly decrease the computation overhead as well as shorten the execution time of mutation analysis, without jeopardizing its effectiveness. The in-depth analysis further shows that the effectiveness of our approach is positively correlated with the number of branches in the program under test. Our approach makes it possible to efficiently execute mutation analysis even without the need of advanced computer architectures.

Nowadays, service oriented architecture (SOA) has become one mainstream paradigm for developing distributed applications. As the basic unit in SOA, web services can be composed to construct complex applications. The quality of web services and their compositions is critical to the success of SOA applications. Testing, as a major quality assurance technique, is confronted with new challenges in the context of service compositions. In this paper, we propose a scenario-oriented testing approach that can automatically generate test cases for service compositions. Our approach is particularly focused on the service compositions specified by Business Process Execution Language for web services (WS-BPEL), a widely recognized executable service composition language. In the approach, a WS-BPEL service composition is first abstracted into a graph model; test scenarios are then derived from the model; finally, test cases are generated according to different scenarios. We also developed a prototype tool implementing the proposed approach, and an empirical study was conducted to demonstrate the applicability and effectiveness of our approach. The experimental results show that the automatic scenario-oriented testing approach is effective in detecting many types of faults seeded in the service compositions.

The publish/subscribe paradigm can be used to build IoT service communication infrastructure owing to its loose coupling and scalability. Its features of decoupling among event producers and event consumers make IoT services collaborations more real-time and flexible, and allow indirect, anonymous and multicast IoT service interactions. However, in this environment, the IoT service cannot directly control the access to the events. This paper proposes a cross-layer security solution to address the above issues. The design principle of our security solution is to embed security policies into events as well as allow the network to route events according to publishers' policies and requirements. This solution helps to improve the system's performance, while keeping features of IoT service interactions and minimizing the event visibility at the same time. Experimental results show that our approach is effective.

Nowadays, applications are increasingly deployed as Web services in the globally distributed cloud computing environment. Multiple services are normally composed to fulfill complex functionalities. Business Process Execution Language for Web Services (WS-BPEL) is an XML-based service composition language that is used to define a complex business process by orchestrating multiple services. Compared with traditional applications, WS-BPEL programs pose many new challenges to the quality assurance, especially testing, of service compositions. A number of techniques have been proposed for testing WS-BPEL programs, but only a few studies have been conducted to systematically evaluate the effectiveness of these techniques. Mutation testing has been widely acknowledged as not only a testing method in its own right but also a popular technique for measuring the fault-detection effectiveness of other testing methods. Several previous studies have proposed a family of mutation operators for generating mutants by seeding various faults into WS-BPEL programs. In this study, we conduct a series of empirical studies to evaluate the applicability and effectiveness of various mutation operators for WS-BPEL programs. The experimental results provide insightful and comprehensive guidance for mutation testing of WS-BPEL programs in practice. In particular, our work is the systematic study in the selection of effective mutation operators specifically for WS-BPEL programs.

Unified Modeling Language (UML) activity diagrams are widely used to model concurrent interaction among multiple objects. In this paper, we propose a transformation-based approach to generating scenario-oriented test cases for applications modeled by UML activity diagrams. Using a set of transformation rules, the proposed approach first transforms a UML activity diagram specification into an intermediate representation, from which it then constructs test scenarios with respect to the given concurrency coverage criteria. The approach then finally derives a set of test cases for the constructed test scenarios. The approach resolves the difficulties associated with fork and join concurrency in the UML activity diagram and enables control over the number of the resulting test cases. We further implemented a tool to automate the proposed approach and studied its feasibility and effectiveness using a case study. Experimental results show that the approach can generate test cases on demand to satisfy a given concurrency coverage criterion and can detect up to 76.5% of seeded faults when a weak coverage criterion is used. With the approach, testers can not only schedule the software test process earlier, but can also better allocate the testing resources for testing concurrent applications. Copyright © 2015 John Wiley & Sons, Ltd.

In cloud computing, variability becomes a crucial ability of process-based cloud applications. Existing solutions either focused on modeling variability in the architectural model or tried to support dynamic variability management in implementation. An integrated approach that can inherit the virtues from both categories is expected. The paper aims to fill the gap by proposing a model-driven variability-based service composition approach. We propose VxUML to model the variability in the architecture model. Then, we define a set of model transformation rules to convert VxUML into VxBPEL (an extension to standard BPEL supporting variability at the implementation level). Finally, we implement a prototype tool, and present a case study to demonstrate the feasibility of our approach.

A great amount of fault-based testing strategies have been proposed to generate test cases for detecting certain types of faults in Boolean specifications. However, most of the previous studies on these strategies were focused on the Boolean expressions in the disjunctive normal form (DNF), even the irredundant DNF (IDNF)—little work has been conducted to comprehensively investigate their performance on general Boolean specifications. In this study, we conducted a series of experiments to evaluate and compare 18 fault-based testing strategies using over 4000 randomly generated fault-seeded Boolean expressions. In the experiments, a testing strategy is regarded as effective and efficient if it can detect most of the seeded faults using a small number of test cases. Our experimental results show that if a testing strategy is highly effective and efficient when testing the Boolean expressions in the IDNF, it also shows high effectiveness and efficiency on general Boolean expressions. It is found that one family of fault-based testing strategies, namely MUMCUT, normally deliver the best performance among all the 18 strategies. Our study provides an in-depth understanding and insight of fault-based testing for general Boolean expressions.

The concept of software as a service has been increasingly adopted to develop distributed applications. Ensuring the reliability of loosely coupled compositions is a challenging task because of the open, dynamic, and independent nature of composable services; this is especially true when the execution of a service-based process relies on independent but correlated services. Transactions are the prototypical case of compositions spanning across multiple services and needing properties to be valid throughout the whole execution. Although transaction protocols and service composition languages have been proposed in the past decade, a true viable and effective solution is still missing. In this article, we propose a systematic aspect-based approach to integrating transactions into service compositions, taking into account the well-known protocols: Web Service Transaction and Business Process Execution Language (BPEL). In our approach, transaction policies are first defined as a set of aspects. They are then converted to standard BPEL elements. Finally, these transaction-related elements and the original BPEL process are weaved together, resulting in a transactional executable BPEL process. At runtime, transaction management is the responsibility of a middleware, which implements the coordination framework and transaction protocols followed by the transactional BPEL process and transaction-aware Web services. To automate the proposed approach, we developed a supporting platform called Salan to aid the tasks of defining, validating, and weaving aspect-based transaction policies, and of deploying the transactional BPEL processes. By means of a case study, we demonstrate the proposed approach and evaluate the performance of the supporting platform. Experimental results show that this approach is effective in producing reliable business processes while reducing the need for direct human involvement.

In recent years, Service Oriented Architecture (SOA) has been increasingly adopted to develop applications in the context of Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of Web services. In this paper, we propose a dynamic random testing (DRT) technique for Web services which is an improvement of the widely practiced random testing. We examine key issues when adapting DRT to the context of SOA and develop a prototype for such an adaptation. Empirical studies are reported where DRT is used to test two real-life Web services and mutation analysis is employed to measure the effectiveness. The experimental results show that DRT can save up to 24% test cases in terms of detecting the first seeded fault, and up to 21% test cases in terms of detecting all seeded faults, both with the cases of uniformed mutation analysis and distribution-aware mutation analysis, which refer to faults being seeded in an even or clustered way, respectively. The proposed DRT and the prototype provide an effective approach to testing Web Services.

Service Oriented Architecture (SOA) has become a major application development paradigm. As a basic unit of SOA applications, Web services significantly affect the quality of the applications constructed from them. Since the development and consumption of Web services are completely separated under SOA environment, the consumers are normally provided with limited knowledge of the services and thus have little information about test oracles. The lack of source code and the restricted control of Web services limit the testability of Web services. To address the prominent oracle problem when testing Web services, we propose a metamorphic testing framework for Web services taking into account the unique features of SOA. We conduct a case study where the new metamorphic testing framework is employed to test a Web service that implements the electronic payment. The results of case study show the feasibility of the framework for web services, and also the efficiency of metamorphic testing. The work presented in the paper alleviates the test oracle problem when testing Web services under SOA.

Service-Oriented Computing (SOC) is becoming the mainstream development paradigm of applications over the Internet, taking advantage of remote independent functionalities. The cornerstone of SOC's success lies in the potential advantage of composing services on the fly. When the control over the communication and the elements of the information system is low, developing solid systems is challenging. In particular, developing reliable web service compositions usually requires the integration of both composition languages, such as the Business Process Execution Language (BPEL), and of coordination protocols, such as WS-AtomicTransaction and WS-BusinessActivity. Unfortunately, the composition and coordination of web services currently have separate languages and specifications. The goal of this paper is twofold. First, we identify the major requirements of transaction management in Service-oriented systems and survey the relevant standards. Second, we propose a semiautomatic approach to integrate BPEL specifications and web service coordination protocols, that is, implementing transaction management within service composition processes, and thus overcoming the limitations of current technologies.

Nowadays applications are increasingly developed based on remote Web services and service composition has become a powerful novel development paradigm. Due to the fact that such applications in the context of Internet are deployed and executed in an open and dynamic environment, adaptability is one of the crucial requirements for developing such applications. In our previous work, we presented a variability-based approach to address the adaptability issue of service compositions, and developed the VxBPEL, an extension of BPEL with respect to variability. This paper presents a variability analysis tool for variability-based adaptive service compositions called ValySeC. ValySeC extracts the variation from service compositions specified using VxBPEL and provides a variation view. With the view, the designer can better understand variation points and the possible instances, and efficiently maintain the variation within service compositions. The paper presents the design and implementation of ValySeC using a case study to handle service compositions with variations to illustrate the main concepts.

Testing concurrent applications is difficult yet important. UML activity diagrams are widely used to model concurrent interactions among multiple objects. We present a transformation-based approach to generating scenario-oriented test cases for testing concurrent applications modeled by UML activity diagrams. The approach first transforms a UML activity diagram specification into an intermediate representation via a set of transformation rules. From the intermediate representation we then construct a set of test scenarios with respect to the given concurrence coverage criteria. Finally, we derive a set of test cases from the constructed test scenarios. The approach employs transformation to resolve the nonstructural problem with activity diagrams, and can generate test cases on demand to satisfy a given concurrence coverage criteria and hence the number of the resulting test cases is controllable. With the approach, testers can not only earlier schedule the software test process but also better allocate the test resource for testing concurrent applications.

As service centric (SC) systems are being increasingly adopted, new challenges and possibilities emerge. Business processes are now able to execute seamlessly across organizations and to coordinate the interaction of loosely coupled services. Often it is necessary to have transactionality for a set of business operations, but the loosely nature of such systems calls for techniques and principles that go beyond traditional ACID transactions. By analyzing existing service composition languages, tools, and needs on a classical example, we provide requirements for transactionality in service centric systems and indications for developing SC systems transactionally capable.

It is a desirable capability for some types of software-intensive systems to be able to modify components or architecture at the run-time. In this paper, we investigate how to enable the run-time modification of components in a system originally developed without catering for this feature at its design phase. In our proposed approach, components are incrementally and optionally wrapped with two categories of reflection interfaces, namely introspection (retrospection) interfaces that are designed to expose the component properties, while intercession (reconfiguration) interfaces that are designed to conduct run-time changes. We implement the incremental reflection interfaces as a library by leveraging the idea of poke technique that is widely used in many practical accessibility projects. A case study is used to demonstrate how our approach can enhance components with the reflection capacity on the basis of the legacy program code. Compared with existing techniques, our approach does not require systems re-developed from scratch in order to support the run-time modification of components.