We propose Escapin, a JavaScript transpiler for developing application programs that consume APIs and are deployed on cloud services in order to obtain new business concepts by trial-and-error iterations. Escapin tackles two difficulties in consuming API and using cloud services; 1) understanding the correct usage of cloud services and APIs requires a lot of time, and 2) programs using cloud service SDKs and HTTP clients are complicated. Escapin introduces two new features to the ECMAScript 2018 specification for simplifying application programs: importing APIs as objects and exporting objects and functions as cloud service resources. Escapin transpiles a JavaScript program independent from any platform into a JavaScript program containing concrete usages of APIs and cloud services. Escapin encourages engineers to write a callback-agnostic program, which represents asynchronous processing as a completely synchronized procedure without any asynchronization feature such as callback, Promise, async/await, etc. Escapin also accepts programs having nesting callbacks. Escapin destructures nesting callbacks in the programs to obtain a callback-agnostic program. Our experimental results show that in the task of developing an application that consumes Uber API and is deployed on Amazon Web Services, Escapin statistically significantly reduces development time and the metrics regarding the simplicity of source code with huge effects: lines of code, cyclomatic complexity, and cognitive complexity.

As Representational State Transfer (REST)-based Web APIs are spreading to enterprise information systems, software development for the use and provision of Web APIs is rapidly increasing. The quality of the Web APIs significantly influences the application quality and development productivity. However, no quality model for Web APIs has been established yet, because Web APIs differ from conventional APIs in that they execute remotely on different servers and may be changed independently of their users. These unique characteristics introduce new problems in the software engineering of Web APIs, and impose risks to the users, especially those using enterprise Web APIs, whose numbers are increasing. To solve these problems, in this paper, we propose a quality model for Web APIs that reflects their unique characteristics. As the main characteristics of this quality model, we propose the concept of Web API learnability to use and stability to change, from the perspective of Web API users. Based on this quality model, we also propose a set of measures and a quantitative evaluation method. In this study, we applied the proposed quality model and evaluation method to four types of actual Web APIs, including Uber, WordPress, OpenStack, and Media Processing. To validate the proposed model, we also conducted an empirical study of the usability of the Web APIs. Our comparison of the proposed quality statistics with those from the empirical study validates the effectiveness of the proposed quality model and its associated measures of the learnability and stability of Web APIs.

Automated unit test generation bears the promise of significantly reducing test cost and hence improving software quality. However, the maintenance cost of the automatically generated tests presents a significant barrier to adoption of this technology. To address this challenge, we propose a novel technique for automated and fine-grained incremental generation of unit tests through minimal augmentation of an existing test suite. The technique uses iterative, incremental refinement of test-drivers and symbolic execution, guided by a diagnostics engine. The diagnostics engine works off a novel precise and efficient byte-level dynamic dependence analysis built using Reduced Ordered Binary Decision Diagrams (ROBDDs). We present a tool FSX implementing this technique and evaluate it under two practical use-cases of incremental unit test generation, on five revisions of the open-source software iPerf, as well as on 3 large subjects, comprising more than 60 thousand lines of code, from in-house commercial network products. The evaluation shows that FSX can generate high-quality unit tests on large industrial software while minimizing the maintenance cost of the overall test-suite.

Automated unit test generation bears the promise of significantly reducing test cost and hence improving software quality. However, the maintenance cost of the automatically generated tests presents a significant barrier to adoption of this technology. To address this challenge, in previous work, we proposed a novel technique for automated and fine-grained incremental generation of unit tests through minimal augmentation of an existing test suite. In this paper we describe a tool FSX, implementing this technique. We describe the architecture, user-interface, and salient features of FSX, and specific practical use-cases of its technology. We also report on a real, large-scale deployment of FSX, as a practical validation of the underlying research contribution and of automated test generation research in general.

Location-based services (LBS) are very important mobile app services, which provide diverse mobility services for mobile users anywhere and anytime.This brings new demands, issues, and challenges in mobile application testing.Today, mobile applications provide location-based service functions based on dynamic location contexts, mobile users and their travel patterns to deliver location-based mobile data, and service actions.Current software testing methods do not consider location-based validation coverage.Hence, there is a lack of research results addressing location-based mobile application testing.This paper focuses on mobile LBS testing.A novel test object model is proposed for quality validation of location-based mobile information services.In addition, the related test coverage metrics are also presented.These metrics can be useful for test engineers in designing test cases.A case study based on student testers is reported to demonstrate the potential application of the proposed model.

After a legacy system is re-engineered, it is important to perform compatibility testing so as to identify the differences and reduce the introduced bugs. We can first apply symbolic execution to obtain an exhaustive set of test cases, then use them to check the compatibility of the old system and the new one. However there may be a lot of failed test cases which are a mix of erroneous and allowable incompatibilities. To locate the causes of failures detected during the testing, we apply multiple statistical bug localization techniques. We are able to localize 90% of the incompatibilities in 10% of the code for an industrial application with around 20k lines by Tarantula. And we identify the characteristics of failure causes which are difficult to be detected by statistical bug localization.

We are developing a test case generation service that uses symbolic execution technique, to support testing of Java programs. In addition to program under test, we need drivers and stubs to execute the service. However, preparing drivers and stubs manually is a time-consuming and sometimes difficult task. In order to reduce the cost of such manual effort, we developed a tool that automates not only driver and stub preparation, but also other necessary processes of our test case generation service. The tool eliminates all manual routine work so that testers can perform the test with only one click. In this paper, we give a description of this tool which includes its implementation details and the techniques developed to automate its internal processes. We also report on its application to real project to evaluate the efficacy of the tool. Our results demonstrate that the tool reduces the amount of manual effort from several minutes to just a few seconds, while preserving a high level of test coverage.

Symbolic execution as a test case generation technique has recently become an active research area. However, since symbolic execution generates a large number of test cases, it is impractical to run all the generated test cases in practice. In this paper, we present a test case selection method for a symbolic execution-based test case generation. This method has the following two characteristics. 1)Test cases which cover atomic conditions collected during symbolic execution, called path condition-based, are selected to keep the fault-detection capability. 2)This method does not depend on a particular symbolic execution engine since it is based on an analysis of path conditions in a general format. We implemented this method in our tool and evaluated this method with real systems. Our evaluation shows that the method produces a significant reduction in the size of the test suite while effectively preserving its fault-detection capability. We also demonstrate that the method is compatible with practically any symbolic execution engine, including the popular tools Java Path Finder and KLEE.

After a legacy system is re-engineered, it is important to perform compatibility testing so as to identify the difference and reduce the introduced bugs. We can first apply symbolic execution to obtain an exhaustive set of test cases, then use them to check the compatibility of the old system and the new old. However, the path explosion problem of symbolic execution makes it difficult to work on realistic non-trivial applications. We show in this paper how to enhance symbolic execution, e.g. with extra constraints, path cutting, variable grouping, and test case selection, to successfully test the compatibility of an SMTP library (used in embedded systems) with around 20K lines of code. Our experience indicates that these enhancements are essential to apply symbolic execution on realistic industrial applications.

With the fast advancements in cloud computing and software- as-a-service (SaaS), testing and evaluation of cloud-based software and SaaS applications became an important task for engineers. Since most existing tools are not developed to support cloud-based software testing and SaaS evaluation, there is a strong demand for a new cloud-based testing infrastructure and evaluation environment for SaaS applications. This paper proposes a testing-as-service (TaaS) infrastructure and reports a cloud-based TaaS environment with tools (known as CTaaS) developed to meet the needs in SaaS testing, performance and scalability evaluation. The paper presents TaaS concepts and CTaaS, including their infrastructure, design and implementation. In addition, the paper demonstrates the application results of our previously proposed graphic models and metrics for SaaS performance and scalability evaluation. Moreover, the paper reports one case study for a selected SaaS (OrangeHRM) using the developed TaaS environment.