Sufficient testing against real-world ML model is mandatory because the safety violation of that behavior causes a huge social impact. Existing methods detect weaknesses in ML models using labeled features associated with data based on the concept of combinatorial test modeling. However, sufficient testing is difficult due to rare features that are difficult to collect from the real world, especially for image data. In this paper, we investigate an approach using an image generation model to solve this problem, and answer the following research question: To what extent can the generated data detect weaknesses by factor-level compared to the test data? Our experiments showed that $74 \%$ of the weaknesses detected by the generated data were detected by the test data as well. Our further investigations showed that the feasibility of generating data in a way that transforms factor levels is influenced by the generative model used for the transformation and the type of transformation.

In this paper, we propose the first verification method for attention robustness, i.e., the local robustness of the changes in the saliency-map against combinations of semantic perturbations. Specmcally, our method determines the range of the perturbation parameters (e.g., the amount of brightness change) that maintains the difference between the actual saliencymap change and the expected saliency-map change below a given threshold value. Our method is based on linear activation region traversals, focusing on the outermost boundary of attention robustness for scalability on larger deep neural networks.

Software development often involves systematic edits, similar but nonidentical changes to many code locations, that are error-prone and laborious for developers. Mining and learning such systematic edit patterns (SEPs) from past code changes enable us to detect and repair overlooked buggy code that requires systematic edits. A recent study presented a promising SEP mining technique that is based on program dependence graphs (PDGs), while traditional approaches leverage syntax-based representations. PDG-based SEPs are highly expressive and can capture more meaningful changes than syntax-based ones. The next challenge to tackle is to apply the same code changes as in PDG-based SEPs to other code locations; detection and repair of overlooked locations that require systematic edits. Existing program transformation techniques cannot well address this challenge because (1) they expect many structural code similarities that are not guaranteed in PDG-based SEPs or (2) they work on the basis of PDGs but are limited to specific domains (e.g., API migrations). We present in this paper a general-purpose program transformation algorithm for applying PDG-based SEPs. Our algorithm identifies a small transplantable structural subtree for each PDG node, thereby adapting code changes from PDG-based SEPs to other locations. We construct a program repair pipeline Sirius that incorporates the algorithm and automates the processes of mining SEPs, detecting overlooked code locations (bugs) that require systematic edits, and repairing them by applying SEPs. We evaluated the repair performance of Sirius with a corpus of open source software consisting of over 80 repositories. The results indicate that Sirius greatly outperformed the state-of-the-art technique for syntax-based SEPs. Sirius achieved a precision of 0.710, recall of 0.565, and F1-score of 0.630, while those of the state-of-the-art technique were 0.470, 0.141, and 0.216, respectively.

Data augmentation techniques that increase the amount of training data by adding realistic transformations are used in machine learning to improve the level of accuracy. Recent studies have demonstrated that data augmentation techniques improve the robustness of image classification models with open datasets; however, it has yet to be investigated whether these techniques are effective for industrial datasets. In this study, we investigate the feasibility of data augmentation techniques for industrial use. We evaluate data augmentation techniques in image classification and object detection tasks using an industrial in-house graphical user interface dataset. As the results indicate, the genetic algorithm-based data augmentation technique outperforms two random-based methods in terms of the robustness of the image classification model. In addition, through this evaluation and interviews with the developers, we learned following two lessons: data augmentation techniques should (1) maintain the training speed to avoid slowing the development and (2) include extensibility for a variety of tasks.

Automated program repair (in short, APR) is a truly desired technique because it can reduce debugging costs drastically. A well-known technique in APR is a reuse-based approach, which inserts existing program statements in a given program to suspicious code for an exposed bug. Some reports show the reuse-based approach was able to fix many bugs in open source software. However, the existing approach often takes very long time to fix bugs. Its main factor is that so many variant programs are generated by insertions and so many test cases are executed for the variant programs before a fixed program is generated. In order to shorten fixing time with the reuse-based approach, a fixed program must be generated much more efficiently. In this paper, we propose two strategies to generate a fixed program more efficiently. We also implement the two strategies and confirm that there are real bugs which the two strategies contribute to shortening fixing time.

Automated program repair is a way to reduce costs on program debuggingto a large extent. Repair techniques using genetic programming havebeen attracting much attention. They were applied to actual softwaresystems and they were able to fix several dozen of actual faults. However, programs generated by such techniques often include some sourcecode changes not related to fixing a given fault even if they pass allgiven test cases. Furthermore, some researchers found that suchtechniques occasionally induce new faults which are not covered byexisting test cases. The reason why those problems arise is that suchtechniques consider only given test cases. On the other hand, developers consider program behaviors not covered by test cases. Thus, those problems arise less frequently in programs modified by developers. Consequently, the authors suppose that if we make automated programmodifications close to developers' ones, we may be able to relieve thoseproblems. At this moment, there is no research study investigatingdifferences between automated modifications and developers' ones. Inthis paper, we compare GenProg's modifications with developers'ones for the same faults. As a result, we found that developers tend to(1) change more different functions, (2) change control flows in sourcecode, and (3) add/delete more code lines.