MobileGraphical User Interface(GUI) testing is a critical component of quality assurance for mobile applications. As GUI designs grow increasingly complex, vision-based testing methods have become essential for improving the quality of test scripts and reports. However, current approaches face significant limitations. For instance, the process of cropping GUI widgets requires significant manual effort and consumes considerable time. Meanwhile, existing automated testing tools still fail to generate satisfactory test sequences. In this paper, we proposeVision-based Test Sequence Generation(VTSG), a novel perception-driven approach for mobile GUI testing. More specifically: (1) VTSG employs a light-weight GUI element detection model to crop widgets from GUI pages automatically; (2) Guided by human perception principles, it sequences widget screenshots through saturation and spatial layout analysis; (3) The system then integrates GUI widget interaction instructions to generate vision-based test scripts that accurately simulate human interaction patterns on mobile devices. We evaluate VTSG against four state-of-the-art (SOTA) GUI testing tools across five applications. Experimental results demonstrate that VTSG significantly outperforms existing methods, achieving 47.44% code coverage and 51.33% activity coverage, respectively, compared to the other approaches. Additionally, we conduct a series of supplementary experiments on two mainstream commercial applications (i.e.,ToutiaoandDouyin). The results further confirm that VTSG maintains higher activity coverage even on these real-world commercial apps.

Several code comment generation approaches based on sequence-to-sequence (Seq2Seq) models have been proposed. Such approaches often extract structure information from abstract syntax trees (ASTs) using a certain serialization method. However, some structural information is inevitably lost while serializing ASTs. Furthermore, existing serialization methods only consider the “type” attribute of the nodes, neglecting the “value” attribute of the nodes. To further improve the performance of code comment generation, we propose a code comment generation approach, called SCATCom, which integrates a more comprehensive set of information from source code and ASTs, encompassing semantic, sequential, syntactic, and hierarchical structure information for code comment generation. Meanwhile, an AST traversal method, called V-POT, is presented, which considers both the “type” and the “value” attributes of the nodes. Experiments were designed and conducted on two commonly used datasets to validate the performance of our approach and the impact of five different serialization ways of ASTs on two code comment generation methods. The BLEU, METEOR, and ROUGE scores for our approach reach 52.6, 34.16, and 63.26 with an improvement of [Formula: see text], [Formula: see text], and [Formula: see text] compared to the baselines. It is evident that V-POT, which retains both the “type” and the “value” attributes, is superior to other methods that use only the “type” attribute.

In recent years, deep learning (DL) systems have been applied in many areas, including image processing and autonomous driving. Software testing is an important way to ensure the quality of software. Among various testing methods, random testing (RT) has been widely used for DL systems, due to its simplicity and efficiency. However, it has been criticized for its poor fault-detection effectiveness. As an enhancement of RT, adaptive random testing (ART) attempts to evenly spread test cases over the input domain, aiming to improve the distribution diversity. However, current ART methods for DL systems have low testing efficiency, particularly for image-based DL systems. This is because of the current reliance on visual geometry group network-16 (VGGNet-16) to extract image features to represent image inputs—VGGNet-16 is a 16-layer, deep convolutional neural network that has been widely used for image classification and feature extraction. Feature extraction with VGGNet-16 is very time-consuming, with each image being extracted as a high-dimensional vector. The (dis)similarity calculations for images with such high-dimensional vectors incur heavy computational overheads. To overcome these challenges, we propose a new ART approach:image-hashing-based ART(IHART). IHART uses image hashing to quickly extract features from each image, storing them as a low-dimensional binary vector. This can significantly reduce the computational costs for dissimilarity calculations during test-case generation. We report on a series of experiments, using several well-known datasets and DL systems, to evaluate the IHART performance. Our results show that, of the three mainstream image-hashing strategies studied, perceptual hashing delivers the best ART test-case generation performance—perceptual hashing, which is used in image deduplication and content searching, uses content features in the hashing process. Compared with current approaches, IHART performs very well in fault-detection effectiveness across most datasets and models, and significantly better fault-detection efficiency.

Automated fault localization has been extensively studied to improve the effectiveness of software debugging. Existing automated fault localization methods neglect the guidance of the simple and easily available debugging information on fault localization. To bridge manual fault localization with automated fault localization, we propose a fault localization approach combining error guessing and logical reasoning via deep learning. The proposed approach simulates the actual debugging process. Specifically, developers’ debugging experience and context dependencies between methods are mapped into two different types of coverage matrices. The constructed matrices are fed to a convolutional neural network (CNN) to predict whether a method is buggy or not. To validate the effectiveness of the proposed approach, we designed and constructed the empirical study on the widely used Defect4J datasets. With respect to the top-n ([Formula: see text]) metric, our approach outperforms the state-of-the-art DeepFL and other five methods including Ochai, Muse, MULTRIC, TraPT and FLUCSS. Particularly, compared with the above methods, our approach has an improvement of 5–182% for top-1. In terms of MFR and MAR, the proposed approach is slightly lower than the best DeepFL but better than the other five methods. The approach we presented achieving the unification of manual and automatic debugging can aid in the improvement of fault localization accuracy.

Software fault localization is notoriously tedious and time-consuming. Developed rapidly, machine learning techniques have been adopted for fault localization by researchers. Most existing approaches use the test coverage information as feature input to the learning model, ignoring the limited ability of the single-dimensional features. The effectiveness of fault localization is not greatly improved. To overcome the limitation, we propose a fault localization approach based on Bidirectional Recurrent Neural Networks (BiRNNs) and multi-dimensional features. Our approach collects suspiciousness-based, text similarity-based and fault-proneness-based features from the traditional fault localization areas and software metrics. To evaluate our approach, the experiments have been studied on the real-fault benchmark Defects4J and seeded fault program NanoXML. The experimental results show that our approach effectively improves fault localization accuracy.

Software fault localization, which is an important software quality assurance technology, provides the location of the faults in software to improve the efficiency of debugging and repairing. In previous research, software fault localization techniques, such as spectrum-based, mutation-based, and program slicing, have been widely used and achieved good results. However, many statements could have same suspicious values by using these techniques, which will consume large amount of manual effort to confirm and affect the accuracy of fault localization. For example, using Ochiai or DStar to locate 395 faulty versions of 6 projects in Defects4J, nearly 70 % of the faulty versions have more than one suspicious statement are ranked as top tied 1. To address the above problem, this paper proposes a complexity-based fault localization (CBFL) technique to further improve the accuracy of fault localization. Firstly, a set of metrics for measuring the complexity of statements is proposed, and the metrics of each statement in projects are extracted to construct a classification model. Then, the classification model is used to predict the faulty probability of the statements which are ranked as top tied 1 by SBFL, MBFL or other techniques, and these statements are reranked according to the estimated faulty probability to improve the accuracy of fault localization. This paper implements a fault localization tool CDStar based on the CBFL, and conducts experiments on the Defects4J dataset. Comparing with the DStar, the results show that CBFL outperforms DStar in terms of Einspect @1 and EXAM.

Integration testing is a very important step in software testing. Existing methods evaluate the stubbing cost for class integration test orders by considering only the interclass direct relationships such as inheritance, aggregation, and association, but they omit the interclass indirect relationship caused by control coupling, which can also affect the test orders and the stubbing cost. In this paper, we introduce an integration test order strategy to consider control coupling. We advance the concept of transitive relationship to describe this kind of interclass dependency and propose a new measurement method to estimate the complexity of control coupling, which is the complexity of stubs created for a transitive relationship. We evaluate our integration test order strategy on 10 programs on various scales. The results show that considering the transitive relationship when generating class integration test orders can significantly reduce the stubbing cost for most programs and that our integration test order strategy obtains satisfactory results more quickly than other methods.

Summary Concurrent programs are notoriously difficult to debug due to shared memory and the non‐determined nature of thread scheduling. Frequent context switches add a huge burden on developers in reasoning about concurrency bugs. To alleviate this problem, we present a bidirectional trace simplification approach based on a context switch linked list. First, we calculate the dependence relations, including local dependences, synchronization dependences, and remote read/write dependences. Second, we construct a context switch linked list according to the original buggy trace. Then, we backward refactor the context switch linked list in sequence to extend thread execution intervals. Finally, we forward check the context switch linked list in sequence to ensure that no nodes can be further merged. We have conducted experiments on eight Java multi‐threaded programs to evaluate our approach. The results show that our approach performs better than or is comparable to the compared static approaches in effectiveness and efficiency.

Regression testing is a very time-consuming and expensive testing activity. Many test case prioritization techniques have been proposed to speed up regression testing. Previous studies show that no one technique is always best. Random strategy, as the simplest strategy, is not always so bad. Particularly, when a test suite has higher fault detection capability, the strategy can generate a better result. Nevertheless, due to the randomness, the strategy is not always as satisfactory as expected. In this context, we present a test case prioritization approach using fixed size candidate set adaptive random testing algorithm to reduce the effect of randomness and improve fault detection effectiveness. The distance between pair-wise test cases is assessed by exclusive OR. We designed and conducted empirical studies on eight C programs to validate the effectiveness of the proposed approach. The experimental results, confirmed by a statistical analysis, indicate that the approach we proposed is more effective than random and the total greedy prioritization techniques in terms of fault detection effectiveness. Although the presented approach has comparable fault detection effectiveness to ART-based and the additional greedy techniques, the time cost is much lower. Consequently, the proposed approach is much more cost-effective.

Software defect prediction (SDP) utilizes the learning models to detect the defective modules in project, and their performance depends on the quality of training data. The previous researches mainly focus on the quality problems of class imbalance and feature redundancy. However, training data often contains some instances that belong to different class but have similar values on features, and this leads to class overlap to affect the quality of training data. Our goal is to investigate the impact of class overlap on software defect prediction. At the same time, we propose an improved K-Means clustering cleaning approach (IKMCCA) to solve both the class overlap and class imbalance problems. Specifically, we check whether K-Means clustering cleaning approach (KMCCA) or neighborhood cleaning learning (NCL) or IKMCCA is feasible to improve defect detection performance for two cases (i) within-project defect prediction (WPDP) (ii) cross-project defect prediction (CPDP). To have an objective estimate of class overlap, we carry out our investigations on 28 open source projects, and compare the performance of state-of-the-art learning models for the above-mentioned cases by using IKMCCA or KMCCA or NCL VS. without cleaning data. The experimental results make clear that learning models obtain significantly better performance in terms of balance, Recall and AUC for both WPDP and CPDP when the overlapping instances are removed. Moreover, it is better to consider both class overlap and class imbalance.

Software testing consumes a significant portion of software effort. Program entities such as branch or definition–use pairs (DUPs) are used in diverse software development tasks. In this study, the authors present a novel evolution‐based approach to generating test data for all definition–use coverage. First, the subset of DUPs, which can ensure the coverage adequacy, is computed by a reduction algorithm for the whole DUPs. Then they apply a genetic algorithm to generate test data for the subset of DUPs. Furthermore, the fitness of an individual depends on the matching degree between the traversed path and the definition‐clear path of each target DUP. They also investigate the coverage and the size of test cases of test data generation by applying the authors’ approach on 15 widely used subject programs. The experimental results show that their approach can reduce the size of test cases that generated without affecting the coverage rate.

Automatic protocol mining is a promising approach for inferring accurate and complete API protocols. However, just as with any data-mining technique, this approach requires sufficient training data (object usage scenarios). Existing approaches resolve the problem by analyzing more programs, which may cause significant runtime overhead. In this paper, we propose an inheritance-based oversampling approach for object usage scenarios (OUSs). Our technique is based on the inheritance relationship in object-oriented programs. Given an object-oriented program p , generally, the OUSs that can be collected from a run of p are not more than the objects used during the run. With our technique, a maximum of n times more OUSs can be achieved, where n is the average number of super-classes of all general OUSs. To investigate the effect of our technique, we implement it in our previous prototype tool, ISpecMiner, and use the tool to mine protocols from several real-world programs. Experimental results show that our technique can collect 1.95 times more OUSs than general approaches. Additionally, accurate and complete API protocols are more likely to be achieved. Furthermore, our technique can mine API protocols for classes never even used in programs, which are valuable for validating software architectures, program documentation, and understanding. Although our technique will introduce some runtime overhead, it is trivial and acceptable.

Dynamic Protocol Mining (DPM) techniques are a promising approach to infer useful API protocols automatically.However, their results are biased to input test cases and the instrumentation overhead discounts their usability in industrial practice.In this paper, we propose a distributed dynamic protocol mining framework NSpecMiner.Our framework is based on a client-server architecture, where the client tracer gathers Program Execution Traces (PETs) and sends them to the server for mining.Mined protocols are saved on the server to provide various kinds of remote services, such as API protocol retrieval and program verification, etc.Compared with local miners, NSpecMiner has many advantages: 1) A large number of diverse PETs are likely to be collected from multiple clients, which is essential for mining accurate and complete API protocols.2) Instrumentation overhead can be balanced among multiple clients.3) Via integrating the client tracer into widely used software, we can mine API protocols transparently and automatically without any human effort.To evaluate our technique, we performed a comparison test with a local miner ISpecMiner and NSpecMiner.Preliminary results show that our approach is effective to mine useful API protocols as local miners.While our method is able to gather PETs concurrently from multiple clients and other merits of the distributed technology will further benefit DPM significantly.

Software defect prediction is aimed to find potential defects based on historical data and software features. Software features can reflect the characteristics of software modules. However, some of these features may be more relevant to the class (defective or non-defective), but others may be redundant or irrelevant. To fully measure the correlation between different features and the class, we present a feature selection approach based on a similarity measure (SM) for software defect prediction. First, the feature weights are updated according to the similarity of samples in different classes. Second, a feature ranking list is generated by sorting the feature weights in descending order, and all feature subsets are selected from the feature ranking list in sequence. Finally, all feature subsets are evaluated on a k-nearest neighbor (KNN) model and measured by an area under curve (AUC) metric for classification performance. The experiments are conducted on 11 National Aeronautics and Space Administration (NASA) datasets, and the results show that our approach performs better than or is comparable to the compared feature selection approaches in terms of classification performance.

Class temporal specification is a kind of important program specifications, which specifies that methods of a class should be called in a particular sequence.Dynamic specification mining is a promising approach to achieve this kind of specifications automatically.However, they always infer partial specifications, that is, the mined specifications are biased to input programs or program execution traces.In this paper, we propose to mine class temporal specifications based on a probabilistic model in an online mode.Since our method can evolve mined specifications persistently, universal specifications can be achieved.To investigate our technique's feasibility and effectiveness, we implemented it in a prototype tool ISpecMiner and used the tool to perform experiments.Experimental results show that our method is promising to infer universal specifications if sufficient traces are provided for mining.

Automatic protocol mining is a promising approach to infer precise and complete API protocols.However, the effect of the approach largely depends upon the quality of input object usage scenarios, in terms of noise and diversity.This paper aims to extract as many object usage scenarios as possible from object-oriented programs for automatic protocol mining.A large corpus of object usage scenarios can help with eliminating noise accurately and is likely to be diverse.Therefore, precise and complete protocols may be achieved.Given an object-oriented program p, generally, object usage scenarios that can be collected from a run of p is not more than the number of instances used in p. Relying on the inheritance relationship among classes, our technique can extract a maximum of n times more object usage scenarios from p, where n is the average inheritance depth of all object usage scenarios in p.In order to investigate the effect of our technique on mining protocols, we implement it in our previous prototype tool ISpecMiner and use the tool to mine protocols from several real-world applications.The experimental results show that our technique is promising to achieve complete and precise API protocols.In addition, protocols of classes that have not been used in programs can be also achieved, which is helpful for program documentation and understanding.

With the continuous evolution of software systems, test suites often grow very large.Rerunning all test cases may be impractical in regression testing under limited resources.Coverage-based test case prioritization techniques have been proposed to improve the effectiveness of regression testing.The original test suite often contains some test cases which are designed for exercising production features or exceptional behaviors, rather than for code coverage.Therefore, coverage-based prioritization techniques do not always generate satisfactory results.In this context, we propose a global similarity-based regression test case prioritization approach.The approach reschedules the execution order of test cases based on the distances between pair-wise test cases.We designed and conducted empirical studies on four C programs to validate the effectiveness of our proposed approach.Moreover, we also empirically compared the effects of six similarity measures on the global similarity-based test case prioritization approach.Experimental results illustrate that the global similarity-based regression test case prioritization approach using Euclidean distance is the most effective.This study aims at providing practical guidelines for picking the appropriate similarity measures.

Regression test case reduction aims at selecting a representative subset from the original test pool, while retaining the largest possible fault detection capability. Cluster analysis has been proposed and applied for selecting an effective test case subset in regression testing. It groups test cases into clusters based on the similarity of historical execution profiles. In previous studies, historical execution profiles are represented as binary or numeric function coverage vectors. The vector-based similarity approaches only consider which functions or statements are covered and the number of times they are executed. However, the vector-based approaches do not take the relations and sequential information between function calls into account. In this paper, we propose cluster analysis of function call sequences to attempt to improve the fault detection effectiveness of regression testing even further. A test is represented as a function call sequence that includes the relations and sequential information between function calls. The distance between function call sequences is measured not only by the Levenshtein distance but also the Euclidean distance. To assess the effectiveness of our approaches, we designed and conducted experimental studies on five subject programs. The experimental results indicate that our approaches are statistically superior to the approaches based on the similarity of vectors (i.e. binary vectors and numeric vectors), random and greedy function-coverage-based maximization test case reduction techniques in terms of fault detection effectiveness. With respective to the cost-effectiveness, cluster analysis of sequences measured using the Euclidean distance is more effective than using the Levenshtein distance.

Combinatorial interaction testing is a well-studied testing strategy, and has been widely applied in practice. Combinatorial interaction test suite, such as fixed-strength and variable-strength interaction test suite, is widely used for combinatorial interaction testing. Due to constrained testing resources in some applications, for example in combinatorial interaction regression testing, prioritization of combinatorial interaction test suite has been proposed to improve the efficiency of testing. However, nearly all prioritization techniques may only support fixed-strength interaction test suite rather than variable-strength interaction test suite. In this paper, we propose two heuristic methods in order to prioritize variable-strength interaction test suite by taking advantage of its special characteristics. The experimental results show that our methods are more effective for variable-strength interaction test suite by comparing with the technique of prioritizing combinatorial interaction test suites according to test case generation order, the random test prioritization technique, and the fixed-strength interaction test suite prioritization technique. Besides, our methods have additional advantages compared with the prioritization techniques for fixed-strength interaction test suite.