Existing research in mobile app compatibility analysis faces two critical limitations: the absence of large-scale real-world datasets and the lack of systematic taxonomies for tool evaluation. To address these gaps, this study proposes a novel two-dimensional classification framework that categorizes compatibility issues through two analytical dimensions: causes (including API-induced issues such as deprecated methods, system-induced issues like permission changes, and hardware-induced issues such as sensor variations) and consequences (encompassing crashes, functional failure, UI inconsistencies, and performance degradation). Based on this classification framework, we developed CompatBench, the first manually curated dataset containing 78 real-world cases collected from GitHub submissions spanning 2017 to 2024. The dataset comprises 45 researcher-reproduced issues and 33 user-reported issues. Key findings reveal significant discrepancies between laboratory reproduction scenarios and real-world cases. API-related causes dominate reproduction cases at 91.1 %, but account for only 42.4 % of user-reported issues. Regarding manifestation visibility, 35.6 % of reproduction cases lack clear symptoms, whereas 96.9 % of real cases exhibit observable failures such as crashes or UI inconsistencies. Repair dynamics analysis shows 88.8 % of reproducible issues resolved within 50 days, contrasting with a 55.7 % fix rate for real issues. These results highlight critical methodological gaps in current research paradigms, particularly in causal inference based on observable consequences and consequence prediction based on potential causes. The study establishes three core contributions for validating compatibility management tools: a standardized taxonomy for issue classification, an openaccess dataset with real-world cases, and empirical evidence demonstrating discrepancies between laboratory and field observations. The findings emphasize the necessity of incorporating real-world datasets to improve evaluation validity in compatibility research.

Abstract The compatibility issues caused by Android fragmentation have become a vital task in the development of Android applications. To locate those issues, thousand of crowd testers run apps on different devices with different configurations to achieve the largest coverage, which might be costly and time‐consuming. Since existing approaches to selecting optimal devices are device‐side analysis without the information of the internal structures of apps, app‐side analysis that flags the essential devices for testers has remained elusive. To mitigate this gap of compatibility crowdtesting, this paper proposes an app constraint analysis approach named CompatDroid to generate the optimal device set to guide crowd testers. By evaluating 46 benchmark apps on 14 SDK versions, the optimal device sets are successfully generated, and CompatDroid only needs no more than 7 Android versions to achieve almost the same code coverage (i.e., 33.13%) testing on all 14 android versions (i.e., 34.65%) in 36 of 46 apps, which indicates that it can drastically reduce the consumption of test resources while losing little test coverage. On a larger dataset, CompatDroid successfully analyzes 98.3% of 645 apps, in which the median number of the optimal SDK versions set is 2.5 versions, and 68.92% of those apps contain the constraints of SDK version (i.e., SDK version) while 84.86% of them do not have the constraints of hardware information (i.e., model name and manufacture name).

Android API-related compatibility issues have be-come a severe problem and significant challenge for app devel-opers due to the well-known Android fragmentation issues. To address this problem, many effective approaches such as app-based and API lifetime-based methods have been proposed to identify incompatible API usages. However, due to the various implementations of API usages and different API invoking paths, there is still a significant weakness of existing approaches, i.e., introducing a massive number of false positives (FP) and false negatives (FN). To this end, in this paper, we propose PSDroid, an automated compatibility detection approach for Android apps, which aims to reduce FPs and FNs by overcoming several technical bottlenecks. Firstly, we make substantial efforts to carry out a preliminary study to summarize a set of novel API usages with diverse checking implementations. Secondly, we construct a refined API lifetime database by leveraging a semantic resolving analysis on all existing Android SDK frameworks. Based on the above two key phases, we design and implement a novel path-sensitive semantic approach to effectively and automatically detect incompatibility issues. To demonstrate the performance, we compared with five existing approaches (i.e., FicFinder, ACRYL, CIDER, IctAPIFinder, and CID) and the results show that PSDroid outperforms existing tools. We also conducted an in-depth root cause analysis to comprehensively explain the ability of PSDroid in reducing FPs and FNs. Finally, 18/30 reported issues have been confirmed and further fixed by app developers.

Deep learning (DL) has shown superior performance in many areas, making the quality assurance of DL-based software particularly important. Adversarial examples are generated by deliberately adding subtle perturbations in input samples and can easily attack less reliable DL models. Most existing works only utilize a single metric to evaluate the generated adversarial examples, such as attacking success rate or structure similarity measure. The problem is that they cannot avoid extreme testing situations and provide multifaceted evaluation results.This paper presents MetaA, a multi-dimensional evaluation framework for testing ability of adversarial examples in deep learning. Evaluating the testing ability represents measuring the testing performance to make improvements. Specifically, MetaA performs comprehensive validation on generating adversarial examples from two horizontal and five vertical dimensions. We design MetaA according to the definition of the adversarial examples and the issue mentioned in [1] that how to enrich the evaluation dimension rather than merely quantifying the improvement of DL and software.We conduct several analyses and comparative experiments vertically and horizontally to evaluate the reliability and effectiveness of MetaA. The experimental results show that MetaA can avoid speculation and reach agreement among different indicators when they reflect inconsistencies. The detailed and comprehensive analysis of evaluation results can further guide the optimization of adversarial examples and the quality assurance of DL-based software.

Modern software brings many conveniences to users through big data, but it also risks privacy leakage. In recent years, privacy leaks have been frequent, and various countries have introduced privacy protection bills to protect users' privacy security and avoid misuse of their private data.The researchers have conducted many studies to protect user privacy, including privacy policy compliance checks and mobile application permission checks. However, little existing work considers the verification of matching software code behavior and privacy policy. In this paper, we propose a set of privacy scanning methods to solve mentioned issues with static code analysis.We first classify privacy text and extracts privacy information. Then we perform static analysis on the code to obtain variable privacy information and privacy propagation paths by combining an abstract syntax tree and the call graph. We also match the results to the text analysis results. The experiments demonstrate that our method outperforms other classification methods in privacy text judgment, with an accuracy rate of 90% in detecting privacy information in the code. Meanwhile, the short running time ensures that no extra overhead is imposed on the user.

For Android malware detection, machine learning-based (ML-based) methods show promising performance. However, limited studies are performed to investigate the impact of factors related to datasets on ML-based methods, while the performance of ML-based methods dramatically relies on datasets. To partially bridge the gap, we conduct an empirical study to investigate the impact of factors related to datasets on ML-based Android malware detection methods. By investigating dataset differences between real-world scenarios and experimental settings, we summarize three dataset factors (i.e., class imbalance, quality, and timelines) and assess the impact of these factors on ML-based Android malware detection methods. We conduct experiments on more than 11K benign and 17K malicious applications. The results show that these three dataset factors yield significant biases in the existing ML-based Android malware detection methods. Based on these results, we learn some lessons about assessing ML-based Android malware detection methods when taking dataset factors into account.

ERC20 is the first token standard on Ethereum and is widely used in ICOs, voting, and various asset representations. However, some methods defined in ERC20 imply potential vulnerabilities and Multiple Withdrawal Attack is one of them. Attackers can transfer more tokens than the actual allowance through this vulnerability. The current prevention methods for Multiple Withdrawal Attack include changing the transaction process, modifying the API of ERC20, and modifying the implementation of functions, etc. However, these methods have disadvantages such as poor compatibility, incomplete resolution, and high gas consumption. In this paper, we describe the process of Multiple Withdrawal Attack and analyze the shortcomings of the existing methods, and then propose a solution with lower gas consumption. In our method, a variable is added to record the allowance in the approval function to prevent tokens from being transferred repeatedly. Finally, the effectiveness and the performance of the proposed method is analyzed. The result shows that the method proposed in this paper is safe and has lower gas consumption than the existing methods.

With the prosperity of Mobile APPs, developers need to dynamically find the compatibility issues by testing APP on all Android versions and devices, which is costly. This paper finds a systematic test method based on source codes of apps to be tested by identify compatibility-related characteristic codes. We propose an automated tool named “Periph” to search the source code of each app that can eventually generate compact versions and devices set to guide the testing. Through testing 21 apps on 13 Android versions, we find that our proposed tool can greatly reduce the consumption of test resources while only losing little test coverage.

Most of metamorphic testing (MT) research works focused on the generation and application of metamorphic relations (MRs). There is no clear conclusion about the relationship between test case generation methods and performance of MT. In this article, we introduce a novel method based on adaptive random testing (ART) and MR for MT test case generation. It proposes a family of algorithms for MT test cases generation, named as MT based ART (MT-ART). Three distances are measured to generate the next MT test case. In order to verify the performance of this method, series of experiments on four programs with different numbers of inputs are introduced. The results show that MT-ART performs better than other ART algorithms not only in test effectiveness, but also in test efficiency and test coverage. Based on this article, the following conclusions can be drawn: first, considering the effectiveness of MRs and test cases in MT may lead to better results. In this way, most of the existing research can be improved by this method. This is the most important contribute of our research. Second, not only the source test cases, but also the follow-up test cases can improve the performance of MT. Therefore, they should be considered together during the process of the next test case generation. Third, the average distance performs better than the max distance and the minimum distance in metamorphic test case selection.

With the rise of crowdsourced software testing in recent years, the issuers of crowd test tasks can usually collect a large number of test reports after the end of the task. These reports have insufficient validity and completeness, and manual review often takes a lot of time and effort. The crowdsourced test task publisher hopes that after the crowdsourced platform collects the test report, it can analyze the validity and completeness of the report to determine the severity of the report and improve the efficiency of crowdsourced software testing. In the past ten years, researchers have used various technologies (such as natural language processing, information retrieval, machine learning, deep learning) to assist in analyzing reports to improve the efficiency of report review. We have summarized the relevant literature of report analysis in the past ten years, and then classified from report classification, duplicate report detection, report prioritization, report refactoring, and summarized the most important research work in each area. Finally, we propose research trends in these areas and analyze the challenges and opportunities facing crowdsourced test report analysis.

A geographic information system (GIS) provides basic location-enabled services for many different applications related to navigation, education, and telecommunications. It is a foundation for analysis and visualization. Testing GIS is critical, but challenging due to the difficulty to assess the correctness of GIS outputs, which is called the test oracle problem of software testing. Metamorphic testing alleviates the problem by constructing metamorphic relations (MRs) among multiple inputs and outputs of the program under test. In this article, a semiautomated metamorphic testing (SAMT) method, based on the formal MR model and an improved adaptive random testing algorithm, was proposed to the GIS. To evaluate the performance of our approach, we conducted a case study on a superficial area calculation program, a typical component of GIS. Six kinds of MR construction methods were suggested for the GIS domain program testing. The experimental results show that SAMT can detect the mutations effectively that could solve the test oracle problem efficiently. More importantly, there is no need to manual participation in the testing process, except for the MR construction.

The generation of multiple‐path test cases can greatly enhance the efficiency of path‐wise testing. Various methods adopting meta‐heuristic algorithm to generate multiple‐path test cases have been proposed, but existing methods focus on improving the meta‐heuristic algorithm to get better test case generation efficiency, and test cases covering each path needs to be generated by meta‐heuristic algorithm searching. To improve efficiency, a test case generation method for multiple‐path coverage is proposed in this study, which combines a particle swarm optimisation (PSO) algorithm with metamorphic relations (MRs). The method first generates a test case using the PSO algorithm, and then generates new test cases by repeatedly using MRs between test cases. This method reduces evolving numbers of PSO algorithm. The experimental results show that the proposed method can significantly enhance the efficiency in terms of fitness evaluations and time consumption.