As Android apps become increasingly dependent on network services, Network-Related Faults (NRFs) are gradually more prevalent and severely degrade user experience. These faults are typically scattered across apps and require complex, often non-trivial network patterns to trigger, which makes their detection challenging. To date, we still lack a general and in-depth understanding of NRFs in real-world Android apps. To fill this gap, we conduct the first empirical study on 154 real-world network-related bugs collected from 42 diverse, representative Android apps, investigating their characteristics, influences, triggering patterns, and origins. Our study reveals several notable findings and practical implications to guide future research on detecting and mitigating NRFs. Motivated by the empirical results and the limitations of existing Android testing approaches—namely, the lack of targeted network events and efficient injection mechanisms—we propose NATE, a novel Network-Aware Testing Enhancer that augments existing general Android testing approaches for NRF detection. NATE leverages curiosity-driven reinforcement learning to provide network-aware guidance and to inject effective network events, enabling testing approaches to explore network-related extra app functionalities and detect NRFs. When integrated with two state-of-the-art general Android testing approaches, experiments conducted on 12 large, active apps demonstrate the effectiveness and efficiency of NATE, with 1.7-5.7× as many faults detected, as well as 8.8% and 12.5% more code covered. Among the network-related faults detected by NATE, 21 have been explicitly confirmed as real-world bugs by the developers (six of which have already been fixed), where 16 of them were first reported by NATE. Notably, none of the 21 bugs were detected by the original general testing approaches, demonstrating the unique contributions of NATE.

As the scale and complexity of Android applications continue to grow in response to increasing market and user demands, quality assurance challenges become more significant. While previous studies have demonstrated the superiority of Reinforcement Learning (RL) in Android GUI testing, its effectiveness remains limited, particularly in large, complex apps. This limitation arises from the ineffectiveness of Tabular RL in learning the knowledge within the large state-action space of the App Under Test (AUT) and from the suboptimal utilization of the acquired knowledge when employing more advanced RL techniques. To address such limitations, this paper presents DQT, a novel automated Android GUI testing approach based on deep reinforcement learning. DQT preserves widgets' structural and semantic information with graph embedding techniques, building a robust foundation for identifying similar states or actions and distinguishing different ones. Moreover, a specially designed Deep Q-Network (DQN) effectively guides curiosity-driven exploration by learning testing knowledge from runtime interactions with the AUT and sharing it across states or actions. Experiments conducted on 30 diverse open-source apps demonstrate that DQT outperforms existing state-of-the-art testing approaches in both code coverage and fault detection, particularly for large, complex apps. The faults detected by DQT have been reproduced and reported to developers; so far, 21 of the reported issues have been explicitly confirmed, and 14 have been fixed.

The prominent role of mobile apps in daily life has underscored the need for robust quality assurance, leading to the development of various automated Android Graphical User Interface (GUI) testing approaches. Code coverage and fault detection are two primary metrics for evaluating the effectiveness of these testing approaches. However, conducting a reliable and robust evaluation based on the two metrics remains challenging, due to the imperfections of the current evaluation system, with a tangle of numerous metric granularities and the interference of multiple nondeterminism in tests. For instance, the evaluation solely based on the mean or total numbers of detected faults lacks statistical robustness, resulting in numerous conflicting conclusions that impede the comprehensive understanding of stakeholders involved in Android testing, thereby hindering the advancement of Android testing methodologies. To mitigate such issues, this paper presents the first comprehensive statistical study of existing Android GUI testing metrics, involving extensive experiments with 8 state-of-the-art testing approaches on 42 diverse apps, examining aspects including statistical significance, correlation, and variation. Our study focuses on two primary areas: (1) The statistical significance and correlation between test metrics and among different metric granularities. (2) The influence of test randomness and test convergence on evaluation results of test metrics. By employing statistical analysis to account for the considerable influence of randomness, we achieve notable findings: (1) Instruction, Executable Lines Of Code (ELOC), and method coverage demonstrate notable consistency across both significance evaluation and mean value evaluation, whereas the evaluation on Fatal Errors compared to Core Vitals, as well as all errors versus the well-selected errors, reveals a similarly high level of consistency. (2) There are evident inconsistencies in the code coverage and fault detection results, indicating both two metrics should be considered for comprehensive evaluation. (3) Code coverage typically exhibits greater stability and robustness in evaluation compared to fault detection, whereas fault detection is quite unstable even with the maximum test rounds ever used in previous research studies. (4) A moderate test duration is sufficient for most approaches to showcase their comprehensive overall effectiveness on most apps in both code coverage and fault detection, indicating the possibility of adopting a moderate test duration to draw preliminary conclusions in Android testing development. These findings inform practical recommendations and support our proposal of an effective framework to enhance future mobile testing evaluations.