The widespread use of smartphones in daily life has raised concerns about privacy and security among researchers and practitioners. Privacy issues are generally highly prevalent in mobile applications, particularly targeting the Android platform - the most popular mobile operating system. For this reason, several techniques have been proposed to identify malicious behavior in Android applications, including the Mining Android Sandbox approach (MAS approach), which aims to identify malicious behavior in repackaged Android applications (apps). However, previous empirical studies evaluated the MAS approach using a small dataset consisting of only 102 pairs of original and repackaged apps. This limitation raises questions about the external validity of their findings and whether the MAS approach can be generalized to larger datasets. To address these concerns, this paper presents the results of a replication study focused on evaluating the performance of the MAS approach regarding its capabilities of correctly classifying malware from different families. Unlike previous studies, our research employs a dataset that is an order of magnitude larger, comprising 4,076 pairs of apps covering a more diverse range of Android malware families. Surprisingly, our findings indicate a poor performance of the MAS approach for identifying malware, with the F1-score decreasing from 0.90 for the small dataset used in the previous studies to 0.54 in our more extensive dataset. Upon closer examination, we discovered that certain malware families partially account for the low accuracy of the MAS approach, which fails to classify a repackaged version of an app as malware correctly. Our findings highlight the limitations of the MAS approach, particularly when scaled, and underscore the importance of complementing it with other techniques to detect a broader range of malware effectively. This opens avenues for further discussion on addressing the blind spots that affect the accuracy of the MAS approach.

Due to the popularization of Android and the full range of applications (apps) targeting this platform, many security issues have emerged, attracting researchers and practitioners' attention. As such, many techniques for addressing security Android issues have emerged, including approaches for mining sandboxes using dynamic analysis tools (i.e., automated testing tools). Undoubtedly, the resulting sandboxes' efficiency depends on the test case generation tools used in the mining procedures. Previous research studies have compared Android test case generation tools for this specific goal. However, it is difficult to increment the research in this field because reproducing these previous empirical studies is a challenging and time-consuming task. This difficulty occurs because it is necessary to integrate test generation tools that often require different and conflicting versions of the Android platform, programming languages (e.g., Python 2 and Python 3), and software libraries. To mitigate this issue, in this paper we present DroidXP, a software infrastructure that allows researchers (and tools developers) to integrate and compare test case generation tools for mining sandboxes. We evaluated DroidXP through a reproduction study of previous research work, though considering additional test case generation tools. Our experiment suggests that DroidXP simplifies the comparison of existing tools for mining sandboxes, and revealed that Sapienz outperforms the other test case generation tools-regardless of the Monkey tool had presented the highest code coverage in our study.