Recent years have witnessed the proliferation of learning-based Android malware detectors. These detectors can be categorized into three types, String-based, Image-based and Graph-based. Most of them have achieved good detection performance under the ideal setting. In reality, however, detectors often face out-of-distribution samples due to the factors such as code obfuscation, concept drift (e.g., software development technique evolution and new malware category emergence), and adversarial examples (AEs). This problem has attracted increasing attention, but there is a lack of comparative studies that evaluate the existing various types of detectors under these challenging environments. In order to fill this gap, we select 12 representative detectors from three types of detectors, and evaluate them in the challenging scenarios involving code obfuscation, concept drift and AEs, respectively. Experimental results reveal that none of the evaluated detectors can maintain their ideal-setting detection performance, and the performance of different types of detectors varies significantly under various challenging environments. We identify several factors contributing to the performance deterioration of detectors, including the limitations of feature extraction methods and learning models. We also analyze the reasons why the detectors of different types show significant performance differences when facing code obfuscation, concept drift and AEs. Finally, we provide practical suggestions from the perspectives of users and researchers, respectively. We hope our work can help understand the detectors of different types, and provide guidance for enhancing their performance and robustness.

Existing Android malware detection methods are usually hard to simultaneously resist various obfuscation techniques. Therefore, bytecode-based code obfuscation becomes an effective means to circumvent Android malware analysis. Building obfuscation-resilient Android malware analysis methods is a challenging task, due to the fact that various obfuscation techniques have vastly different effects on code and detection features. To mitigate this problem, we propose combining multiple features that are complementary in combating code obfuscation. Accordingly, we develop an obfuscation-resilient Android malware analysis methodCorDroid, based on two new features: Enhanced Sensitive Function Call Graph (E-SFCG) and Opcode-based Markov transition Matrix (OMM). The first describes sensitive function call relationships, while the second reflects transition probabilities among opcodes. Combining E-SFCG and OMM can well characterize the runtime behavior of Android apps from different perspectives, hence increasing the difficulty of misleading malware analysis through using code obfuscation to affect detection features. To evaluate CorDroid, we generate 74,138 obfuscated samples with 14 different obfuscation techniques, and compare CorDroid with the state-of-the-art detection methods (e.g., MaMaDroid, RevealDroid and APIGraph). In terms of average F1-Score, CorDroid is 29.69% higher than MaMaDroid, 21.80% higher than APIGraph, and 9.71% higher than RevealDroid, respectively. Experiments also validate the complementarity between E-SFCG and OMM, and exhibit the high execution efficiency of CorDroid.