With the rise of large language models, such as ChatGPT, non-decisional models have been applied to various tasks. Moreover, ChatGPT has drawn attention to the traditional decision-centric task of Android malware detection. Despite effective detection methods proposed by scholars, they face low interpretability issues. Specifically, while these methods excel in classifying applications as benign or malicious and can detect malicious behavior, they often fail to provide detailed explanations for the decisions they make. This challenge raises concerns about the reliability of existing detection schemes and questions their true ability to understand complex data. In this study, we investigate the influence of the non-decisional model, ChatGPT, on the traditional decision-centric task of Android malware detection. We choose three state-of-the-art solutions, Drebin , \(XM_{AL}\) , and MaMaDroid , conduct a series of experiments on publicly available datasets, and carry out a comprehensive comparison and analysis. Our findings indicate that these decision-driven solutions primarily rely on statistical patterns within datasets to make decisions, rather than genuinely understanding the underlying data. In contrast, ChatGPT, as a non-decisional model, excels in providing comprehensive analysis reports, substantially enhancing interpretability. Furthermore, we conduct surveys among experienced developers. The result highlights developers’ preference for ChatGPT, as it offers in-depth insights and enhances efficiency and understanding of challenges. Meanwhile, these studies and analyses offer profound insights, presenting developers with a novel perspective on Android malware detection—enhancing the reliability of detection results from a non-decisional perspective.

With the emergence of smartphones, Android has become a widely used mobile operating system. However, it is vulnerable when encountering various types of attacks. Every day, new malware threatens the security of users’ devices and private data. Many methods have been proposed to classify malicious applications, utilizing static or dynamic analysis for classification. However, previous methods still suffer from unsatisfactory performance due to two challenges. First, they are unable to address the imbalanced data distribution problem, leading to poor performance for malware families with few members. Second, they are unable to address the zero-day malware (zero-day malware refers to malicious applications that exploit unknown vulnerabilities) classification problem. In this article, we introduce an innovative meta -learning approach for m ulti-family A ndroid m alware c lassification named Meta-MAMC , which uses meta-learning technology to learn meta-knowledge (i.e., the similarities and differences among different malware families) of few-family samples and combines new sampling algorithms to solve the above challenges. Meta-MAMC integrates (i) the meta-knowledge contained within the dataset to guide models in learning to identify unknown malware; and (ii) more accurate and diverse tasks based on novel sampling strategies, as well as directly adapting meta-learning to a new few-sample and zero-sample task to classify families. We have evaluated Meta-MAMC on two popular datasets and a corpus of real-world Android applications. The results demonstrate its efficacy in accurately classifying malicious applications belonging to certain malware families, even achieving 100% classification in some families.

In recent years, the performance of machine translation systems has made remarkable progress, primarily due to the rapid advancements in neural network language models. These state-of-the-art models enable the swift translation of vast amounts of text, leading to considerable time and cost savings. In pursuit of enhancing machine translation accuracy, researchers have devoted attention to developing automated translation testing tools. A prominent approach in this context involves comparing the translation results of “similar” source sentences, anticipating the correctness of translation by similarities in sentence structure. However, despite the potential of this approach, the current studies still face certain challenges. Notably, false negatives and false positives persist as issues. Moreover, achieving high detection accuracy for all types of translation errors remains an ongoing challenge. To address these challenges, we propose the StructureTester, a novel approach that not only leverages the differences between the structure trees of two sentences but also employs changes in sentence purpose as crucial judgmental features. Our proposed method yields significant improvements, elevating the overall detection accuracy to an impressive 98.17%. Furthermore, StructureTester effectively identifies various types of translation errors.

Abstract Each malware application belongs to a specific malware family, and each family has unique characteristics. However, existing Android malware detection schemes do not pay attention to the use of malware family information. If the family information is exploited well, it could improve the accuracy of malware detection. In this paper, we propose a general Ensemble framework combining Family Information for Android Malware Detector, called EFIMDetector. First, eight categories of features are extracted from Android application packages. Then, we define the malware family with a large sample size as a prosperous family and construct a classifier for each prosperous family as a conspicuousness evaluator for the family characteristics. These conspicuousness evaluators are combined with a general classifier (which can be a base or ensemble classifier in itself), called the final classifier, to form a two-layer ensemble framework. For the samples of prosperous families with conspicuous family characteristics, the conspicuousness evaluators directly provide detection results. For other samples (including the samples of prosperous families with nonconspicuous family characteristics and the samples of nonprosperous families), the final classifier is responsible for detection. Seven common base classifiers and three common ensemble classifiers are used to detect malware in the experiment. The results show that the proposed ensemble framework can effectively improve the detection accuracy of these classifiers.

Artificial intelligence (AI) for software engineering (SE) tasks has recently achieved promising performance. In this article, we investigate to what extent the pre-trained language model truly understands those SE tasks such as code search, code summarization, etc. We conduct a comprehensive empirical study on a board set of AI for SE (AI4SE) tasks by feeding them with variant inputs: 1) with various masking rates and 2) with sufficient input subset method. Then, the trained models are evaluated on different SE tasks, including code search, code summarization, and duplicate bug report detection. Our experimental results show that pre-trained language models are insensitive to the given input, thus they achieve similar performance in these three SE tasks. We refer to this phenomenon asoverinterpretation, where a model confidently makes a decision without salient features, or where a model finds some irrelevant relationships between the final decision and the dataset. Our study investigates two approaches to mitigate the overinterpretation phenomenon: whole word mask strategy and ensembling. To the best of our knowledge, we are thefirstto reveal this overinterpretation phenomenon to the AI4SE community, which is an important reminder for researchers to design the input for the models and calls for necessary future work in understanding and implementing AI4SE tasks.