Abstract Botnets continue to pose a significant threat to internet-based information systems, with Domain Generation Algorithm (DGA) botnets being particularly challenging due to their dynamic and evasive behavior. Accurate classification of DGA domains is critical for identifying botnet families and deploying effective countermeasures. While deep learning models have shown promise in this area, recent studies indicate that conventional training approaches offer limited gains in classification performance, especially in multi-label scenarios involving a large number of classes. In this paper, we propose a supervised training strategy called Prioritized Selection Learning, which enhances model performance without altering the existing architecture. The method enables the model to focus on learning from more challenging or underperforming labels during an additional training phase, improving the model’s ability to distinguish between difficult cases. Experimental evaluations conducted on three widely used and reputable DGA datasets demonstrate that the proposed method consistently outperforms the baseline model in terms of accuracy, recall, and F1-score. Notably, the F1-score ranges from 88.0% to 94.4% across datasets, indicating strong generalization and robustness. The results suggest that the proposed training strategy is an effective and flexible solution for improving deep learning-based DGA botnet classification and other multi-label classification tasks.

Botnets are a frequent threat to information systems on the Internet, capable of launching denial-of-service attacks, spreading spam and malware on a large scale. Detecting and preventing botnets is very important in cybersecurity. Previous studies have suggested anomaly-based, signature-based, or HoneyNet-based botnet detection solutions. This paper presents new solutions for detecting and classifying families of Domain Generation Algorithm (DGA) botnets. Our solution can be applied in practice to disable botnets even if they have infected the computer. Our works help solve two problems, including binary classification and multiclass classification, specifically: (1) Determining whether a domain name is malicious or benign; (2) For malicious domains, identify their DGA botnet family. We proposed two deep learning models called LA_Bin07 and LA_Mul07 by combining the LSTM network and Attention layer. Our evaluation used the UMUDGA dataset recently published in 2020, with 50 DGA botnet families. The experimental results show that the LA_Bin07 and LA_Mul07 models solve the DGA botnets problem for binary and multiclass classification problems with very high accuracy.