This article addresses the pressing need for accurate anomaly detection techniques, particularly for cybersecurity applications. We emphasize the effectiveness of ensemble and machine learning techniques, as well as the parallelizability of the Unite and Conquer approach, to improve the efficiency, speed, and accuracy of calculations. More precisely, we introduce a variant of an existing framework for its optimization by taking into account the asynchronicity of the communications and evaluate its large-scale performance on the Fugaku supercomputer. Our evaluation focuses on the detection rate and response time of expertise using extensive datasets, including the UNSW-NB15 dataset, in the cybersecurity domain. Additionally, we discuss the framework’s expanded functionality and its potential integration into existing Security Orchestration, Automation, and Response (SOAR) systems, thereby strengthening cyber threat detection and response capabilities.

Introducing the Scalable Anomaly Detection with UC2B framework, this paper addresses the critical task of identifying unusual patterns in data, crucial for effective cyber threat defense. By leveraging ensemble learning methods and the parallel processing capabilities of the Unite and Conquer approach, the framework demonstrates its proficiency in handling large datasets. It strives to offer computational efficiency, scalability, and high accuracy in real-world applications. Notably, this paper places special emphasis on the diversity of components and acknowledges their substantial influence on the overall framework functionality. It encompasses features such as fault tolerance, adaptability to various architectures, and efficient load balancing. Experimental validation on the Ruche Cluster within the realm of cybersecurity provides valuable insights into its potential in detecting anomalies.

Deep learning (DL) requires high-performance processing on big data. Graph Neural Networks, a challenging topic in DL using linear algebra methods, need algorithmic solutions to efficiently assign and process graph data on modern distributed and parallel machines, which are considered with mixed arithmetic and various types of tensor/matrix accelerators. Determining compression techniques for the graph’s sparse data structures is one of the key elements.Our first objective is to design and implement a reusable parallel numerical library to resolve large neural network graphs. Our design strategy is drawn on a component-based approach and targets maximum code reuse in various parallel contexts while allowing for performance optimization. The solution could be later integrated into a DL framework like MindSpore.

Graph Neural Networks (GNNs) play a very important role today. It does analyze not only the graph data itself, but also the data connectivity of the graph. The quality of a GNN is thus altered by the result of extracted graph structure information. The extraction could be enhanced by GNN model design or directly from the training dataset with a GNN-decoupled method. In this paper, we propose RankedDrop, a new sampling method to improve the extraction of graph structure information. This approach is based on droppingout technique, and it adopts a spatial-aware selection of edges to drop. It takes into account structure information of the graph to control the dropping-out, and its random selection of edges to be dropped is under the control of a probability generated with respect to graph’s topological importance. Our experiments point out that RankedDrop provides high-quality and robust training results compared to the leading solutions. Furthermore, RankedDrop could be a framework plugin and combined with GNN model improvements to maximize GNN quality. Furthermore, RankedDrop could be a plugin for AI frameworks like MindSpore and combined with GNN model improvements to maximize GNN quality.