In distributed machine learning scenarios, the difference in data distribution among different nodes is a key issue that cannot be ignored. However, existing methods make it difficult to autonomously adjust model parameters for dynamically changing data distributions, leading to inflexible global decision boundaries with insufficient local adaptation. To address this problem, we propose a distributed hybrid support vector machine (SVM) based on the adaptive ensemble selection of regularization parameters, DHS-AE. The model utilizes the data structure information to cut the data space and thus identify data distribution characteristics. The SVM, integrated with regularization parameters that are adaptively determined within specific ranges, is utilized in the local subspace to enable real-time adjustment of decision boundaries in response to distribution changes, thereby further reducing the computational overhead. The generalization bound of DHS-AE is theoretically established using covering numbers, and the fast convergence speed and consistency are derived. In practical applications, we verify the excellent performance of the DHS-AE using a large number of real datasets.

Fuzzy logic-based graph neural networks (FL-GNN) have recently garnered growing attention in node classification, which aims to enhance the ability of GNN in modeling uncertain relationships between nodes. However, existing FL-GNN typically assume that nodes in the source domain (training set) and target domain (test set) follow the identical data distribution and class sets. Real-world scenarios often exhibit significant distribution shifts and target domain even contains classes that were not present in the source domain, termed open-set cross-domain node classification (OSCD-NC), which seriously damages their superior performance. Thus, how to leverage the strong uncertain knowledge representation capacity of FL-GNN to learn a well-defined boundary between seen and unseen classes for improving OSCD-NC performance remains an open and underexplored research problem. In this paper, we propose an effective domain-adaptive fuzzy graph diffusion network (DFGDN) for OSCD-NC. Specifically, with the help of a fuzzy adjacency matrix, fuzzy graph diffusion networks are proposed to generate robust fuzzy node representations by adaptively enhancing feature collaboration between low-pass and high-pass graph filters. Then, a peer ($M$+1)-class classifier is introduced to learn a rough class boundary by measuring their class prediction probability difference for target domain. After that, the ($M$+1)-means clustering and decoder modules are simultaneously designed to discover more supervision guidance from target domain for learned class boundary optimization. Finally, we jointly optimize the above modules in an adversarial manner via classification loss, classifier discrepancy loss and mean squared error loss, which further improves the accuracy of the learned class boundary by pulling seen nodes from the source domain and target domain closer, and pushing unseen nodes away. Extensive experiments on three cross-domain data pairs and various openness rates demonstrate the effectiveness of the proposed DFGDN framework.

Distributed machine learning provides an efficient solution for large-scale data processing through parallel computing. However, current distributed learning relies on global or local paradigms and cannot adaptively adjust decision boundaries in complex data environments. To address this problem, we propose a Distributed Hybrid Learning algorithm based on Fisher Linear Discriminant (DHL-FLD). Specifically, DHL-FLD consists of a global pre-learning phase and a subspace local learning phase. On the one hand, the global pre-learning phase is designed to divide the data space, which can obtain the data structure information. On the other hand, the local learning phase dynamically adjusts and optimizes the decision boundaries, guided by the structural information and distributional properties of the data. Theoretically, we establish the generalization bound of DHL-FLD using the integral operator technique and verify the scalability and robustness of DHL-FLD. The effectiveness of DHL-FLD is demonstrated through extensive experiments on real datasets.

Unsupervised graph-structure learning (GSL) which aims to learn an effective graph structure applied to arbitrary downstream tasks by data itself without any labels' guidance, has recently received increasing attention in various real applications. Although several existing unsupervised GSL has achieved superior performance in different graph analytical tasks, how to utilize the popular graph masked autoencoder to sufficiently acquire effective supervision information from the data itself for improving the effectiveness of learned graph structure has been not effectively explored so far. To tackle the above issue, we present a multilevel contrastive graph masked autoencoder (MCGMAE) for unsupervised GSL. Specifically, we first introduce a graph masked autoencoder with the dual feature masking strategy to reconstruct the same input graph-structured data under the original structure generated by the data itself and learned graph-structure scenarios, respectively. And then, the inter- and intra-class contrastive loss is introduced to maximize the mutual information in feature and graph-structure reconstruction levels simultaneously. More importantly, the above inter- and intra-class contrastive loss is also applied to the graph encoder module for further strengthening their agreement at the feature-encoder level. In comparison to the existing unsupervised GSL, our proposed MCGMAE can effectively improve the training robustness of the unsupervised GSL via different-level supervision information from the data itself. Extensive experiments on three graph analytical tasks and eight datasets validate the effectiveness of the proposed MCGMAE.

Sparse additive machines (SAMs) have shown competitive performance on variable selection and classification in high-dimensional data due to their representation flexibility and interpretability. However, the existing methods often employ the unbounded or nonsmooth functions as the surrogates of 0-1 classification loss, which may encounter the degraded performance for data with outliers. To alleviate this problem, we propose a robust classification method, named SAM with the correntropy-induced loss (CSAM), by integrating the correntropy-induced loss (C-loss), the data-dependent hypothesis space, and the weighted -norm regularizer ( ) into additive machines. In theory, the generalization error bound is estimated via a novel error decomposition and the concentration estimation techniques, which shows that the convergence rate can be achieved under proper parameter conditions. In addition, the theoretical guarantee on variable selection consistency is analyzed. Experimental evaluations on both synthetic and real-world datasets consistently validate the effectiveness and robustness of the proposed approach.

Recently emerged label noise-resistant graph representation learning (LNR-GRL) has received increasing attention, which aims to enhance the generalization of graph neural networks (GNNs) in semi-supervised node classification with noisy and limited labels. Most of the existing LNR-GRL tend to introduce more complex sample selection strategies developed in nongraph areas to distinguish more noisy nodes to alleviate their misguidance. However, these proposed methods neglect the importance of inaccurate graph structure relationships rectification, and information collaboration between inaccurate graph structure relationships and noisy node label rectification in improving the quality of noisy node identification and its rectified node labels. To solve the above-mentioned issues, we propose a novel multiplex experts governance collaboration (MEGC) framework for LNR-GRL. Specifically, an unsupervised graph structure governance expert is first designed to rectify inaccurate graph structure relationships. Based on the rectified graph structure, a simple label noise governance expert is proposed to accurately identify noisy node labels and further improve the quality of noisy nodes’ rectified labels and unlabeled nodes’ pseudo-labels. Finally, the above-proposed governance experts can be effectively combined with GNNs to jointly guide their training via the introduced cross-view graph contrastive loss and cross-entropy loss, which can maximally limit the effect of noisy node labels and discover more effective supervision guidance from data itself for GNNs optimization. Extensive experiments on three benchmarks, two label noise types, four noise rates, and four training label rates demonstrate the superiority of the proposed method in comparison to the existing LNR-GRL methods.

Abstract Malware classification has been successful in utilizing machine learning methods. However, it is limited by the reliance on a large number of high-quality labeled datasets and the issue of overfitting. These limitations hinder the accurate classification of advanced malware with only a few samples available. Meta-learning methods offer a solution by allowing models to quickly adapt to new tasks, even with a small number of samples. However, the effectiveness of meta-learning approaches in malware classification varies due to the diverse nature of malware types. Most meta-learning-based methodologies for malware classification either focus solely on data augmentation or utilize existing neural networks and learning rate schedules to adapt to the meta-learning model. These approaches do not consider the integration of both processes or tailor the neural network and learning rate schedules to the specific task. As a result, the classification performance and generalization capabilities are suboptimal. In this paper, we propose a multi-improved model-agnostic meta-learning (MI-MAML) model that aims to address the challenges encountered in few-shot malware classification. Specifically, we propose two data augmentation techniques to improve the classification performance of few-shot malware. These techniques involve utilizing grayscale images and the Lab color space. Additionally, we customize neural network architectures and learning rate schemes based on the representative few-shot classification method, MAML, to further enhance the model’s classification performance and generalization ability for the task of few-shot malware classification. The results obtained from multiple few-shot malware datasets demonstrate that MI-MAML outperforms other models in terms of categorical accuracy, precision, and f1-score. Furthermore, we have conducted ablation experiments to validate the effectiveness of each stage of our work.

The divide-and-conquer strategy is a very effective method of dealing with big data. Noisy samples in big data usually have a great impact on algorithmic performance. In this article, we introduce Markov sampling and different weights for distributed learning with the classical support vector machine (cSVM). We first estimate the generalization error of weighted distributed cSVM algorithm with uniformly ergodic Markov chain (u.e.M.c.) samples and obtain its optimal convergence rate. As applications, we obtain the generalization bounds of weighted distributed cSVM with strong mixing observations and independent and identically distributed (i.i.d.) samples, respectively. We also propose a novel weighted distributed cSVM based on Markov sampling (DM-cSVM). The numerical studies of benchmark datasets show that the DM-cSVM algorithm not only has better performance but also has less total time of sampling and training compared to other distributed algorithms.

In this article we consider Incremental Fisher linear discriminant (IFLD) based on data denoising. The data denoising is completed by Markov sampling such that the generated non-noise sample sequence is an uniformly ergodic Markov chain (u.e.M.c.). We first establish the generalization bounds of IFLD with u.e.M.c. samples, and prove that the IFLD algorithm with u.e.M.c. samples is consistent. We also present two new IFLD classification algorithms based on Markov sampling, IFLD based on Markov sampling (IFLD-MS) and improved IFLD based on Markov sampling (IIFLD-MS). Experimental results of benchmark repository suggest that IFLD-MS and IIFLD-MS have better performance than the classical IFLD, the incremental support vector machine (ISVM) and other IFLD algorithms.

Incremental learning is one of the most effective methods of learning accumulated data and large-scale data. The newly increased samples of the previously known works on incremental learning are usually independent and identically distributed. To study how dependent sampling methods influence the learning ability of incremental support vector machines (ISVM) algorithm, in this paper we introduce an ISVM based on Markov resampling (MR-ISVM), and give the experimental research on the learning ability of the MR-ISVM algorithm. The experimental results indicate that the MR-ISVM algorithm has not only smaller misclassification rates and sparser of the obtained classifiers, but also less total time of sampling and training compared to ISVM based on randomly independent sampling. We also compare it with other ISVM algorithms.

Big data research has become a globally hot topic in recent years. One of the core problems in big data learning is how to extract effective information from the huge data. In this paper, we propose a Markov resampling algorithm to draw useful samples for handling coefficient-based regularized regression (CBRR) problem. The proposed Markov resampling algorithm is a selective sampling method, which can automatically select uniformly ergodic Markov chain (u.e.M.c.) samples according to transition probabilities. Based on u.e.M.c. samples, we analyze the theoretical performance of CBRR algorithm and generalize the existing results on independent and identically distributed observations. To be specific, when the kernel is infinitely differentiable, the learning rate depending on the sample size $m$ can be arbitrarily close to $\mathcal {O}(m^{-1})$ under a mild regularity condition on the regression function. The good generalization ability of the proposed method is validated by experiments on simulated and real data sets.

Support vector machine (SVM) is one of the most widely used learning algorithms for classification problems. Although SVM has good performance in practical applications, it has high algorithmic complexity as the size of training samples is large. In this paper, we introduce SVM classification (SVMC) algorithm based on -times Markov sampling and present the numerical studies on the learning performance of SVMC with -times Markov sampling for benchmark data sets. The experimental results show that the SVMC algorithm with -times Markov sampling not only have smaller misclassification rates, less time of sampling and training, but also the obtained classifier is more sparse compared with the classical SVMC and the previously known SVMC algorithm based on Markov sampling. We also give some discussions on the performance of SVMC with -times Markov sampling for the case of unbalanced training samples and large-scale training samples.

Learning with l1 -regularizer has brought about a great deal of research in learning theory community. Previous known results for the learning with l1 -regularizer are based on the assumption that samples are independent and identically distributed (i.i.d.), and the best obtained learning rate for the l1 -regularization type algorithms is O(1/√m) , where m is the samples size. This paper goes beyond the classic i.i.d. framework and investigates the generalization performance of least square regression with l1 -regularizer ( l1 -LSR) based on uniformly ergodic Markov chain (u.e.M.c) samples. On the theoretical side, we prove that the learning rate of l1 -LSR for u.e.M.c samples l1 -LSR(M) is with the order of O(1/m) , which is faster than O(1/√m) for the i.i.d. counterpart. On the practical side, we propose an algorithm based on resampling scheme to generate u.e.M.c samples. We show that the proposed l1 -LSR(M) improves on the l1 -LSR(i.i.d.) in generalization error at the low cost of u.e.M.c resampling.

UNLABELLED: The previously known works studying the generalization ability of support vector machine classification (SVMC) algorithm are usually based on the assumption of independent and identically distributed samples. In this paper, we go far beyond this classical framework by studying the generalization ability of SVMC based on uniformly ergodic Markov chain (u.e.M.c.) samples. We analyze the excess misclassification error of SVMC based on u.e.M.c. samples, and obtain the optimal learning rate of SVMC for u.e.M.c. SAMPLES: We also introduce a new Markov sampling algorithm for SVMC to generate u.e.M.c. samples from given dataset, and present the numerical studies on the learning performance of SVMC based on Markov sampling for benchmark datasets. The numerical studies show that the SVMC based on Markov sampling not only has better generalization ability as the number of training samples are bigger, but also the classifiers based on Markov sampling are sparsity when the size of dataset is bigger with regard to the input dimension.

In this paper, we consider online support vector machine (SVM) classification learning algorithms with uniformly ergodic Markov chain (u.e.M.c.) samples. We establish the bound on the misclassification error of an online SVM classification algorithm with u.e.M.c. samples based on reproducing kernel Hilbert spaces and obtain a satisfactory convergence rate. We also introduce a novel online SVM classification algorithm based on Markov sampling, and present the numerical studies on the learning ability of online SVM classification based on Markov sampling for benchmark repository. The numerical studies show that the learning performance of the online SVM classification algorithm based on Markov sampling is better than that of classical online SVM classification based on random sampling as the size of training samples is larger.

This paper considers the generalization ability of two regularized regression algorithms [least square regularized regression (LSRR) and support vector machine regression (SVMR)] based on non-independent and identically distributed (non-i.i.d.) samples. Different from the previously known works for non-i.i.d. samples, in this paper, we research the generalization bounds of two regularized regression algorithms based on uniformly ergodic Markov chain (u.e.M.c.) samples. Inspired by the idea from Markov chain Monto Carlo (MCMC) methods, we also introduce a new Markov sampling algorithm for regression to generate u.e.M.c. samples from a given dataset, and then, we present the numerical studies on the learning performance of LSRR and SVMR based on Markov sampling, respectively. The experimental results show that LSRR and SVMR based on Markov sampling can present obviously smaller mean square errors and smaller variances compared to random sampling.

Fisher linear discriminant (FLD) is a well-known method for dimensionality reduction and classification that projects high-dimensional data onto a low-dimensional space where the data achieves maximum class separability. The previous works describing the generalization ability of FLD have usually been based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper, we go far beyond this classical framework by studying the generalization ability of FLD based on Markov sampling. We first establish the bounds on the generalization performance of FLD based on uniformly ergodic Markov chain (u.e.M.c.) samples, and prove that FLD based on u.e.M.c. samples is consistent. By following the enlightening idea from Markov chain Monto Carlo methods, we also introduce a Markov sampling algorithm for FLD to generate u.e.M.c. samples from a given data of finite size. Through simulation studies and numerical studies on benchmark repository using FLD, we find that FLD based on u.e.M.c. samples generated by Markov sampling can provide smaller misclassification rates compared to i.i.d. samples.