The goal of data reduction is to remove noise features or samples from the original data and hence improve the performance of algorithms. Although Fisher-regularized support vector machine (FisherSVM) has good separability, FisherSVM lacks sparsity to reduce noise data. To alleviate this issue, we propose a novel sparse support vector machine with Fisher-regularizer (SF-SVM). The$L_{1}$-norm of model coefficients and the Fisher-regularizer term used in SF-SVM can induce a sparse model and improve the generalization performance, respectively. Besides, the proposed SF-SVM includes two versions: linear SF-SVM and kernel SF-SVM, which can perform feature and sample reduction with favorable sparsity, respectively. Experimental results on seven UCI datasets show that the proposed SF-SVM has a satisfactory data reduction ability and good classification performance.

Fisher-regularized support vector machine (Fish-erSVM) can approximatively fulfill the Fisher criterion and obtain good statistical separability, which is a combined method of the support vector machine and Fisher discriminant analysis. However, the hinge loss function is related to the shortest distance between two-class sets, and FisherSVM may be hence sensitive to noise. To remedy it, the pinball loss function, which is related to the quantile distance, is introduced into FisherSVM and then a Fisher-regularized support vector machine with pinball loss function (Pin-FisherSVM) is proposed, which combines the noise insensitivity of the pinball loss function with the statistical separability of FisherSVM well. Pin-FisherSVM can be cast as a quadratic programming, which results a globally optimal solution. Experimental results on artificial and real-world datasets demonstrate that our proposed method is insensitive to label noise or feature noise. Compared to FisherSVM, the proposed Pin-FisherSVM has the same computational complexity and exhibits superior noise insensitivity.