Multiview learning (MVL) concentrates on the problem where each instance is represented by multiple different feature sets. Efficiently exploring and exploiting the common and complementary information among different views remains challenging in MVL. Nevertheless, many existing algorithms deal with multiview problems via pairwise strategies, which limit the exploration of relationships among different views and dramatically increase the computational cost. In this article, we propose a multiview structural large margin classifier (MvSLMC) that simultaneously satisfies the consensus and complementarity principles in all views. Specifically, on the one hand, MvSLMC employs a structural regularization term to promote cohesion within-class and separability between-class in each view. On the other hand, different views provide extra structural information to each other, which favors the diversity of the classifier. Moreover, the introduction of hinge loss in MvSLMC results in sample sparsity, which we leverage to construct a safe screening rule (SSR) for accelerating MvSLMC. To the best of our knowledge, this is the first attempt at safe screening in MVL. Numerical experimental results demonstrate the effectiveness of MvSLMC and its safe acceleration method.

Rank support vector machine (RSVM) is widely used in multilabel classification problems. However, as the number of labels and instances soars, the training efficiency of the model will be greatly reduced. Unfortunately, few effective methods can solve this problem. In this article, we propose a safe screening rule (SSR) for RSVM to improve its training speed. This is the first attempt to construct SSR for multilabel learning problems. SSR for RSVM can screen and delete most of the instances based on their relevant–irrelevant label pairs, which is the biggest difference in the existing SSR. After this process, the scale of RSVM can be substantially reduced before solving it. The sequential version of SSR for RSVM is further introduced to accelerate the whole parameter tuning process. One important advantage of SSR is that it is safe, which means we can obtain the same optimal solution as the original problem by utilizing it. Extensive experiments with five benchmark datasets, three large-scale datasets, and one type 2 diabetes dataset show that our approach is efficient and safe.

-regularized logistic regression (L1-LR) is popular for classification problems. To accelerate its training speed for high-dimensional data, techniques named safe screening rules have been proposed recently. They can safely delete the inactive features in data so as to greatly reduce the training cost of L1-LR. The screening power of these rules is determined by their corresponding safe regions, which is also the core technique of safe screening rules. In this paper, we introduce a new safe feature elimination rule (SFER) for L1-LR. Compared to existing safe rules, the safe region of SFER is improved in two aspects: (1) a smaller sphere region is constructed by using the strong convexity of dual L1-LR twice; (2) multiple half-spaces, which correspond to the potential active constraints, are added for further contraction. Both improvements can enhance the screening ability of SFER. As for the complexity of SFER, an iterative filtering framework is given by decomposing the safe region into multiple "domes". In this way, SFER admits a closed form solution and the identified features will not be scanned repeatedly. Experiments on ten benchmark data sets demonstrate that SFER gives superior performance than existing methods on training efficiency.

Weakly supervised learning has drawn considerable attention recently to reduce the expensive time and labor consumption of labeling massive data. In this paper, we investigate a novel weakly supervised learning problem of learning from similarity-confidence (Sconf) data, where only unlabeled data pairs equipped with confidence that illustrates their degree of similarity (two examples are similar if they belong to the same class) are needed for training a discriminative binary classifier. We propose an unbiased estimator of the classification risk that can be calculated from only Sconf data and show that the estimation error bound achieves the optimal convergence rate. To alleviate potential overfitting when flexible models are used, we further employ a risk correction scheme on the proposed risk estimator. Experimental results demonstrate the effectiveness of the proposed methods.

KSVCR is an effective algorithm to handle multiclass problems. But it cannot do variable selection and is time-consuming on large datasets. In this article, we propose a doubly sparse multiclass model DKSVCR which employs elastic net regularization term to improve the performance of KSVCR. And then, motivated by the sparsity of DKSVCR, a simultaneous safe feature and sample screening rule MFSS is further constructed to accelerate the solving speed of DKSVCR, which is termed as MFSS-DKSVCR. It has two major benefits. First, both classification and variable selection could be realized, the highly correlated features tend to be selected or removed together. Second, by using feature screening and sample screening rule alternatively rather than using each of them individually, our MFSS-DKSVCR can delete more redundant features and samples before the training stage. Hence, the solving speed is improved a lot. And our MFSS-DKSVCR is safe in the sense that the solutions obtained from the reduced problem and original problem are identical. Besides, a fast algorithm SDCA is used to solve the problem more efficiently. The experimental results on one artificial dataset, 28 benchmark datasets, and an image dataset verify the validity of our method.

To make support vector machine (SVM) applicable to large-scale data sets, safe screening rules are developed recently. The main idea is to reduce the scale of SVM by safely discarding the redundant training samples. Among existing safe screening rules, the dual screening method with variational inequalities (DVI) and the dynamic screening rule (DSR) based on duality gap are two representative strategies. DVI is efficient, while its safety may be affected by inaccurate solving algorithms. DSR is guaranteed to be safe; however, accurate feasible solutions are required for good efficiency. Based on the above-mentioned studies, in this paper, a two-stage screening (TSS) rule, which fully exploits the advantages of the above-mentioned two approaches and improves their shortcomings, is proposed. First, DVI is applied prior to training for sample screening. It reduces the scale of SVM and, meanwhile, produces a better initial feasible solution for DSR. Then, by embedding DSR into the solving algorithm, the solver becomes more accurate, and the safety of DVI can be strengthened. In the end, for safety guarantee, a postchecking step is added to search the wrongly identified samples and retrain them. To theoretically analyze the safety of DVI, an upper bound of the deviation in DVI is estimated, and a Safe-DVI is given based on it. To ensure the efficiency of TSS, the superiority of DVI over initial DSR is verified. In addition, kernel version of TSS is also given for nonlinear SVM. Numerical experiments on synthetic data sets and 12 real-world data sets verify the efficiency and safety of this TSS.

The twin support vector machine (TSVM) is widely used in classification problems, but it is not efficient enough for large-scale data sets. Furthermore, to get the optimal parameter, the exhaustive grid search method is applied to TSVM. It is very time-consuming, especially for multiparameter models. Although many techniques have been presented to solve these problems, all of them always affect the performance of TSVM to some extent. In this paper, we propose a safe screening rule (SSR) for linear-TSVM, and give a modified SSR (MSSR) for nonlinear TSVM, which contains multiple parameters. The SSR and MSSR can delete most training samples and reduce the scale of TSVM before solving it. Sequential versions of SSR and MSSR are further introduced to substantially accelerate the whole parameter tuning process. One important advantage of SSR and MSSR is that they are safe, i.e., we can obtain the same solution as the original problem by utilizing them. Experiments on eight real-world data sets and an imbalanced data set with different imbalanced ratios demonstrate the efficiency and safety of SSR and MSSR.

Twin support vector machine (TSVM) finds two nonparallel planes by solving a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one as in the conventional support vector machine (SVM); this makes the learning speed of TSVM approximately four times faster than that of the standard SVM. One major limitation of TSVM is that it involves an expensive matrix inverse operation when solving the dual problem. In addition, TSVM is less effective when dealing with the imbalanced data. In this paper, we propose a maximum margin of twin spheres support vector machine (MMTSSVM) for imbalanced data classification. MMTSSVM only needs to find two homocentric spheres. On one hand, the small sphere captures as many samples in the majority class as possible; on the other hand, the large sphere pushes out most samples in the minority class by increasing the margin between two homocentric spheres. MMTSSVM involves a QPP and a linear programming problem as opposed to a pair of QPPs as in classical TSVM or a larger-sized QPP in SVM, thus it greatly increases the computational speed. More importantly, MMTSSVM avoids the matrix inverse operation. The property of parameters in MMTSSVM is discussed and testified by one artificial experiment. Experimental results on nine benchmark datasets demonstrate the effectiveness of the proposed MMTSSVM in comparison with state-of-the-art algorithms. Finally, we apply MMTSSVM into Alzheimer's disease medical experiment and also obtain a better experimental result.

Twin support-vector machine (TSVM), which generates two nonparallel hyperplanes by solving a pair of smaller-sized quadratic programming problems (QPPs) instead of a single larger-sized QPP, works faster than the standard SVM, especially for the large-scale data sets. However, the traditional TSVM adopts hinge loss which easily leads to its sensitivity of the noise and instability for resampling. To enhance the performance of the TSVM, we present a novel TSVM with the pinball loss (Pin-TSVM) which deals with the quantile distance and is less sensitive to noise points. We further investigate its properties, including the noise insensitivity, between-class distance maximization, and within-class scatter minimization. In addition, we compare our Pin-TSVM with the twin parametric-margin SVM and the SVM with the pinball loss in theory. Numerical experiments on a synthetic data set and 14 benchmark data sets with different noises demonstrate the feasibility and validity of our proposed method.