Image denoising is an appealing and challenging task, in that noise statistics of real-world observations may vary with local image contents and different image channels. Specifically, the green channel usually has twice the sampling rate in raw data. To handle noise variances and leverage such channel-wise prior information, we propose a simple and effective green channel prior-based image denoising (GCP-ID) method, which integrates GCP into the classic patch-based denoising framework. Briefly, we exploit the green channel to guide the search for similar patches, which aims to improve the patch grouping quality and encourage sparsity in the transform domain. The grouped image patches are then reformulated into RGGB arrays to explicitly characterize the density of green samples. Furthermore, to enhance the adaptivity of GCP-ID to various image contents, we cast the noise estimation problem into a classification task and train an effective estimator based on convolutional neural networks (CNNs). Experiments on real-world datasets demonstrate the competitive performance of the proposed GCP-ID method for image and video denoising applications in both raw and sRGB spaces. Our code is available at https://github.com/ZhaomingKong/GCP-ID.

Unsupervised feature selection is an important tool in data mining, machine learning, and pattern recognition. Although data labels are often missing, the number of data classes can be known and exploited in many scenarios. Therefore, a structured graph, whose number of connected components is identical to the number of data classes, has been proposed and is frequently applied in unsupervised feature selection. However, methods based on the structured graph learning face two problems. First, their structured graphs are not always guaranteed to maintain the same number of connected components as the data classes with existing optimization algorithms. Second, they usually lack strategies for choosing moderate hyperparameters. To solve these problems, an efficient and stable unsupervised feature selection method based on a novel structured graph and data discrepancy learning (ESUFS) is proposed. Specifically, the novel structured graph, consisting of a pairwise data similarity matrix and an indicator matrix, can be efficiently learned by solving a discrete optimization problem. Data discrepancy learning focuses on features that maximize the difference among data and helps in selecting discriminative features. Extensive experiments conducted on various datasets show that ESUFS outperforms state-of-the-art methods not only in accuracy (ACC) but also in stability and speed.

Unsupervised feature selection is challenging in machine learning, pattern recognition, and data mining. The crucial difficulty is to learn a moderate subspace that preserves the intrinsic structure and to find uncorrelated or independent features simultaneously. The most common solution is first to project the original data into a lower dimensional space and then force them to preserve the similar intrinsic structure under linear uncorrelation constraint. However, there are three shortcomings. First, the final graph generated by the iterative learning process differs significantly from the initial graph in which the original intrinsic structure is embedded. Second, it requires prior knowledge about a moderate dimension of subspace. Third, it is inefficient when dealing with high-dimensional datasets. The first shortcoming, which is longstanding and undiscovered, makes the previous methods fail to achieve their expected results. The last two ones increase the difficulty of applying in different fields. Therefore, two unsupervised feature selection methods are proposed based on controllable adaptive graph learning and uncorrelated/independent feature learning (CAG-U and CAG-I) to address the abovementioned issues. In the proposed methods, the final graph that preserves intrinsic structure can be adaptively learned while the difference between the two graphs can be well controlled. Besides, relatively uncorrelated/independent features can be selected using a discrete projection matrix. The experimental results on 12 datasets in different fields show the superiority of CAG-U and CAG-I.

Multichannel image completion with mixture noise is a challenging problem in the fields of machine learning, computer vision, image processing, and data mining. Traditional image completion models are not appropriate to deal with this problem directly since their reconstruction priors may mismatch corruption priors. To address this issue, we propose a novel nonlocal self-similarity-based weighted tensor low-rank decomposition (NSWTLD) model that can achieve global optimization and local enhancement. In the proposed model, based on the corruption priors and the reconstruction priors, a pixel weighting strategy is given to characterize the joint effects of missing data, the Gaussian noise, and the impulse noise. To discover and utilize the accurate nonlocal self-similarity information to enhance the restoration quality of the details, the traditional nonlocal learning framework is optimized by employing improved index determination of patch group and handling strip noise caused by patch overlapping. In addition, an efficient and convergent algorithm is presented to solve the NSWTLD model. Comprehensive experiments are conducted on four types of multichannel images under various corruption scenarios. The results demonstrate the efficiency and effectiveness of the proposed model.

A Software Product Line (SPL) is a set of software products that are built from a variability model. Real-world SPLs typically involve a vast number of valid products, making it impossible to individually test each of them. This arises the need for automated test suite generation, which was previously modeled as either a single-objective or a multi-objective optimization problem considering only objective functions. This article provides a completely different mathematical model by exploiting the benefits of Quality-Diversity (QD) optimization that is composed of not only an objective function (e.g., t -wise coverage or test suite diversity) but also a user-defined behavior space (e.g., the space with test suite size as its dimension). We argue that the new model is more suitable and generic than the two alternatives because it provides at a time a large set of diverse (measured in the behavior space) and high-performing solutions that can ease the decision-making process. We apply MAP-Elites, one of the most popular QD algorithms, to solve the model. The results of the evaluation, on both realistic and artificial SPLs, are promising, with MAP-Elites significantly and substantially outperforming both single- and multi-objective approaches, and also several state-of-the-art SPL testing tools. In summary, this article provides a new and promising perspective on the test suite generation for SPLs.

Constrained multiobjective optimization problems widely exist in real-world applications. To handle them, the balance between constraints and objectives is crucial, but remains challenging due to non-negligible impacts of problem types. In our context, the problem types refer particularly to those determined by the relationship between the constrained Pareto-optimal front (PF) and the unconstrained PF. Unfortunately, there has been little awareness on how to achieve this balance when faced with different types of problems. In this article, we propose a new constraint handling technique (CHT) by taking into account potential problem types. Specifically, inspired by the prior work, problems are classified into three primary types: 1) I; 2) II; and 3) III, with the constrained PF being made up of the entire, part and none of the unconstrained counterpart, respectively. Clearly, any problem must be one of the three types. For each possible type, there exists a tailored mechanism being used to handle the relationships between constraints and objectives (i.e., constraint priority, objective priority, or the switch between them). It is worth mentioning that exact problem types are not required because we just consider their possibilities in the new CHT. Conceptually, we show that the new CHT can make a tradeoff among different types of problems. This argument is confirmed by experimental studies performed on 38 benchmark problems, whose types are known, and a real-world problem (with unknown types) in search-based software engineering. Results demonstrate that within both decomposition-based and nondecomposition-based frameworks, the new CHT can indeed achieve a good tradeoff among different problem types, being better than several state-of-the-art CHTs.

Multichannel image completion with mixture noise is a common but complex problem in the fields of machine learning, image processing, and computer vision. Most existing algorithms devote to explore global low-rank information and fail to optimize local and joint-mode structures, which may lead to oversmooth restoration results or lower quality restoration details. In this study, we propose a novel model to deal with multichannel image completion with mixture noise based on adaptive sparse low-rank tensor subspace and nonlocal self-similarity (ASLTS-NS). In the proposed model, a nonlocal similar patch matching framework cooperating with Tucker decomposition is used to explore information of global and joint modes and optimize the local structure for improving restoration quality. In order to enhance the robustness of low-rank decomposition to data missing and mixture noise, we present an adaptive sparse low-rank regularization to construct robust tensor subspace for self-weighing importance of different modes and capturing a stable inherent structure. In addition, joint tensor Frobenius and$l_{1}$regularizations are exploited to control two different types of noise. Based on alternating directions method of multipliers (ADMM), a convergent learning algorithm is designed to solve this model. Experimental results on three different types of multichannel image sets demonstrate the advantages of ASLTS-NS under five complex scenarios.

Imbalanced classification is a challenging task in the fields of machine learning, data mining and pattern recognition. Cost-sensitive online algorithms are very important methods for large-scale imbalanced classification problems. At present, most of the cost-sensitive classification algorithms focus on the accuracy of the minority class and ignore the accuracy of the majority class. In order to better balance the accuracy between the minority class and the majority class, in this article, a misclassification cost is presented to ensure that the cost-sensitive online algorithm can better deal with the imbalanced classification problems without signifificantly reducing the accuracy of the majority class. Based on the proposed misclassification cost, a novel cost-sensitive online adaptive kernel learning algorithm is proposed to boost the adaptability of kernel function when data arrives one by one. According to the essential characteristics of the imbalanced binary classification, a cost-sensitive online adaptive kernel learning algorithm is given to handle the large-scale imbalanced multi-class classification problems. Theoretical analysis of the proposed algorithms are provided. Extensive experiments demonstrate that compared with the state-of-the-art imbalanced classification algorithms, the proposed algorithms can significantly improve the classification performances on most of the large-scale imbalanced data sets.

Domain adaptation is concerned with the problem of generalizing a classification model to a target domain with little or no labeled data, by leveraging the abundant labeled data from a related source domain. The source and target domains possess different joint probability distributions, making it challenging for model generalization. In this article, we introduce domain neural adaptation (DNA): an approach that exploits nonlinear deep neural network to 1) match the source and target joint distributions in the network activation space and 2) learn the classifier in an end-to-end manner. Specifically, we employ the relative chi-square divergence to compare the two joint distributions, and show that the divergence can be estimated via seeking the maximal value of a quadratic functional over the reproducing kernel hilbert space. The analytic solution to this maximization problem enables us to explicitly express the divergence estimate as a function of the neural network mapping. We optimize the network parameters to minimize the estimated joint distribution divergence and the classification loss, yielding a classification model that generalizes well to the target domain. Empirical results on several visual datasets demonstrate that our solution is statistically better than its competitors.

Real-world software product lines (SPLs) often encompass enormous valid configurations that are impossible to enumerate. To understand properties of the space formed by all valid configurations, a feasible way is to select a small and valid sample set. Even though a number of sampling strategies have been proposed, they either fail to produce diverse samples with respect to the number of selected features (an important property to characterize behaviors of configurations), or achieve diverse sampling but with limited scalability (the handleable configuration space size is limited to 1013). To resolve this dilemma, we propose a scalable diverse sampling strategy, which uses a distance metric in combination with the novelty search algorithm to produce diverse samples in an incremental way. The distance metric is carefully designed to measure similarities between configurations, and further diversity of a sample set. The novelty search incrementally improves diversity of samples through the search for novel configurations. We evaluate our sampling algorithm on 39 real-world SPLs. It is able to generate the required number of samples for all the SPLs, including those which cannot be counted by sharpSAT, a state-of-the-art model counting solver. Moreover, it performs better than or at least competitively to state-of-the-art samplers regarding diversity of the sample set. Experimental results suggest that only the proposed sampler (among all the tested ones) achieves scalable diverse sampling.

Hyperspectral image mixed denoising is a challenging task in the fields of remote sensing, environmental monitoring, mineral exploration, etc. A crucial difficulty is to acquire clean restoration from hyperspectral image (HSI) that encounters Gaussian noise, impulse noise, strip noise and deadlines. In the previous works, combining global information and nonlocal information is a popular way to learn the comprehensive characteristics of the clean HSI. However, the advantages of 2D spatial structure similarity and spectral low-rankness may not be fully exploited at the same time in global prior learning. The iterative update between global restoration and nonlocal restoration may cause high time consumption and certain loss of information. To address these issues, we propose a three-stage mixed denoising model based on novel hybrid low-rank tensor approximation and global-guided-nonlocal prior mechanism (HLTA-GN). Firstly, to learn a good global prior, hybrid low-rank tensor approximation incorporated with a useful nonconvex tensor rank estimation is presented to balance 2D spatial similarity and spectral low-rankness. Secondly, to learn a high-quality nonlocal prior, global-guided-nonlocal prior mechanism is proposed to help nonlocal restoration suppress the residual noise. At the same time, a regularized sequential low-rank tensor approximation is proposed to enhance the robustness to noisy patch groups. Thirdly, a weighted fusion on global prior and nonlocal prior helps to further balance global denoising and patch processing. An efficient learning algorithm is provided to solve HLTA-GN. Abundant experiments are conducted on various HSIs with several scenarios. The experimental results demonstrate the superiority of HLTA-GN.

Testing software product lines (SPLs) is difficult due to a huge number of possible products to be tested. Recently, there has been a growing interest in similarity-based testing of SPLs, where similarity is used as a surrogate metric for the$t$-wise coverage. In this context, one of the primary goals is to sample, by optimizing similarity metrics using search-based algorithms, a small subset of test cases (i.e., products) as dissimilar as possible, thus potentially making more$t$-wise combinations covered. Prior work has shown, by means of empirical studies, the great potential of current similarity-based testing approaches. However, the rationale of this testing technique deserves a more rigorous exploration. To this end, we perform correlation analyses to investigate how similarity metrics are correlated with the$t$-wise coverage. We find that similarity metrics generally have significantly positive correlations with the$t$-wise coverage. This well explains why similarity-based testing works, as the improvement on similarity metrics will potentially increase the$t$-wise coverage. Moreover, we explore, for the first time, the use of the novelty search (NS) algorithm for similarity-based SPL testing. The algorithm rewards “novel” individuals, i.e., those being different from individuals discovered previously, and this well matches the goal of similarity-based SPL testing. We find that the novelty score used in NS has (much) stronger positive correlations with the$t$-wise coverage than previous approaches relying on a genetic algorithm (GA) with a similarity-based fitness function. Experimental results on 31 software product lines validate the superiority of NS over GA, as well as other state-of-the-art approaches, concerning both$t$-wise coverage and fault detection capacity. Finally, we investigate whether it is useful to combine two satisfiability solvers when generating new individuals in NS, and how the performance of NS is affected by its key parameters. In summary, looking for novelty provides a promising way of sampling diverse test cases for SPLs.

An intrinsic problem in domain adaptation is the joint distribution mismatch between the source and target domains. Therefore, it is crucial to match the two joint distributions such that the source domain knowledge can be properly transferred to the target domain. Unfortunately, in semi-supervised domain adaptation (SSDA) this problem still remains unsolved. In this article, we therefore present an asymmetric joint distribution matching (AJDM) approach, which seeks a couple of asymmetric matrices to linearly match the source and target joint distributions under the relative chi-square divergence. Specifically, we introduce a least square method to estimate the divergence, which is free from estimating the two joint distributions. Furthermore, we show that our AJDM approach can be generalized to a kernel version, enabling it to handle nonlinearity in the data. From the perspective of Riemannian geometry, learning the linear and nonlinear mappings are both formulated as optimization problems defined on the product of Riemannian manifolds. Numerical experiments on synthetic and real-world data sets demonstrate the effectiveness of the proposed approach and testify its superiority over existing SSDA techniques.

This paper integrates an estimation of distribution (EoD)-based update operator into decomposition-based multiobjective evolutionary algorithms for binary optimization. The probabilistic model in the update operator is a probability vector, which is adaptively learned from historical information of each subproblem. We show that this update operator can significantly enhance decomposition-based algorithms on a number of benchmark problems. Moreover, we apply the enhanced algorithms to the constrained optimal software product selection (OSPS) problem in the field of search-based software engineering. For this real-world problem, we give its formal definition and then develop a new repair operator based on satisfiability solvers. It is demonstrated by the experimental results that the algorithms equipped with the EoD operator are effective in dealing with this practical problem, particularly for large-scale instances. The interdisciplinary studies in this paper provide a new real-world application scenario for constrained multiobjective binary optimizers and also offer valuable techniques for software engineers in handling the OSPS problem.

We propose a new many-objective evolutionary algorithm with Pareto-adaptive reference points. In this algorithm, the shape of the Pareto-optimal front (PF) is estimated based on a ratio of Euclidean distances. If the estimated shape is likely to be convex, the nadir point is used as the reference point to calculate the convergence and diversity indicators for individuals. Otherwise, the reference point is set to the ideal point. In addition, the estimation of the nadir point is different from what was widely used in the literature. The nadir point, together with the ideal point, provides a feasible way to deal with dominance resistant solutions, which are difficult to be detected and eliminated in Pareto-based algorithms. The proposed algorithm is compared with the state-of-the-art many-objective optimization algorithms on a number of unconstrained and constrained test problems with up to 15 objectives. The experimental results show that it performs better than other algorithms in most of the test instances. Moreover, the new algorithm shows good performance on problems whose PFs are irregular (being discontinuous, degenerated, bent, or mixed). The observed high performance and inherent good properties (such as being free of weight vectors and control parameters) make the new proposal a promising tool for other similar problems.

Domain adaptation addresses the learning problem where the training data are sampled from a source joint distribution (source domain), while the test data are sampled from a different target joint distribution (target domain). Because of this joint distribution mismatch, a discriminative classifier naively trained on the source domain often generalizes poorly to the target domain. In this paper, we therefore present a Joint Distribution Invariant Projections (JDIP) approach to solve this problem. The proposed approach exploits linear projections to directly match the source and target joint distributions under the L2-distance. Since the traditional kernel density estimators for distribution estimation tend to be less reliable as the dimensionality increases, we propose a least square method to estimate the L2-distance without the need to estimate the two joint distributions, leading to a quadratic problem with analytic solution. Furthermore, we introduce a kernel version of JDIP to account for inherent nonlinearity in the data. We show that the proposed learning problems can be naturally cast as optimization problems defined on the product of Riemannian manifolds. To be comprehensive, we also establish an error bound, theoretically explaining how our method works and contributes to reducing the target domain generalization error. Extensive empirical evidence demonstrates the benefits of our approach over state-of-the-art domain adaptation methods on several visual data sets.

Domain adaptation tries to adapt a model trained from a source domain to a different but related target domain. Currently, prevailing methods for domain adaptation rely on either instance reweighting or feature transformation. Unfortunately, instance reweighting has difficulty in estimating the sample weights as the dimension increases, whereas feature transformation sometimes fails to make the transformed source and target distributions similar when the cross-domain discrepancy is large. In order to overcome the shortcomings of both methodologies, in this article, we model the unsupervised domain adaptation problem under the generalized covariate shift assumption and adapt the source distribution to the target distribution in a subspace by applying a distribution adaptation function. Accordingly, we propose two frameworks: Bregman-divergence-embedded structural risk minimization (BSRM) and joint structural risk minimization (JSRM). In the proposed frameworks, the subspace distribution adaptation function and the target prediction model are jointly learned. Under certain instantiations, convex optimization problems are derived from both frameworks. Experimental results on the synthetic and real-world text and image data sets show that the proposed methods outperform the state-of-the-art domain adaptation techniques with statistical significance.

The problem of recovering missing data of an incomplete tensor has drawn more and more attentions in the fields of pattern recognition, machine learning, data mining, computer vision, and signal processing. Researches on this problem usually share a common assumption that the original tensor is of low-rank. One of the important ways to capture the low-rank structure of the incomplete tensor is based on tensor factorization. For the traditional tensor factorization algorithms, the tensor ranks should be specified ahead, which is not reasonable in real applications. To overcome this drawback, an adaptive algorithm is first presented based on sequentially truncated higher order singular value decomposition (ST-HOSVD) for fast low-rank approximation of complete tensor, in which the tensor ranks can be obtained adaptively. Then for tensor with missing data, we use adaptive ST-HOSVD and the average operator of low-rank approximation to improve the accuracy of the fulfilled tensor. Convergence analysis of the proposed algorithm is also given in this paper. The experimental results on 14 image datasets and three video datasets show that the proposed method outperforms the state-of-the-art methods in terms of running time and the accuracy.

In sampling-based matting methods, the alpha is estimated by choosing the best pair of foreground and background color samples. The lack of true samples is the major obstacle in obtaining high-quality alpha mattes. Regrettably, several proposed approaches did not address the conflicts among multiple sampling criteria and the effects of incomplete sample spaces. To address this issue, we propose a pixel-level discrete multiobjective sampling (PDMS) method. The color sampling process at each unknown pixel is formalized as a multiobjective optimization problem (MOP). The strength of PDMS includes its ability to minimize both color difference and spatial distance between unknown and known pixels, and its capacity to adaptively make trade-offs among conflicting sampling criteria. To mitigate the effects of incomplete sample spaces, the sample space is extended to complete known regions in PDMS, which means that the colors of all known pixels can be sampled, instead of mean colors of superpixels. Our experimental results show that PDMS collects a small set of samples while achieving smaller minimum absolute difference in alpha estimation. Moreover, PDMS implements pixel-level sampling by using the proposed multiobjective optimization algorithm to efficiently solve sampling MOPs. The PDMS-based matting method provides high-quality alpha mattes with sharp boundaries and thus outperforms those prior image matting methods in terms of gradient error.

Filtering images of more than one channel are challenging in terms of both efficiency and effectiveness. By grouping similar patches to utilize the self-similarity and sparse linear approximation of natural images, recent nonlocal and transform-domain methods have been widely used in color and multispectral image (MSI) denoising. Many related methods focus on the modeling of group level correlation to enhance sparsity, which often resorts to a recursive strategy with a large number of similar patches. The importance of the patch level representation is understated. In this paper, we mainly investigate the influence and potential of representation at patch level by considering a general formulation with a block diagonal matrix. We further show that by training a proper global patch basis, along with a local principal component analysis transform in the grouping dimension, a simple transform-threshold-inverse method could produce very competitive results. Fast implementation is also developed to reduce the computational complexity. The extensive experiments on both the simulated and real datasets demonstrate its robustness, effectiveness, and efficiency.

The simultaneous removal of noise and preservation of the integrity of 3-D magnetic resonance (MR) images is a difficult and important task. In this paper, we consider characterizing MR images with 3-D operators, and present a novel 4-D transform-domain method termed 'modified nonlocal tensor-SVD (MNL-tSVD)' for MR image denoising. The proposed method is based on the grouping, hard-thresholding and aggregation paradigms, and can be viewed as a generalized nonlocal extension of tensor-SVD (t-SVD). By keeping MR images in its natural three-dimensional form, and collaboratively filtering similar patches, MNL-tSVD utilizes both the self-similarity property and 3-D structure of MR images to preserve more actual details and minimize the introduction of new artifacts. We show the adaptability of MNL-tSVD by incorporating it into a two-stage denoising strategy with a few adjustments. In addition, analysis of the relationship between MNL-tSVD and current the state-of-the-art 4-D transforms is given. Experimental comparisons over simulated and real brain data sets at different Rician noise levels show that MNL-tSVD can produce competitive performance compared with related approaches.

In the fields of machine learning, pattern recognition, image processing, and computer vision, the data are usually represented by the tensors. For the semisupervised tensor classification, the existing transductive support tensor machine (TSTM) needs to resort to iterative technique, which is very time-consuming. In order to overcome this shortcoming, in this paper, we extend the concave-convex procedure-based transductive support vector machine (CCCP-TSVM) to the tensor patterns and propose a low-rank approximation-based TSTM, in which the tensor rank-one decomposition is used to compute the inner product of the tensors. Theoretically, concave-convex procedure-based TSTM (CCCP-TSTM) is an extension of the linear CCCP-TSVM to tensor patterns. When the input patterns are vectors, CCCP-TSTM degenerates into the linear CCCP-TSVM. A set of experiments is conducted on 23 semisupervised classification tasks, which are generated from seven second-order face data sets, three third-order gait data sets, and two third-order image data sets, to illustrate the performance of the CCCP-TSTM. The results show that compared with CCCP-TSVM and TSTM, CCCP-TSTM provides significant performance gain in terms of test accuracy and training speed.

The current trend of growth of information reveals that it is inevitable that large-scale learning problems become the norm. In this paper, we propose and analyze a novel Low-density Cut based tree Decomposition method for large-scale SVM problems, called LCD-SVM. The basic idea here is divide and conquer: use a decision tree to decompose the data space and train SVMs on the decomposed regions. Specifically, we demonstrate the application of low density separation principle to devise a splitting criterion for rapidly generating a high-quality tree, thus maximizing the benefits of SVMs training. Extensive experiments on 14 real-world datasets show that our approach can provide a significant improvement in training time over state-of-the-art methods while keeps comparable test accuracy with other methods, especially for very large-scale datasets.

With advances in data collection technologies, tensor data is assuming increasing prominence in many applications and the problem of supervised tensor learning has emerged as a topic of critical significance in the data mining and machine learning community. Conventional methods for supervised tensor learning mainly focus on learning kernels by flattening the tensor into vectors or matrices, however structural information within the tensors will be lost. In this paper, we introduce a new scheme to design structure-preserving kernels for supervised tensor learning. Specifically, we demonstrate how to leverage the naturally available structure within the tensorial representation to encode prior knowledge in the kernel. We proposed a tensor kernel that can preserve tensor structures based upon dual-tensorial mapping. The dual-tensorial mapping function can map each tensor instance in the input space to another tensor in the feature space while preserving the tensorial structure. Theoretically, our approach is an extension of the conventional kernels in the vector space to tensor space. We applied our novel kernel in conjunction with SVM to real-world tensor classification problems including brain fMRI classification for three different diseases (i.e., Alzheimer's disease, ADHD and brain damage by HIV). Extensive empirical studies demonstrate that our proposed approach can effectively boost tensor classification performances, particularly with small sample sizes.

There has been growing interest in developing more effective learning machines for tensor classification. At present, most of the existing learning machines, such as support tensor machine (STM), involve nonconvex optimization problems and need to resort to iterative techniques. Obviously, it is very time-consuming and may suffer from local minima. In order to overcome these two shortcomings, in this paper, we present a novel linear support higher-order tensor machine (SHTM) which integrates the merits of linear C-support vector machine (C-SVM) and tensor rank-one decomposition. Theoretically, SHTM is an extension of the linear C-SVM to tensor patterns. When the input patterns are vectors, SHTM degenerates into the standard C-SVM. A set of experiments is conducted on nine second-order face recognition datasets and three third-order gait recognition datasets to illustrate the performance of the proposed SHTM. The statistic test shows that compared with STM and C-SVM with the RBF kernel, SHTM provides significant performance gain in terms of test accuracy and training speed, especially in the case of higher-order tensors.

The support vector machine (SVM) has provided higher performance than traditional learning machines and has been widely applied in real-world classification problems and nonlinear function estimation problems. Unfortunately, the training process of the SVM is sensitive to the outliers or noises in the training set. In this paper, a common misunderstanding of Gaussian-function-based kernel fuzzy clustering is corrected, and a kernel fuzzy c-means clustering-based fuzzy SVM algorithm (KFCM-FSVM) is developed to deal with the classification problems with outliers or noises. In the KFCM-FSVM algorithm, we first use the FCM clustering to cluster each of two classes from the training set in the high-dimensional feature space. The farthest pair of clusters, where one cluster comes from the positive class and the other from the negative class, is then searched and forms one new training set with membership degrees. Finally, we adopt FSVM to induce the final classification results on this new training set. The computational complexity of the KFCM-FSVM algorithm is analyzed. A set of experiments is conducted on six benchmarking datasets and four artificial datasets for testing the generalization performance of the KFCM-FSVM algorithm. The results indicate that the KFCM-FSVM algorithm is robust for classification problems with outliers or noises.

Support vector machines (SVMs) are powerful tools in machine learning community, but it is not easy to select suitable parameters for them. And, very often SVMs show slow speeds in test phase due to their large number of support vectors. To remedy SVMs deficiencies, we propose a novel SVM-like method, which is called evolutionary support center machine (ESCM) in this paper. The key idea behind ESCM is to apply evolutionary algorithm to construct the separation hyperplane with the similar form to those constructed by SVMs in an incremental way. ESCM can not only optimize the support centers and tune the kernel parameters adaptively, but also control the number of support centers appropriately. Numerical experiments on several UCI benchmarks verify the efficiency of ESCM.