Few-shot remote sensing scene classification (FSRSC) is hindered by large intra-class variance, inter-class similarity, and frequent noise in satellite imagery. To address these challenges, a lightweight framework named GDPNet is proposed, coupling multi-scale feature fusion with dynamic prototype selection. Specifically, a Gated Cross-Stage Fusion (GCF) module adaptively aggregates features from a MobileViT backbone via channel–spatial attention, enhancing scale-aware discriminability. A Dynamic Prototype Selection (DPS) module further introduces category-specific thresholds derived from intra-class statistics, yielding more robust and reliable decision boundaries. Experiments on four benchmarks demonstrate that GDPNet achieves state-of-the-art performance in both 5-way 1-shot and 5-way 5-shot settings, while requiring only 2.2M parameters. These results highlight GDPNet as a compact and effective solution for real-world FSRSC on edge devices.

In multi-instance nonparallel plane learning (NPL), the training set is comprised of bags of instances and the nonparallel planes are trained to classify the bags. Most of the existing multi-instance NPL methods are proposed based on a twin support vector machine (TWSVM). Similar to TWSVM, they use only a single plane to generalize the data occurrence of one class and do not sufficiently consider the boundary information, which may lead to the limitation of their classification accuracy. In this article, we propose a multi-instance nonparallel tube learning (MINTL) method. Distinguished from the existing multi-instance NPL methods, MINTL embeds the boundary information into the classifier by learning a large-margin-based -tube for each class, such that the boundary information can be incorporated into refining the classifier and further improving the performance. Specifically, given a -class multi-instance dataset, MINTL seeks -tubes, one for each class. In multi-instance learning, each positive bag contains at least one positive instance. To build up the -tube of class , we require that each bag of class should have at least one instance included in the -tube. Moreover, except for one instance included in the -tube, the remaining instances in the positive bag may include positive instances or irrelevant instances, and their labels are unavailable. A large margin constraint is presented to assign the remaining instances either inside the -tube or outside the -tube with a large margin. Substantial experiments on real-world datasets have shown that MINTL obtains significantly better classification accuracy than the existing multi-instance NPL methods.

Federated learning (FL) belongs to distributed machine learning . It allows data information sharing between users while protecting their data privacy at the same time. However, in many real-world scenarios, the data collected by client devices may be affected by noise in the working environment, leading to the decreased accuracy and confidence. Therefore, it is necessary to take measures to reduce data uncertainty in order to enhance the performance of FL algorithms. Traditional FL methods encounter challenges in handling uncertain data, which motivates the introduction of multi-view learning in this paper which designs an FL model suitable for highly variable data characteristics . We first achieve information complementarity among data views while ensuring the consistency in data representation . Furthermore, we quantify data uncertainty using the reachable region of data noise, thereby improving model robustness. To maintain data privacy between clients, we design an adaptive Kalman filter-based differential protection security protocol . Clients use the protocol to process local data and upload it to the master server, which returns the updated model parameters to the clients. The experimental results demonstrate the effectiveness of the federated learning model proposed in this paper.

Multi-view learning can improve classification performance by combining information between different views. Due to the similarity in different views of the dataset, sometimes the features obtained are highly limited and redundant. At the same time, different views accumulate a large amount of noisy information, which will affect the classification performance of the model. To solve these problems, we embed privileged information in the model and introduce dictionary learning, and proposed a new dictionary-based multi-view learning method with privileged information (MVDL-PI). First, two sets of dictionaries (synthetic dictionary and analysis dictionary) and sparse representation matrices of different information domains are obtained for each view information and privilege information through dictionary learning. Then, we obtain consistency information from the regularization terms of the two different sets of synthetic dictionaries and construct a LUPI (Learning using privileged information) classifier by the sparse representation. In addition, we use alternating convex optimization and Lagrange multiplier methods to optimize the model and prove its convergence. In the experiment, we did a number of experiments comparing this method with similar recent methods. The experimental results show that the MVDL-PI method is superior to other methods in terms of stability and classification accuracy.

Maximum margin clustering (MMC) is a typical clustering method which aims to maximize the margin between different clusters. However, in practical applications, a data object may be represented by multiple feature sets (views), with each feature set representing different information of the underlying data. The traditional MMC methods can handle only the data from a single view and are unable to utilize the multi-view data to enhance the clustering model. In multi-view clustering, there are two basic principles: the consensus principle and complementarity principle. Most multi-view clustering methods implement mainly the consensus principle, while the complementarity principle has not been sufficiently taken into account. Distinguished from the existing methods,$\text {M}^{3}\text {CP}$introduces the idea of privileged information learning into multi-view clustering and implements both of the consensus principle and complementarity principle. Based on privileged information learning,$\text {M}^{3}\text {CP}$embodies the complementarity principle by considering one view as the main learning information and the other views as the privileged information, so that multiple views can provide information to complement each other. The derived learning problem is then solved by applying the constrained concave–convex procedure and cutting plane techniques. By employing these techniques, the computational time of$\text {M}^{3}\text {CP}$is able to scale linearly with respect to the dataset size. Numerical experiments on real-life multi-view datasets demonstrate that$\text {M}^{3}\text {CP}$is able to achieve better clustering accuracy and meanwhile needs less computational time, compared to state-of-the-art multi-view clustering methods.

Few-shot object counting (FSOC) estimates object quantities in query images using a few of support information. Unlike traditional counting methods, FSOC prioritizes more discriminative and generalized similarity measures between query and support data. This facilitates counting objects from new categories without extensive dataset creation or costly retraining. However, existing approaches often rely on fixed similarity rules, leading to spatial information loss. Limited training data can yield sparse similarity feature distribution, hampering the model’s learning and its ability to handle objects with large intraclass differences. In this study, we introduce a novel FSOC network named DSALVANet that comprises the dynamic similarity-aware module (DSAM) and the latent variable augmentation module (LVAM). DSAM establishes adaptive metric rules for support features to find similar regions in the metric space for accurate object counting. LVAM utilizes prior similarity knowledge from DSAM to model the latent distribution of the density map, improving the decoder’s robustness by sampling diverse latent variables during training. Extensive experiments on the FSOC benchmark and remote-sensing datasets demonstrate our method’s effectiveness and state-of-the-art performance. The code and model are available at DSALVANet.

Transfer learning (TL) is an information reuse learning tool, which can help us learn better classification effect than traditional single task learning, because transfer learning can share information within the task-to-task model. Most TL algorithms are studied in the field of data improvement, doing some data extraction and transformation. However, it ignores that existing the additional information to improve the model’s accuracy, like Universum samples in the training data with privileged information. In this article, we focus on considering prior data to improve the TL algorithm, and the additional features also called privileged information are incorporated into the learning to improve the learning paradigm. In addition, we also carry out the Universum samples which do not belong to any indicated categories into the transfer learning paradigm to improve the utilization of prior knowledge. We propose a new TL Model (PU-TLSVM), in which each task with corresponding privileged features and Universum data is considered in the proposed model, so as to apply tasks with a priori data to the training stage. Then, we use Lagrange duality theorem to optimize our model to obtain the optimal discriminant for target task classification. Finally, we make a lot of predictions and tests to compare the actual effectiveness of the proposed method with the previous methods. The experiment results indicate that the proposed method is more effective and robust than other baselines.

Few-shot learning (FSL) aims to learn novel concepts quickly from a few novel labeled samples with the transferable knowledge learned from base dataset. The existing FSL methods usually treat each sample as a single feature point in embedding space and classify through one single comparison task. However, the few-shot single feature points on the novel meta-testing episode are still vulnerable to noise easily although with the good transferable knowledge, because the novel categories are never seen on base dataset. Besides, the existing FSL models are trained by only one single comparison task and ignore that different semantic feature maps have different weights on different comparison objects and tasks, which cannot take full advantage of the valuable information from different multiple comparison tasks and objects to make the latent features (LFs) more robust based on only few-shot samples. In this article, we propose a novel multitask LF augmentation (MTLFA) framework to learn the meta-knowledge of generalizing key intraclass and distinguishable interclass sample features from only few-shot samples through an LF augmentation (LFA) module and a multitask (MT) framework. Our MTLFA treats the support features as sampling from the class-specific LF distribution, enhancing the diversity of support features and reducing the impact of noise based on few-shot support samples. Furthermore, an MT framework is introduced to obtain more valuable comparison-task-related and episode-related comparison information from multiple different comparison tasks in which different semantic feature maps have different weights, adjusting the prior LFs and generating the more robust and effective episode-related classifier. Besides, we analyze the feasibility and effectiveness of MTLFA from theoretical views based on the Hoeffding's inequality and the Chernoff's bounding method. Extensive experiments conducted on three benchmark datasets demonstrate that the MTLFA achieves the state-of-the-art performance in FSL. The experimental results verify our theoretical analysis and the effectiveness and robustness of MTLFA framework in FSL.

Ordinal regression (OR) aims to solve multiclass classification problems with ordinal classes. Support vector OR (SVOR) is a typical OR algorithm and has been extensively used in OR problems. In this article, based on the characteristics of OR problems, we propose a novel pinball loss function and present an SVOR method with pinball loss (pin-SVOR). Pin-SVOR is fundamentally different from traditional SVOR with hinge loss. Traditional SVOR employs the hinge loss function, and the classifier is determined by only a few data points near the class boundary, called support vectors, which may lead to a noise sensitive and re-sampling unstable classifier. Distinctively, pin-SVOR employs the pinball loss function. It attaches an extra penalty to correctly classified data that lies inside the class, such that all the training data is involved in deciding the classifier. The data near the middle of each class has a small penalty, and that near the class boundary has a large penalty. Thus, the training data tend to lie near the middle of each class instead of on the class boundary, which leads to scatter minimization in the middle of each class and noise insensitivity. The experimental results show that pin-SVOR has better classification performance than state-of-the-art OR methods.

Existing few-shot learning (FSL) methods usually treat each sample as a single feature point or utilize intra-class feature transformation to augment features. However, few-shot novel features are always vulnerable to noise, intra-class features have large variance and the direction of intra-class feature transformations is uncontrollable, which result in degradation of FSL models. Besides, existing FSL methods are one-generation based which do not utilize the prior knowledge obtained in the prior generation model to generate more robust posterior model in FSL and lack the interpretability in FSL. In this paper, we propose a novel two-generation based Latent Feature Augmentation and Distribution Regularization framework (LFADR) including prior relation net (PRN) and vae-based posterior relation net (VPORN) to generate a more robust VPORN based on PRN by transferring the prior knowledge in FSL. Firstly, we utilize a simple single-original-feature-point-based PRN to generate the more informative prior knowledge. We then propose a regularized-distribution-based VPORN driven by VAE to augment latent features by sampling from regularized class-specific distribution based on the prior knowledge transferred from PRN in order to guarantee and control the diversity of intra-class features which avoids uncontrollable feature transformations and reduces the variance of intra-class features and the impact of noise. As a result, LFADR can learn more key and robust intra-class and discriminative inter-class features and make decision boundary clearer in FSL, which is optimized as a variational inference problem. Furthermore, we analyse the feasibility and effectiveness of our framework based on Hoeffding’s inequality and Chernoff’s bounding method. Finally, experimental results validate our theoretical analysis and the effectiveness of our proposed FSL framework.

Image classification is an important part of pattern recognition. With the development of convolutional neural networks (CNNs), many CNN methods are proposed, which have a large number of samples for training, which can have high performance. However, there may exist limited samples in some real-world applications. In order to improve the performance of CNN learning with insufficient samples, this article proposes a new method called the classifier method based on a variational autoencoder (CFVAE), which is comprised of two parts: 1) a standard CNN as a prior classifier and 2) a CNN based on variational autoencoder (VAE) as a posterior classifier. First, the prior classifier is utilized to generate the prior label and information about distributions of latent variables; and the posterior classifier is trained to augment some latent variables from regularized distributions to improve the performance. Second, we also present the uniform objective function of CFVAE and put forward an optimization method based on the stochastic gradient variational Bayes method to solve the objective model. Third, we analyze the feasibility of CFVAE based on Hoeffding's inequality and Chernoff's bounding method. This analysis indicates that the latent variables augmentation method based on regularized latent variables distributions can generate samples fitting well with the distribution of data such that the proposed method can improve the performance of CNN with insufficient samples. Finally, the experiments manifest that our proposed CFVAE can provide more accurate performance than state-of-the-art methods.

In multi-instance learning (MIL), labels are associated with bags rather than the instances in the bag. Most of the previous MIL methods assume that each bag has the actual label in the training set. However, from the process of labeling work, the label of a bag is always evaluated by the calculation of the labels obtained from a number of labelers. In the calculation, the weight of each labeler is always unknown and people always assign the weight for each labeler by random or equally, and this may result in the ambiguous labels for the bags, which is called weak labels here. In addition, we always meet the problem of knowledge transfer from the source task to the target task, and this leads to the study of multiple instance transfer learning. In this article, we propose a new framework called transfer learning-based multiple instance learning (TMIL) framework to address the problem of multiple instance transfer learning in which both the source task and the target task contain the weak labels. We first construct a TMIL model with weak labels, which can transfer knowledge from the source task to the target task where both source and target tasks contain weak labels. We then put forward an iterative framework to solve the transfer learning model with weak labels so that we can update the label of the bag to improve the performance of multiple instance learning. We then present the convergence analysis of the proposed method. The experiments show that the proposed method outperforms the existing multiple instance learning methods and can correct the initial labels to obtain the actual labels for the bags.

Over the past few years, we have made great progress in image categorization based on convolutional neural networks (CNNs). These CNNs are always trained based on a large-scale image data set; however, people may only have limited training samples for training CNN in the real-world applications. To solve this problem, one intuition is augmenting training samples. In this article, we propose an algorithm called Lavagan ( La tent V ariables A ugmentation Method based on G enerative A dversarial N ets) to improve the performance of CNN with insufficient training samples. The proposed Lavagan method is mainly composed of two tasks. The first task is that we augment a number latent variables (LVs) from a set of adaptive and constrained LVs distributions. In the second task, we take the augmented LVs into the training procedure of the image classifier. By taking these two tasks into account, we propose a uniform objective function to incorporate the two tasks into the learning. We then put forward an alternative two-play minimization game to minimize this uniform loss function such that we can obtain the predictive classifier. Moreover, based on Hoeffding’s Inequality and Chernoff Bounding method, we analyze the feasibility and efficiency of the proposed Lavagan method, which manifests that the LV augmentation method is able to improve the performance of Lavagan with insufficient training samples. Finally, the experiment has shown that the proposed Lavagan method is able to deliver more accurate performance than the existing state-of-the-art methods.

Multi-view classification aims at designing a multi-view learning strategy to train a classifier from multi-view data, which are easily collected in practice. Most of the existing works focus on multi-view classification by assuming the multi-view data are collected with precise information. However, we always collect the uncertain multi-view data due to the collection process is corrupted with noise in real-life application. In this case, this article proposes a novel approach, called uncertain multi-view learning with support vector machine (UMV-SVM) to cope with the problem of multi-view learning with uncertain data. The method first enforces the agreement among all the views to seek complementary information of multi-view data and takes the uncertainty of the multi-view data into consideration by modeling reachability area of the noise. Then it proposes an iterative framework to solve the proposed UMV-SVM model such that we can obtain the multi-view classifier for prediction. Extensive experiments on real-life datasets have shown that the proposed UMV-SVM can achieve a better performance for uncertain multi-view classification in comparison to the state-of-the-art multi-view classification methods.

Distance metric learning (DML) aims to learn a distance metric to process the data distribution. However, most of the existing methods are k NN DML methods and employ the k NN model to classify the test instances. The drawback of k NN DML is that all training instances need to be accessed and stored to classify the test instances, and the classification performance is influenced by the setting of the nearest neighbor number k . To solve these problems, there are several DML methods that employ the SVM model to classify the test instances. However, all of them are nonconvex and the convex support vector DML method has not been explicitly proposed. In this article, we propose a convex model for support vector DML (CSV-DML), which is capable of replacing the k NN model of DML with the SVM model. To make CSV-DML can use the most kernel functions of the existing SVM methods, a nonlinear mapping is used to map the original instances into a feature space. Since the explicit form of nonlinear mapped instances is unknown, the original instances are further transformed into the kernel form, which can be calculated explicitly. CSV-DML is constructed to work directly on the kernel-transformed instances. Specifically, we learn a specific Mahalanobis distance metric from the kernel-transformed training instances and train a DML-based separating hyperplane based on it. An iterated approach is formulated to optimize CSV-DML, which is based on generalized block coordinate descent and can converge to the global optimum. In CSV-DML, since the dimension of kernel-transformed instances is only related to the number of original training instances, we develop a novel parameter reduction scheme for reducing the feature dimension. Extensive experiments show that the proposed CSV-DML method outperforms the previous methods.

In the past few years, people have made great progress in image categorization based on convolutional neural networks (CNN) with the large-scale training set. However, in real-world applications, we may only have insufficient training samples for training CNN. To solve this problem, in this paper, we propose a latent variable augmentation method based on adversarial training (Lagat) to increase the categorization accuracy of CNN with insufficient training samples. In the proposed Lagat model, we propose a uniform loss function, where we take the following two tasks into account. Firstly, we consider the task that we augment latent variables (LVs) from a set of class-specific adaptive distributions and we propose constraints to adjust these LVs distributions. Secondly, we use these sampled LVs to train a predictive image classifier. Moreover, we present an alternative two-play minimization game to optimize this uniform loss function. In addition, the experiment results also demonstrate that the proposed Lagat method delivers higher accuracy than the existing state-of-the-art methods. The source code of the Lagat algorithm is available in Github11https://github.com/itapty-ily/Lagat.

Ordinal regression (OR) is a paradigm in supervised learning, which aims at learning a prediction model for ordered classes. The existing studies mainly focus on single-instance OR, and the multi-instance OR problem has not been explicitly addressed. In many real-world applications, considering the OR problem from a multiple-instance aspect can yield better classification performance than from a single-instance aspect. For example, in image retrieval, an image may contain multiple and possibly heterogeneous objects. The user is usually interested in only a small part of the objects. If we represent the whole image as a global feature vector, the useful information from the targeted objects that the user is of interest may be overridden by the noisy information from irrelevant objects. However, this problem fits in the multiple-instance setting well. Each image is considered as a bag, and each object region is treated as an instance. The image is considered as of the user interest if it contains at least one targeted object region. In this paper, we address the multi-instance OR where the OR classifier is learned on multiple-instance data, instead of single-instance data. To solve this problem, we present a novel multiple-instance ordinal regression (MIOR) method. In MIOR, a set of parallel hyperplanes is used to separate the classes, and the label ordering information is incorporated into learning the classifier by imputing the parallel hyperplanes with an order. Moreover, considering that a bag may contain instances not belonging to its class, for each bag, the instance which is nearest to the middle of the corresponding class is selected to learn the classifier. Compared with the existing single-instance OR work, MIOR is able to learn a more accurate OR classifier on multiple-instance data where only the bag label is available and the instance label is unknown. Extensive experiments show that MIOR outperforms the existing single-instance OR methods.

Multiple-instance learning (MIL) is a generalization of supervised learning which addresses the classification of bags. Similar to traditional supervised learning, most of the existing MIL work is proposed based on the assumption that a representative training set is available for a proper learning of the classifier. That is to say, the training data can appropriately describe the distribution of positive and negative data in the testing set. However, this assumption may not be always satisfied. In real-world MIL applications, the negative data in the training set may not sufficiently represent the distribution of negative data in the testing set. Hence, how to learn an appropriate MIL classifier when a representative training set is not available becomes a key challenge for real-world MIL applications. To deal with this problem, we propose a novel Sphere-Description-Based approach for Multiple-Instance Learning (SDB-MIL). SDB-MIL learns an optimal sphere by determining a large margin among the instances, and meanwhile ensuring that each positive bag has at least one instance inside the sphere and all negative bags are outside the sphere. Enclosing at least one instance from each positive bag in the sphere enables a more desirable MIL classifier when the negative data in the training set cannot sufficiently represent the distribution of negative data in the testing set. Substantial experiments on the benchmark and real-world MIL datasets show that SDB-MIL obtains statistically better classification performance than the MIL methods compared.

Ordinal regression (OR) is generally defined as the task where the input samples are ranked on an ordinal scale. OR has found a wide variety of applications, and a great deal of work has been done on it. However, most of the existing work focuses on supervised/semisupervised OR classification, and the semisupervised OR clustering problems have not been explicitly addressed. In real-world OR applications, labeling a large number of training samples is usually time-consuming and costly, and instead, a set of unlabeled samples can be utilized to set up the OR model. Moreover, although the sample labels are unavailable, we can sometimes get the relative ranking information of the unlabeled samples. This sample ranking information can be utilized to refine the OR model. Hence, how to build an OR model on the unlabeled samples and incorporate the sample ranking information into the process of improving the clustering accuracy remains a key challenge for OR applications. In this paper, we consider the semisupervised OR clustering problems with sample-ranking constraints, which give the relative ranking information of the unlabeled samples, and put forward a maximum margin approach for semisupervised OR clustering ( [Formula: see text]SORC). On one hand, [Formula: see text]SORC seeks a set of parallel hyperplanes to partition the unlabeled samples into clusters. On the other hand, a loss function is put forward to incorporate the sample ranking information into the clustering process. As a result, the optimization function of [Formula: see text]SORC is formulated to maximize the margins of the closest neighboring clusters and meanwhile minimize the loss associated with the sample-ranking constraints. Extensive experiments on OR data sets show that the proposed [Formula: see text]SORC method outperforms the traditional semisupervised clustering methods considered.

Multiple-instance learning (MIL) is a generalization of supervised learning that attempts to learn useful information from bags of instances. In MIL, the true labels of instances in positive bags are not available for training. This leads to a critical challenge, namely, handling the instances of which the labels are ambiguous (ambiguous instances). To deal with these ambiguous instances, we propose a novel MIL approach, called similarity-based multiple-instance learning (SMILE). Instead of eliminating a number of ambiguous instances in positive bags from training the classifier, as done in some previous MIL works, SMILE explicitly deals with the ambiguous instances by considering their similarity to the positive class and the negative class. Specifically, a subset of instances is selected from positive bags as the positive candidates and the remaining ambiguous instances are associated with two similarity weights, representing the similarity to the positive class and the negative class, respectively. The ambiguous instances, together with their similarity weights, are thereafter incorporated into the learning phase to build an extended SVM-based predictive classifier. A heuristic framework is employed to update the positive candidates and the similarity weights for refining the classification boundary. Experiments on real-world datasets show that SMILE demonstrates highly competitive classification accuracy and shows less sensitivity to labeling noise than the existing MIL methods.

This paper presents a novel framework to uncertain one-class learning and concept summarization learning on uncertain data streams. Our proposed framework consists of two parts. First, we put forward uncertain one-class learning to cope with data of uncertainty. We first propose a local kernel-density-based method to generate a bound score for each instance, which refines the location of the corresponding instance, and then construct an uncertain one-class classifier (UOCC) by incorporating the generated bound score into a one-class SVM-based learning phase. Second, we propose a support vectors (SVs)-based clustering technique to summarize the concept of the user from the history chunks by representing the chunk data using support vectors of the uncertain one-class classifier developed on each chunk, and then extend k-mean clustering method to cluster history chunks into clusters so that we can summarize concept from the history chunks. Our proposed framework explicitly addresses the problem of one-class learning and concept summarization learning on uncertain one-class data streams. Extensive experiments on uncertain data streams demonstrate that our proposed uncertain one-class learning method performs better than others, and our concept summarization method can summarize the evolving interests of the user from the history chunks.

The task of outlier detection is to identify data objects that are markedly different from or inconsistent with the normal set of data. Most existing solutions typically build a model using the normal data and identify outliers that do not fit the represented model very well. However, in addition to normal data, there also exist limited negative examples or outliers in many applications, and data may be corrupted such that the outlier detection data is imperfectly labeled. These make outlier detection far more difficult than the traditional ones. This paper presents a novel outlier detection approach to address data with imperfect labels and incorporate limited abnormal examples into learning. To deal with data with imperfect labels, we introduce likelihood values for each input data which denote the degree of membership of an example toward the normal and abnormal classes respectively. Our proposed approach works in two steps. In the first step, we generate a pseudo training dataset by computing likelihood values of each example based on its local behavior. We present kernel \(k\) -means clustering method and kernel LOF-based method to compute the likelihood values. In the second step, we incorporate the generated likelihood values and limited abnormal examples into SVDD-based learning framework to build a more accurate classifier for global outlier detection. By integrating local and global outlier detection, our proposed method explicitly handles data with imperfect labels and enhances the performance of outlier detection. Extensive experiments on real life datasets have demonstrated that our proposed approaches can achieve a better tradeoff between detection rate and false alarm rate as compared to state-of-the-art outlier detection approaches.

In textual data stream environment, concept drift can occur at any time, existing approaches partitioning streams into chunks can have problem if the chunk boundary does not coincide with the change point which is impossible to predict. Since concept drift can occur at any point of the streams, it will certainly occur within chunks, which is called random concept drift. The paper proposed an approach, which is called chunk level-based concept drift method (CLCD), that can overcome this chunking problem by continuously monitoring chunk characteristics to revise the classifier based on transfer learning in positive and unlabeled (PU) textual data stream environment. Our proposed approach works in three steps. In the first step, we propose core vocabulary-based criteria to justify and identify random concept drift. In the second step, we put forward the extension of LELC (PU learning by extracting likely positive and negative micro-clusters)[1], called soft-LELC, to extract representative examples from unlabeled data, and assign a confidence score to each extracted example. The assigned confidence score represents the degree of belongingness of an example towards its corresponding class. In the third step, we set up a transfer learning-based SVM to build an accurate classifier for the chunks where concept drift is identified in the first step. Extensive experiments have shown that CLCD can capture random concept drift, and outperforms state-of-the-art methods in positive and unlabeled textual data stream environments.

This paper presents a novel approach, called MODS, to build an accurate time evolving classifier from multiple one-class data streams learning time evolving classifier. Our proposed MODS approach works in two steps. In the first step, we first construct local one-class classifiers on the labeled positive examples from each sub-data stream respectively. We then collect the informative examples (support vectors) around each local one-class classifier, which can support the decision boundary of the classifier. This is called support vector preservation principle. In the second step, we construct a global one-class classifier on the collected informative examples. By using the support vector preservation principle, our proposed MODS explicitly addresses the problem of building accurate classifier from multiple one-class data streams. Extensive experiments on real life data streams have demonstrated that our MODS approach can achieve high performance and efficiency for the multiple one-class data streams learning in comparison with other approaches.

Positive and unlabelled learning (PU learning) has been investigated to deal with the situation where only the positive examples and the unlabelled examples are available. Most of the previous works focus on identifying some negative examples from the unlabelled data, so that the supervised learning methods can be applied to build a classifier. However, for the remaining unlabelled data, which can not be explicitly identified as positive or negative (we call them ambiguous examples), they either exclude them from the training phase or simply enforce them to either class. Consequently, their performance may be constrained. This paper proposes a novel approach, called similarity-based PU learning (SPUL) method, by associating the ambiguous examples with two similarity weights, which indicate the similarity of an ambiguous example towards the positive class and the negative class, respectively. The local similarity-based and global similarity-based mechanisms are proposed to generate the similarity weights. The ambiguous examples and their similarity-weights are thereafter incorporated into an SVM-based learning phase to build a more accurate classifier. Extensive experiments on real-world datasets have shown that SPUL outperforms state-of-the-art PU learning methods.

This paper presents a novel approach to one-class-based uncertain data stream learning. Our proposed approach works in three steps. Firstly, we put forward a local kernel-density-based method to generate a bound score for each instance, which refines the location of the corresponding instance. Secondly, we construct an uncertain one-class classifier by incorporating the generated bound score into a one-class SVM-based learning phase. Thirdly, we devise an ensemble classifier, integrated from uncertain one-class classifiers built on the current and historical chunks, to cope with the concept drift involved in the uncertain data stream environment. Our proposed method explicitly handles the uncertainty of the input data and enhances the ability of one-class learning in reducing the sensitivity to noise. Extensive experiments on uncertain data streams demonstrate that our proposed approach can achieve better performance and is highly robust to noise in comparison with state-of-the-art one-class learning method.

Feature extraction is an effective step in data mining and machine learning. While many feature extraction methods have been proposed for clustering, classification and regression, very limited work has been done on multi-class classification problems. In fact, the accuracy of multi-class classification problems relies on well-extracted features, the modeling part aside. This paper proposes a new feature extraction method, namely extracting orientation distance-based discriminative (ODD) features, which is particularly designed for multi-class classification problems. The proposed method works in two steps. In the first step, we extend the Fisher Discriminant idea to determine more appropriate kernel function and map the input data with all classes into a feature space. In the second step, the ODD features are extracted based on the one-vs-all scheme to generate discriminative features between a pattern and each hyperplane. These newly extracted features are treated as the representative features and are further used in the subsequent classification procedure. Substantial experiments on both UCI and real-world datasets have been conducted to investigate the performance of ODD features based multi-class classification. The statistical results show that the classification accuracy based on ODD features outperforms that of the state-of-the-art feature extraction methods.

Support vector data description (SVDD) is very useful for one-class classification. However, it incurs high time complexity in handling large scale data. In this paper, we propose a novel and efficient method, named K-Farthest-Neighbors-based Concept Boundary Detection (KFN-CBD for short), to improve the SVDD learning efficiency on large datasets. This work is motivated by the observation that SVDD classifier is determined by support vectors (SVs), and removing the non-support vectors (non-SVs) will not change the classifier but will reduce computational costs. Our approach consists of two steps. In the first step, we propose the K-farthest-neighbors method to identify the samples around the hyper-sphere surface, which are more likely to be SVs. At the same time, a new tree search strategy of M-tree is presented to speed up the K-farthest neighbor query. In the second step, the non-SVs are eliminated from the training set, and only the identified boundary samples are used to train the SVDD classifier. By removing the non-SVs, the training time of SVDD can be substantially reduced.Extensive experiments have shown that KFN-CBD achieves around 6 times speedup compared to the standard SVDD, and obtains the comparable classification quality as the entire dataset used.

This paper presents a novel hybrid approach to outlier detection by incorporating local data uncertainty into the construction of a global classifier. To deal with local data uncertainty, we introduce a confidence value to each data example in the training data, which measures the strength of the corresponding class label. Our proposed method works in two steps. Firstly, we generate a pseudo training dataset by computing a confidence value of each input example on its class label. We present two different mechanisms: kernel k-means clustering algorithm and kernel LOF-based algorithm, to compute the confidence values based on the local data behavior. Secondly, we construct a global classifier for outlier detection by generalizing the SVDD-based learning framework to incorporate both positive and negative examples as well as their associated confidence values. By integrating local and global outlier detection, our proposed method explicitly handles the uncertainty of the input data and enhances the ability of SVDD in reducing the sensitivity to noise. Extensive experiments on real life datasets demonstrate that our proposed method can achieve a better tradeoff between detection rate and false alarm rate as compared to four state-of-the-art outlier detection algorithms.

Multiple instance learning (MIL) is a generalization of supervised learning which attempts to learn useful information from bags of instances. In MIL, the true labels of the instances in positive bags are not always available for training. This leads to a critical challenge, namely, handling the ambiguity of instance labels in positive bags. To address this issue, this paper proposes a novel MIL method named SMILE (Similarity-based Multiple Instance LEarning). It introduces a similarity weight to each instance in positive bag, which represents the instance similarity towards the positive and negative classes. The instances in positive bags, together with their similarity weights, are thereafter incorporated into the learning phase to build an extended SVM-based predictive classifier. Experiments on three real-world datasets consisting of 12 subsets show that SMILE achieves markedly better classification accuracy than state-of-the-art MIL methods.

In SVMs-based multiple classification, it is not always possible to find an appropriate kernel function to map all the classes from different distribution functions into a feature space where they are linearly separable from each other. This is even worse if the number of classes is very large. As a result, the classification accuracy is not as good as expected. In order to improve the performance of SVMs-based multi-classifiers, this paper proposes a method, named multi-space-mapped SVMs, to map the classes into different feature spaces and then classify them. The proposed method reduces the requirements for the kernel function. Substantial experiments have been conducted on one-against-all, one-against-one, FSVM, DDAG algorithms and our algorithm using six UCI data sets. The statistical results show that the proposed method has a higher probability of finding appropriate kernel functions than traditional methods and outperforms others.

Support vector machines (SVMs), which were originally designed for binary classifications, are an excellent tool for machine learning. For the multiclass classifications, they are usually converted into binary ones before they can be used to classify the examples. In the one-against-one algorithm with SVMs, there exists an unclassifiable region where the data samples cannot be classified by its decision function. This paper extends the one-against-one algorithm to handle this problem. We also give the convergence and computational complexity analysis of the proposed method. Finally, one-against-one, fuzzy decision function (FDF), and decision-directed acyclic graph (DDAG) algorithms and our proposed method are compared using five University of California at Irvine (UCI) data sets. The results report that the proposed method can handle the unclassifiable region better than others.