Federated Continual Learning (FCL) approaches exist two major problems of the probability bias and the imbalance in parameter variations. These two problems lead to catastrophic forgetting of the network in the FCL process . Therefore, this paper proposes a novel FCL framework, Federated Probability Memory Recall (FedPMR), to mitigate the probability bias problem and the imbalance in parameter variations. Firstly, for the probability bias problem, this paper designs the Probability Distribution Alignment (PDA) module, which consolidates the memory of old probability experience. Specifically, PDA maintains a replay buffer and uses the probability memory stored in the buffer to correct the offset probabilities of the previous tasks during the two-stage training. Secondly, to alleviate the imbalance in parameter variations, this paper designs the Parameter Consistency Constraint (PCC) module, which constrains the magnitude of neural weight changes for previous tasks. Concretely, PCC applies a set of adaptive weights to subsets of the regularization term that constrains parameter changes, forcing the current model to be sufficiently close to the past model in parameter space distance. Experiments with various levels of task similitude across clients demonstrate that our technique establishes the new state-of-the-art performance when compared to previous FCL approaches.

In the face of high-dimensional and complex data, effective subspace can preserve specific statistical properties and provide an appropriate representation of data, which generally facilitates the underlying tasks such as clustering or classification. Meanwhile, multiple kernel learning is a technique to combine multiple kernels from different feature spaces effectively. Thus, by incorporating multiple kernels into the process of subspace learning, different feature spaces can be projected into a unified subspace. This paper proposes the Multiple Kernel Subspace Learning (MKSL) for embedding the original space into a unified subspace. Multiple kernels of different feature spaces are combined by MKSL in the process of learning, which can extend the suitability for various applications. Moreover, to generate the optimal combination kernel of subspace learning, we propose a two-step iteration strategy to learn the appropriate kernel weights and transformation matrix of projecting simultaneously. Furthermore, our proposed formulation of MKSL can introduce different prior knowledge such as class information and neighborhood relationships. Thus it is competent to the unsupervised learning, semi-supervised learning, and supervised learning. Extensive experiments are conducted on diverse datasets, and the performances are comprehensively evaluated on different tasks. The experimental results indicate that the proposed algorithm is outstanding in unsupervised clustering task and effective in supervised and semi-supervised classification tasks.

Semisupervised multiview learning gains extensive research attention due to its strong capability to utilize the heterogeneous features and the label information of a few labeled samples. However, the supervision information is not well utilized in the process of exploring the consensus structure of the multiview data. In this paper, we propose a novel unified pseudolabel-guided multiview consensus (PMvC) learning framework for the semisupervised classification problem, which learns the consensus structure of multiview data by fully exploiting the supervised information of labeled samples. Specifically, PMvC first assigns multiple pseudolabels to the unlabeled samples by selecting the nearest labeled sample in each view separately, and then labels the part of unlabeled samples by selecting the pseudolabel that agrees across all views. By doing so, the high-confident pseudolabeled samples can be selected to enlarge the labeled sample pool and the supervision information can be exploited further in the learning process. In addition, to capture the consensus structure of the multiview data, PMvC learns a consensus graph from the view-specific self-representation graph guided by enhanced supervision information, which better preserves the manifold structure of samples. Meanwhile, the label information is also propagated from the labeled samples to the unlabeled samples by the learned consensus graph simultaneously. Accordingly, an effective optimization algorithm is derived to find the optimal solution for PMvC. Extensive experiment results on several real-world data sets demonstrate the feasibility and superiority of PMvC. The source code of PMvC is available at https://github.com/justcallmewilliam/PMvC.

Multiple empirical kernel learning (MEKL) is a scalable and efficient supervised algorithm based on labeled samples. However, there is still a huge amount of unlabeled samples in the real-world application, which are not applicable for the supervised algorithm. To fully utilize the spatial distribution information of the unlabeled samples, this paper proposes a novel semi-supervised multiple empirical kernel learning (SSMEKL). SSMEKL enables multiple empirical kernel learning to achieve better classification performance with a small number of labeled samples and a large number of unlabeled samples. First, SSMEKL uses the collaborative information of multiple kernels to provide a pseudo labels to some unlabeled samples in the optimization process of the model, and SSMEKL designs pseudo-empirical loss to transform learning process of the unlabeled samples into supervised learning. Second, SSMEKL designs the similarity regularization for unlabeled samples to make full use of the spatial information of unlabeled samples. It is required that the output of unlabeled samples should be similar to the neighboring labeled samples to improve the classification performance of the model. The proposed SSMEKL can improve the performance of the classifier by using a small number of labeled samples and numerous unlabeled samples to improve the classification performance of MEKL. In the experiment, the results on four real-world data sets and two multiview data sets validate the effectiveness and superiority of the proposed SSMEKL.

Sampling method is one of the most commonly used techniques in dealing with imbalanced data. Most of the existing undersampling methods randomly select samples from negative class with replacement. However, it may lose some important information of the training data. Moreover, increasing the positive data by oversampling in high imbalanced situations may cause the overlapping problem. To overcome these problems, this paper proposes a hybrid sampling method. The method takes the distributions of the training data into consideration by the information entropy, thus distinguishing the important samples in the undersampling procedure. Meanwhile, since the positive data only extend to the size of each subset of the negative class in the oversampling, the overlapping problem is relieved. Further, the method retains all the data in the training procedure and generates various data views from the original training data. Then each view is handled with an individual basic classifier. Finally, all the basic classifiers are combined by the ensemble method. The newly proposed method is named as Entropy-based Hybrid Sampling Ensemble Learning (EHSEL). In addition, the EHSEL is applied to three different kinds of basic classifiers to validate its robustness. Experiments results show the great effectiveness of the EHSEL on real-world imbalanced data sets.