Label shift, which investigates the adaptation of label distributions between the fixed source and target domains, has attracted significant research interests and broad applications in offline settings. In real-world scenarios, however, data often arrives as a continuous stream. Addressing label shift in online learning settings is paramount. Existing strategies, which tailor traditional offline label shift techniques to online settings, have degraded performance due to the inconsistent estimation of label distributions and violation of convex assumption for theoretical guarantee. In this paper, we propose a novel method to ensure consistent adaptation to online label shift. We construct a new convex risk estimator that is pivotal for both online optimization and theoretical analysis. Furthermore, we enhance an optimistic online algorithm as the base learner and refine the classifier using an ensemble method. Theoretically, we derive a universal dynamic regret which achieves minimax optimal. Extensive experiments on both real-world datasets and human motion task demonstrate the superiority of our method comparing existing methods.

Label shift is a prevalent phenomenon encountered in open environments, characterized by a notable discrepancy in the label distributions between the source (training) and target (test) domains, whereas the conditional distributions given the labels remain invariant. Existing label shift methods adopt a two-step strategy: initially computing the importance weight and subsequently utilizing it to calibrate the target outputs. However, this conventional strategy overlooks the intricate interplay between output adjustment and weight estimation. In this paper, we introduce a novel approach termed as One-step Label Shift Adaptation (OLSA). Our methodology jointly learns the predictive model and the corresponding weights through a bi-level optimization framework, with the objective of minimizing an upper bound on the target risk. To enhance the robustness of our proposed model, we incorporate a debiasing term into the upper-level classifier training and devise a regularization term for the lower-level weight estimation. Furthermore, we present theoretical analyses about the generalization bounds, offering guarantees for the model's performance. Extensive experimental results substantiate the efficacy of our proposal.

Multi-view subspace clustering, which aims to partition a set of multi-source data into a common space, has recently attracted wide attention in the field of data analysis and machine learning. Traditional algorithms may face the problem of high complexity in calculating the self-expression matrix and in turning parameter. This paper proposes a novel multi-view subspace clustering model termed as Low-rank Multi-view Subspace Clustering via Adaptive Weight (LMSCAW). LMSCAW decomposes the self-expression matrix into the product of two low-rank representation matrices and thus can fix the rank of the self-expression matrix of each view to increase the stability of the algorithm. In addition, in order to learn the common representation matrix better, LMSCAW fuses the self-expression matrices among multiple views and implicitly weights each view by the Frobenius norm without additional parameters. Extensive experimental results on multiple benchmark datasets are provided to show the effectiveness of the proposed algorithm and its superior performance over other state-of-the-art methods.

In many image processing tasks, e.g., 3D reconstruction of dynamic scenes, different types of descriptions, a.k.a., views, of an object are emerging in a streaming way. Streaming view learning provides an effective solution to this dynamic view problem. In this paradigm, existing streaming view learning methods typically assume that all labels are accurate. However, in many real-world applications, the initial views may be not good enough for characterizing, leading to noisy labels that degrade classification performance. How to learn a model for simultaneous view evolving and label ambiguity is critical yet unexplored. In this paper, we propose a novel method called Streaming View Classification with Noisy Label (SVCNL). We calibrate noisy labels according to the emerging of new views, thereby reflecting the dynamic changes in the data more accurately. Leveraging the sequential and non-revisitable nature of views, the method tunes existing models to inherit information from previous stages by utilizing current-stage data. It reconstructs noisy labels through a label transition matrix and establishes relationships between true labels and samples using a graph embedding strategy, progressively correcting noisy labels. Together with the theoretical analyses about generalization bounds, extensive experiments demonstrate the effectiveness of the proposed approach.

Learning from data streams collected sequentially over time are widely spread in real-world applications. Previous methods typically assume that the data stream has a feature space with a fixed or clearly defined evolution pattern, as well as a balanced class distribution. However, in many practical scenarios, such as environmental monitoring systems, the frequency of anomalous events is significantly imbalanced compared to normal ones and the feature space dynamically changes due to ecological evolution and sensor lifespan. To alleviate this important but rarely studied problem, we propose the Adaptive Learning in Imbalace data streams with Unpredictable feature evolution (ALIU) algorithm. As data streams with imbalanced class distribution arrive, ALIU first mitigates the model's bias for the majority class by reweighting the adaptive gradient descent magnitudes between different classes. Then, a new loss function is proposed that simultaneously focuses on misclassifications and maintains model robustness. Further, when imbalanced data streams arrive with feature evolutions, we reuse the previously learned model and update the incomplete and augmented features by adopting the adaptive gradient strategy and ensemble method, respectively. Finally, we utilize the projected technique to build a sparse yet efficient model. Based on a few common and mild assumptions, we theoretically analyze that the ALIU satisfies a sub-linear regret bound under both convex and strong convex loss functions and the performance of model can be improved with the assistance of old features. Besides, extensive experimental results further demonstrate the effectiveness of our proposed algorithm.

Due to the timeliness and uncertainty of data acquisition, label shift, which assumes that the source (training) and target (test) label distributions differ, occurs with the changing environment and reduces the generalization ability of traditional models. To correct the label shift, existing methods estimate the true label distribution by prediction of target data from a source classifier, which results in high variance, especially with large label shift. In this paper, we tackle this problem by proposing a novel approach termed as Label Shift Correction via Bidirectional Marginal Distribution Matching (BMDM). Our approach matchs the label and feature marginal distributions simultaneously to ensure the stability of estimated class proportions. We prove theoretically that there is a unique optimal solution, i.e., true target label distribution, for our approach under mild conditions, and an efficient optimization strategy is also proposed. On this basis, in multi-shot scenario where label distribution changes continuously, we extend BMDM by designing a new distribution matching mechanism and constructing a regularization term that constrains the direction of label distribution change. Extensive experimental results validate the effectiveness of our approach over existing state-of-the-arts methods.

With the advent of vast data collection ways, data are often with multiple modalities or coming from multiple sources. Traditional multiview learning often assumes that each example of data appears in all views. However, this assumption is too strict in some real applications such as multisensor surveillance system, where every view suffers from some data absent. In this article, we focus on how to classify such incomplete multiview data in semisupervised scenario and a method called absent multiview semisupervised classification (AMSC) has been proposed. Specifically, partial graph matrices are constructed independently by anchor strategy to measure the relationships among between each pair of present samples on each view. And to obtain unambiguous classification results for all unlabeled data points, AMSC learns view-specific label matrices and a common label matrix simultaneously. AMSC measures the similarity between pair of view-specific label vectors on each view by partial graph matrices, and consider the similarity between view-specific label vectors and class indicator vectors based on the common label matrix. To characterize the contributions of different views, the p th root integration strategy is adopted to incorporate the losses of different views. By further analyzing the relation between the p th root integration strategy and exponential decay integration strategy, we develop an efficient algorithm with proved convergence to solve the proposed nonconvex problem. To validate the effectiveness of AMSC, comparisons are made with some benchmark methods on real-world datasets and in the document classification scenario as well. The experimental results demonstrate the advantages of our proposed approach.

Clustering is a fundamental and important step in many image processing tasks, such as face recognition and image segmentation. The performance of clustering can be largely enhanced if relevant weak supervision information is appropriately exploited. To achieve this goal, in this paper, we propose the Compound Weakly Supervised Clustering (CSWC) method. Concretely, CSWC incorporates two types of widely available and easily accessed weak supervision information from the label and feature aspects, respectively. To be specific, at the label level, the pairwise constraints are utilized as a kind of typical weak label supervision information. At the feature level, the partial instances collected from multiple perspectives have internal consistency and they are regarded as weak structure supervision information. To achieve a more confident clustering partition, we learn a unified graph with its similarity matrix to incorporate the above two types of weak supervision. On one hand, this similarity matrix is constructed by self-expression across the partial instances collected from multiple perspectives. On the other hand, the pairwise constraints, i.e., must-links and cannot-links, are considered by formulating a regularizer on the similarity matrix. Finally, the clustering results can be directly obtained according to the learned graph, without performing additional clustering techniques. Besides evaluating CSWC on 7 benchmark datasets, we also apply it to the application of face clustering in video data since it has vast application potentiality. Experimental results demonstrate the effectiveness of our algorithm in both incorporating compound weak supervision and identifying faces in real applications.

In many dynamic environment applications, with the evolution of data collection ways, the data attributes are incremental and the samples are stored with accumulated feature spaces gradually. For instance, in the neuroimaging-based diagnosis of neuropsychiatric disorders, with emerging of diverse testing ways, we get more brain image features over time. The accumulation of different types of features will unavoidably bring difficulties in manipulating the high-dimensional data. It is challenging to design an algorithm to select valuable features in this feature incremental scenario. To address this important but rarely studied problem, we propose a novel Adaptive Feature Selection method (AFS). It enables the reusability of the feature selection model trained on previous features and adapts it to fit the feature selection requirements on all features automatically. Besides, an ideal$\ell _{0}$-norm sparse constraint for feature selection is imposed with a proposed effective solving strategy. We present the theoretical analyses about the generalization bound and convergence behavior. After tackling this problem in a one-shot case, we extend it to the multi-shot scenario. Plenty of experimental results demonstrate the effectiveness of reusing previous features and the superior of$\ell _{0}$-norm constraint in various aspects, together with its effectiveness in discriminating schizophrenic patients from healthy controls.

In image processing, images are usually composed of partial views due to the uncertainty of collection and how to efficiently process these images, which is called incomplete multi-view learning, has attracted widespread attention. The incompleteness and diversity of multi-view data enlarges the difficulty of annotation, resulting in the divergence of label distribution between the training and testing data, named as label shift. However, existing incomplete multi-view methods generally assume that the label distribution is consistent and rarely consider the label shift scenario. To address this new but important challenge, we propose a novel framework termed as Incomplete Multi-view Learning under Label Shift (IMLLS). In this framework, we first give the formal definitions of IMLLS and the bidirectional complete representation which describes the intrinsic and common structure. Then, a multilayer perceptron which combines the reconstruction and classification loss is employed to learn the latent representation, whose existence, consistency and universality are proved with the theoretical satisfaction of label shift assumption. After that, to align the label distribution, the learned representation and trained source classifier are used to estimate the importance weight by designing a new estimation scheme which balances the error generated by finite samples in theory. Finally, the trained classifier reweighted by the estimated weight is fine-tuned to reduce the gap between the source and target representations. Extensive experimental results validate the effectiveness of our algorithm over existing state-of-the-arts methods in various aspects, together with its effectiveness in discriminating schizophrenic patients from healthy controls.

Actual condition-based maintenance for traction power supply equipment (TPSE) is almost based on completely observable equipment state. However, it is unpractical to accurately reveal the equipment state due to the inescapably uncertainty of state assessment. In order to optimize the maintenance of TPSE, a maintenance model based on partially observable Markov decision process is proposed in this paper. Firstly, the degradation process of the TPSE is described by a four-state Markov process, and the state residence time and its transition probability of the equipment are obtained by equaling fault times in the statistical period. Then, the imperfect maintenance is considered in this paper. And the failure risk of the TPSE after maintenance is quantified for optimizing both the economic cost and the reliability of maintenance strategy. Finally, the practical fault record data of 27.5 kV vacuum circuit breakers for a traction power supply system (TPSS) are used to verify the proposed model. The results show that the maintenance model can provide guidance on decision-making for the maintenance under uncertainty, and the determination of maintenance schemes to optimize both TPSE reliability and operational cost.