Concept drift poses a challenge in the field of data stream mining. Most existing online deep learning methods rely on a single objective, failing to sufficiently capture the latent feature information embedded in the data, which hinders rapid adaptation to distribution changes. To address these issues, this paper proposes a Multi-objective Joint Optimization of Deep Network (MJOD) model. Specifically, the feature connection network (FCN) connects features from different layers via dense feature units, constructing an integrated classification objective to fully utilize both shallow and deep information for precise prediction. Subsequently, the feature refinement network (FRN) compares historical representative information with current data to build a compressed reconstruction objective, thereby capturing changes in data distribution. Furthermore, the feature enhancement network (FEN) extracts spatial structure information from the data and constructs an information balance objective to retain the information most relevant to the task. By jointly optimizing the integrated classification, compressed reconstruction, and information balance objectives, the model maximizes the utilization of feature information within the streaming data, enabling rapid learning of new distributions. Experimental results demonstrate that MJOD is superior to other baseline methods across various drift scenarios.

Data quality is essential for the performance of deep neural networks in various fields. However, label noise and class imbalance are common data issues, which cause deep learning models to overfit in real-world scenarios. Recent research solves the learning with noisy labels (LNL) problem by employing label correction or loss adjustment methods, which often rely on uncertainty estimation. Unfortunately, these methods usually do not work well with imbalanced datasets. To address both noisy and imbalanced data biases, we analyze the limitations of current discriminative models in uncertainty estimation, and propose Class-aware Multi-granularity Co-Diffusion models (CaMCoD), which leverage a generative uncertainty and inconsistency loss adjustment method to generate labels more robustly. Specifically, we reframe the LNL problem as a robust diffusion-generative process, i.e., labels are generated by gradually refining an initial random guess. First, we use coarse-grained uncertainty from the diffusion model to achieve more accurate confidence estimates. This will guide the model to generate correct labels on a broader level. Then, we leverage the fine-grained inconsistency of co-diffusion models during reverse denoising to determine the learnable weight for each sample, which can mitigate the risk of the model overfitting to noisy samples. Finally, we apply class-aware loss adjustments to reduce data bias caused by class imbalance. Experiments on both synthetic and real-world datasets demonstrate that our method perform well in imbalanced and noisy scenarios. We provide our code on GitHub:https://github.com/SenyuHou/CaMCoD.

Asynchronous pipeline model parallelism with a “1F1B” (one forward, one backward) schedule generates little bubble overhead and always provides quite a high throughput. However, the “1F1B” schedule inevitably leads to weight inconsistency and weight staleness issues due to the cross-training of different mini-batches across GPUs. To simultaneously address these two problems, in this paper, we propose an optimizer-dependent weight prediction strategy (a.k.a PipeOptim) for asynchronous pipeline training. The key insight of our proposal is that we employ a weight prediction strategy in the forward pass to approximately ensure that each mini-batch uses consistent and staleness-free weights to compute the forward pass of the “1F1B” schedule. To be concrete, we first construct the weight prediction scheme based on the update rule of the used optimizer when training the deep neural network models. Then throughout the “1F1B” pipeline training, each mini-batch is mandated to execute weight prediction, subsequently employing the predicted weights to perform the forward pass. As a result, PipeOptim 1) inherits the advantage of the “1F1B” schedule and generates high throughput, and 2) can ensure effective parameter learning regardless of the type of the used optimizer. We conducted extensive experimental evaluations using nine different deep-learning models to verify the effectiveness of our proposal. The experiment results demonstrate that PipeOptim outperforms the other five popular pipeline approaches including GPipe, PipeDream, PipeDream-2BW, SpecTrain, and XPipe.

Class overlap is a major factor of data complexity that hampers classifier performance, particularly in imbalanced learning scenarios. Most existing oversampling methods rely on conservative seed sample selection and decoupled synthesis strategies, which limit sample diversity and fail to effectively control overlap risk. This paper proposes a novel oversampling framework called TMACO (Class Alliance-Constrained Oversampling), which integrates data complexity considerations into both seed selection and sample generation. First, TMACO selects seed sample units using a class alliance constraint that jointly considers spatial geometry and class distribution to enhance diversity and representativeness. Second, it generates synthetic samples based on three-point units to ensure regional stability. Third, a region-level filtering mechanism is applied to prevent synthetic samples from intruding into majority class areas. Extensive experiments on benchmark and real-world datasets demonstrate that TMACO consistently improves minority class performance and overall classification accuracy compared to state-of-the-art oversampling techniques. The proposed method also offers interpretable parameter control and adapts well to varying task objectives.

Concept drift is an important characteristic and inevitable difficult problem in streaming data mining. Ensemble learning is commonly used to deal with concept drift. However, most ensemble methods cannot balance the accuracy and diversity of base learners after drift occurs, and cannot adjust adaptively according to the drift type. To solve these problems, this paper proposes a targeted ensemble learning (Targeted EL) method to improve the accuracy and diversity of ensemble learning for streaming data with abrupt and gradual concept drift. First, to improve the accuracy of the base learners, the method adopts different sample weighting strategies for different types of drift to realize bidirectional transfer of new and old distributed samples. Second, the difference matrix is constructed by the prediction results of the base learners on the current samples. According to the drift type, the submatrix with appropriate size and maximum difference sum is extracted adaptively to select appropriate, accuracy and diverse base learners for ensemble. The experimental results show that the proposed method can achieve good generalization performance when dealing with the streaming data with abrupt and gradual concept drift.

The linear regression model is simple in form and easy to estimate; nevertheless, irrelevant features will raise the difficulty of its tasks. Feature selection is generally adopted to improve the model performance. Unfortunately, traditional regression feature selection methods may not work for data with noise or outliers. Although some robust methods for certain specific error distributions have been proposed, they may not perform well because the distribution of representation error is often unknown for real data. This paper proposes a regression feature selection method for unknown noise named Mixture of Gaussians LASSO (MoG-LASSO), in which feature selection and model training will be achieved simultaneously. MoG is adopted to model unknown noises, and M-estimation is used to acquire the weighted squared error loss. By alternatively and iteratively updating the regression coefficient and parameters of MoG, the influence of unknown noise can be reduced effectively. Furthermore, MoG-LASSO achieves feature selection by the$L_{1}$regularization term, which can further improve the performance of the model. Experimental results on artificial data and benchmark data sets demonstrate that MoG-LASSO has better robustness and sparsity for data sets with irrelevant features. Additionally, experimental results on face recognition databases show the performance advantage of MoG-LASSO over state-of-the-art methods in the presence of illumination variations.

Monotonic classification is a kind of classification task in which a monotonicity constraint exist between features and class, i.e., if sample xihas a higher value in each feature than sample xj, it should be assigned to a class with a higher level than the level of xj's class. Several methods have been proposed, but they have some limits such as with limited kind of data or limited classification accuracy. In our former work, the classification accuracy on monotonic classification has been improved by fusing monotonic decision trees, but it always has a complex classification model. This work aims to find a monotonic classifier to process both nominal and numeric data by fusing complete monotonic decision trees. Through finding the completed feature subsets based on discernibility matrix on ordinal dataset, a set of monotonic decision trees can be obtained directly and automatically, on which the rank is still preserved. Fewer decision trees are needed, which will serve as base classifiers to construct a decision forest fused complete monotonic decision trees. The experiment results on 10 datasets demonstrate that the proposed method can reduce the number of base classifiers effectively and then simplify classification model, and obtain good classification performance simultaneously.