Time series forecasting relies on predicting future values from historical data, yet most state-of-the-art approaches—including transformer and multilayer perceptron-based models—optimize using Mean Squared Error (MSE), which has two fundamental weaknesses: its point-wise error computation fails to capture temporal relationships, and it does not account for inherent noise in the data. To overcome these limitations, we introduce the Residual-Informed Loss (RI-Loss), a novel objective function based on the Hilbert-Schmidt Independence Criterion (HSIC). RI-Loss explicitly models noise structure by enforcing dependence between the residual sequence and a random time series, enabling more robust, noise-aware representations. Theoretically, we derive the first non-asymptotic HSIC bound with explicit double-sample complexity terms, achieving optimal convergence rates through Bernstein-type concentration inequalities and Rademacher complexity analysis. This provides rigorous guarantees for RI-Loss optimization while precisely quantifying kernel space interactions. Empirically, experiments across eight real-world benchmarks and five leading forecasting models demonstrate improvements in predictive performance, validating the effectiveness of our approach.

Time series forecasting is an important task that involves analyzing temporal dependencies and underlying patterns (such as trends, cyclicality, and seasonality) in historical data to predict future values or trends. Current deep learning-based forecasting models primarily employ Mean Squared Error (MSE) loss functions for regression modeling. Despite enabling direct value prediction, this method offers no uncertainty estimation and exhibits poor outlier robustness. To address these limitations, we propose OCE-TS, a novel ordinal classification approach for time series forecasting that replaces MSE with Ordinal Cross-Entropy (OCE) loss, preserving prediction order while quantifying uncertainty through probability output. Specifically, OCE-TS begins by discretizing observed values into ordered intervals and deriving their probabilities via a parametric distribution as supervision signals. Using a simple linear model, we then predict probability distributions for each timestep. The OCE loss is computed between the cumulative distributions of predicted and ground-truth probabilities, explicitly preserving ordinal relationships among forecasted values. Through theoretical analysis using influence functions, we establish that cross-entropy (CE) loss exhibits superior stability and outlier robustness compared to MSE loss. Empirically, we compared OCE-TS with five baseline models—Autoformer, DLinear, iTransformer, TimeXer, and TimeBridge—on seven public time series datasets. Using MSE and Mean Absolute Error (MAE) as evaluation metrics, the results demonstrate that OCE-TS consistently outperforms benchmark models.

Multi-view data is widely present in the real world. Multi-view clustering is an unsupervised method for capturing the grouping structure of such data. However, multi-view clustering struggles to meet the requirements of real-world scenarios, such as distributed storage of different views and data protection needs. These requirements align with the setting of vertical federated clustering. However, vertical federated clustering still faces two challenges: (1) Under the constraints of privacy protection mechanisms, how to theoretically analyze the clustering consistency between the data uploaded by clients to the server and the original client data is challenging. (2) The feature space differences among different clients make cross-view information sharing and fusion difficult. To address the first challenge, we provide a theoretical analysis of the upper bound of the loss of k-means for transformation matrix mapping, revealing the relationship between the k-means loss of the transformed data and the original data. We then propose a vertical federated clustering method (V-HDKM). In this method, clients handle the second challenge by transposing the feature matrix. Guided by the projected k-means loss bound, we expand the feature space and perform k-means clustering to obtain feature cluster centers, which are then uploaded to the server. The server aggregates the global centers and feeds back the optimized results, achieving cross-view knowledge fusion through iterative interactions. Experimental results show that V-HDKM significantly improves local clustering performance and performances better than other seven vertical federated mthods on 20 multi-view datasets. Furthermore, sensitivity analysis on 8 UCI datasets with respect to the number of clients demonstrates the stability of the method. The code is available at https://github.com/jiangjh/V-HDKM.

Multi-view clustering aims to discover the group knowledge in the widely existing multi-view data. Consistency is one of the fundamental factors for effectively handling the multi-view data clustering problem. It has been observed that there are two types of consistent relations, consistently ambiguous relations and consistently determined relations, which have negative and positive impacts on clustering, respectively. However, most of the existing multi-view clustering methods treat the consistency relation without distinction. In this article, the sample’s stability in the sense of multi-view clustering is defined to recognize the consistently determined relations. Theoretically, it is revealed that the samples with higher stability have consistently determined relations with more other samples in all views, indicating a clear cluster structure. The rationality of the sample’s stability in multi-view is illustrated by experimental analysis. Further, a Multi-View Clustering Method Based on Sample’s Stability (MCSS) is proposed. This method first calculates the sample’s stability and divides the samples into the stable region and unstable region. Then, the cluster structure in the stable region is discovered. Finally, the samples in the unstable region are assigned based on the pre-discovered cluster structure. The effectiveness of the proposed method based on sample’s stability is illustrated on nine benchmark multi-view datasets compared with ten multi-view clustering methods. The demo code is available at https://github.com/FeijiangLi/MCSS .

Clustering ensemble has been a popular research topic in data science due to its ability to improve the robustness of the single clustering method. Many clustering ensemble methods have been proposed, most of which can be categorized into clustering-view and sample-view methods. The clustering-view method is generally efficient, but it could be affected by the unreliability that existed in base clustering results. The sample-view method shows good performance, while the construction of the pairwise sample relation is time-consuming. In this paper, the clustering ensemble is formulated as a k-HyperEdge Medoids discovery problem and a clustering ensemble method based on k-HyperEdge Medoids that considers the characteristics of the above two types of clustering ensemble methods is proposed. In the method, a set of hyperedges is selected from the clustering view efficiently, then the hyperedges are diffused and adjusted from the sample view guided by a hyperedge loss function to construct an effective k-HyperEdge Medoid set. The loss function is mainly reduced by assigning samples to the hyperedge with the highest degree of belonging. Theoretical analyses show that the solution can approximate the optimal, the assignment method can gradually reduce the loss function, and the estimation of the belonging degree is statistically reasonable. Experiments on artificial data show the working mechanism of the proposed method. The convergence of the method is verified by experimental analysis of twenty data sets. The effectiveness and efficiency of the proposed method are also verified on these data, with nine representative clustering ensemble algorithms as reference.

Deep learning excels at capturing complex data representations, yet quantifying the discriminative quality of these representations remains challenging. While unsupervised metrics often assess pairwise sample similarity, classification tasks fundamentally require class-level discrimination. To bridge this gap, we propose a novel loss function that evaluates representation discriminability via the Euclidean distance between the learned similarity matrix and the true class adjacency matrix. We identify random consistency—an inherent bias in Euclidean distance metrics—as a key obstacle to reliable evaluation, affecting both fairness and discrimination. To address this, we derive the expected Euclidean distance under uniformly distributed label permutations and introduce its closed-form solution, the Pure Square Euclidean Distance (PSED), which provably eliminates random consistency. Theoretically, we demonstrate that PSED satisfies heterogeneity and unbiasedness guarantees, and establish its generalization bound via the exponential Orlicz norm, confirming its statistical learnability. Empirically, our method surpasses conventional loss functions across multiple benchmarks, achieving significant improvements in accuracy, $F_1$ score, and class-structure differentiation. (Code is published in https://github.com/FeijiangLi/ICML2025-PSED)

Quantifying the informativeness of the similarity matrix has been successfully applied in kernel width selection, dimension size selection, and so on. Traditionally, the informational content is defined by calculating the distance between a matrix and non informative matrices. However, this method has limitations when dealing with adjacency matrices with the same marginal distribution but different structures, as it assigns the same distance values to these matrices. To address this issue, we introduce a new distance metric based on the adjacency matrix generated by label vectors, which accurately reflects the correct similarity structure. Through experimental analysis, we have demonstrated that the proposed distance metric can effectively distinguish matrices with different adjacency structures. This feature is crucial for capturing data diversity as it provides a stable measure of sample similarity for classification tasks. In graph neural networks (GNNs), reconstruction loss is crucial for model performance. To verify the effectiveness of the proposed metric, we apply it as a loss function in the training process of the GNN model. Extensive experimental results have shown that the proposed loss function can achieve higher accuracy compared to traditional loss functions.

Shape from focus (SFF) is a technique used to estimate the depth of a scene from a sequence of multifocus images. Existing SFF methods can be categorized into two groups: traditional methods and deep learning-based methods. Traditional methods typically employ a focus measure (FM) operator to assess the sharpness of individual pixels in a single-frame image, often overlooking the associations within the image sequence. Deep learning methods generally rely on labeled datasets, which are often challenging to obtain in real-world scenarios. Based on these observations, we propose a novel sequence association-based (SAS) framework aimed at enhancing the generalizability of SFF methods. In the SAS framework, an image sequence is treated as complete three-dimensional (3D) data throughout the processes of multiview decomposition, selective fusion and multiscale feature aggregation. Furthermore, the framework includes a tighter multiview learning generalization error bound to guide the development of the selective fusion method. This method leverages isomorphisms among multiple views to effectively mitigate the adverse effects of outlier noise on the reconstruction of various scenes. Comprehensive experiments on seven synthetic datasets and two real scenes with unknown labels demonstrate the effectiveness and generalizability of the SAS framework compared to state-of-the-art SFF methods.

Monotonic classification is a special ordinal classification task that involves monotonicity constraints between features and the decision. Monotonic feature selection can reduce dimensionality while preserving the monotonicity constraints, ultimately improving the efficiency and performance of monotonic classifiers. However, existing feature selection algorithms cannot handle large-scale monotonic data sets due to their lack of consideration for monotonic constraints or their high computational complexities. To address these issues, building on our team's previous research, we define the monotonic related family method with lower time complexity to select informative features and obtain multi-reducts carrying complementary information from multi-view for raw feature space. Using bi-directional rank mutual information, we build two trees for each feature subset and fuse all trees using the corresponding decision support level (BFMDT). Compared with six representative algorithms for monotonic feature selection, BFMDT's average classification accuracy increased by 4.06% (FFREMT), 6.77% (FCMT), 5.61% (FPRS_up), 6.05% (FPRS_down), 5.86%(FPRS_global), 4.41% (Bagging), 7.65% (REMT) and 21.89% (FMKNN), the average execution time compared to tree-based algorithms decreased by 83.41% (FFREMT), 96.96% (FCMT), 75.64% (FPRS_up), 59.43% (FPRS_down), 84.65%(FPRS_global), 81.50% (Bagging) and 63.41% (REMT), while most of comparing algorithms were unable to complete computation on six high-dimensional datasets.

The bagging method has received much application and attention in recent years due to its good performance and simple framework. It has facilitated the advanced random forest method and accuracy-diversity ensemble theory. Bagging is an ensemble method based on simple random sampling (SRS) method with replacement. However, SRS is the most foundation sampling method in the field of statistics, where exists some other advanced sampling methods for probability density estimation. In imbalanced ensemble learning, down-sampling, over-sampling, and SMOTE methods have been proposed for generating base training set. However, these methods aim at changing the underlying distribution of data rather than simulating it better. The ranked set sampling (RSS) method uses auxiliary information to get more effective samples. The purpose of this article is to propose a bagging ensemble method based on RSS, which uses the ordering of objects related to the class to obtain more effective training sets. To explain its performance, we give a generalization bound of ensemble from the perspective of posterior probability estimation and Fisher information. On the basis of RSS sample having a higher Fisher information than SRS sample, the presented bound theoretically explains the better performance of RSS-Bagging. The experiments on 12 benchmark datasets demonstrate that RSS-Bagging statistically performs better than SRS-Bagging when the base classifiers are multinomial logistic regression (MLR) and support vector machine (SVM).

Whole-body hydration (WBH) assessment is crucial in reducing the risk of dehydration, which can have significant health impacts when left untreated. Existing methods for monitoring WBH are invasive or require bulky equipment, making them impractical for everyday or continuous use. In this work, we introduce an arm-wearable electronic tattoo (e-tattoo) utilizing bioimpedance (Bio-Z) sensing for noninvasive, continuous, and mobile WBH monitoring. Although whole-body Bio-Z is a proven method for indicating WBH status, the effectiveness of using arm Bio-Z for this purpose has been unclear. Our IRB-approved study, which involved diuretic-induced dehydration, demonstrated a strong positive linear correlation between cross-arm Bio-Z and percent body weight loss, with Pearson's r = 0.967 ± 0.031 achieved in six human participants. Furthermore, results indicated that arm Bio-Z outperformed traditional whole-body Bio-Z recordings, likely due to the stretchable, flexible, and bodyconformal nature of the proposed e-tattoo. These findings suggest that the upper arm Bio-Z can serve as a reliable, continuous proxy for WBH, offering a cost-effective and highly accessible solution. The potential uses of this wearable technology range from improving personal wellness to helping professional sports, healthcare, and occupational safety.

The pure accuracy measure is used to eliminate random consistency from the accuracy measure. Biases to both majority and minority classes in the pure accuracy are lower than that in the accuracy measure. In this paper, we demonstrate that compared with the accuracy measure and F-measure, the pure accuracy measure is class distribution insensitive and discriminative for good classifiers. The advantages make the pure accuracy measure suitable for traditional classification. Further, we mainly focus on two points: exploring a tighter generalization bound on pure accuracy based learning paradigm and designing a learning algorithm based on the pure accuracy measure. Particularly, with the self-bounding property, we build an algorithm-independent generalization bound on the pure accuracy measure, which is tighter than the existing bound of an order O(1/√N) (N is the number of instances). The proposed bound is free from making a smoothness or convex assumption on the hypothesis functions. In addition, we design a learning algorithm optimizing the pure accuracy measure and use it in the selective ensemble learning setting. The experiments on sixteen benchmark data sets and four image data sets demonstrate that the proposed method statistically performs better than the other eight representative benchmark algorithms.

Fuzzy clustering ensemble that combines multiple fuzzy clustering results can obtain more robust, novel, stable, and consistent clustering result. The research about fuzzy clustering ensemble is still in the initial stage. Due to the special information expression, excellent clustering ideas are not well-practiced in fuzzy clustering ensemble and the performance of fuzzy clustering ensemble still has a large improvement space. In data clustering, prototype-based clustering is effective and efficient. The main idea of prototype-based clustering is discovering prototype samples to represent clusters and assigning samples to the represented clusters. In this article, we draw the idea of prototype-based clustering to fuzzy clustering ensemble and handle the problems of how to discover prototype samples based on a set of fuzzy clustering results and how to assign the samples without accessing the original data features. First, we propose a self-coassociation measure of a sample and discover its natural ability to evaluate the sample's local density. The rationality of the prototype samples discovered based on self-coassociation is theoretically analyzed and visually shown on eight artificial data sets. Then, we propose a prototype propagation method to assign data samples gradually. The working mechanism of the proposed sample assignment method is visually shown in the image segmentation scene. Finally, we develop a fuzzy clustering ensemble method based on self-coassociation and prototype propagation. The effectiveness of the proposed method is illustrated by comparing it with eight representative methods on benchmark datasets.

The clustering ensemble technique that integrates multiple clustering results can improve the accuracy and robustness of the final clustering. In many clustering ensemble algorithms, the co-association matrix (CA matrix), which reflects the frequency of any two samples being partitioned into the same cluster, plays an important role. However, generally, the CA matrix is highly sparse with low value density, which may limit the performance of an algorithm based on it. To handle these issues, in this paper, we propose a growing tree model (GoT). In this model, the CA matrix is firstly refined by the shortest path technique so that its sparsity will be mitigated. Then, a set of representative prototype examples is discovered. Finally, to handle the low value density of the CA matrix, the prototypes gradually connect to their neighborhood, which likes a set of trees growing up. The rationality of the discovered prototype examples is illustrated by theoretical analysis and experimental analysis. The working mechanism of the GoT is visually shown on synthetic data sets. Experimental analyses on eight UCI data sets and eight image data sets show that the GoT outperforms nine representative clustering ensemble algorithms.

Abstract Human beings may make random guesses in decision-making. Occasionally, their guesses may generate consistency with the real situation. This kind of consistency is termed random consistency. In the area of machine leaning, the randomness is unavoidable and ubiquitous in learning algorithms. However, the accuracy (A), which is a fundamental performance measure for machine learning, does not recognize the random consistency. This causes that the classifiers learnt by A contain the random consistency. The random consistency may cause an unreliable evaluation and harm the generalization performance. To solve this problem, the pure accuracy (PA) is defined to eliminate the random consistency from the A. In this paper, we mainly study the necessity, learning consistency and leaning method of the PA. We show that the PA is insensitive to the class distribution of classifier and is more fair to the majority and the minority than A. Subsequently, some novel generalization bounds on the PA and A are given. Furthermore, we show that the PA is Bayes-risk consistent in finite and infinite hypothesis space. We design a plug-in rule that maximizes the PA, and the experiments on twenty benchmark data sets demonstrate that the proposed method performs statistically better than the kernel logistic regression in terms of PA and comparable performance in terms of A. Compared with the other plug-in rules, the proposed method obtains much better performance.

Ordinal classification is an important classification task, in which there exists a monotonic constraint between features and the decision class. In this article, we aim to develop a method of fusing ordinal decision trees with fuzzy rough-set-based attribute reduction. Most of the existing attribute reduction methods for ordinal decision tables are based on the dominance rough set theory or significance measures. However, the crisp dominance relation is difficult in making full use of the information of attribute values; and the reducts based on significance measures are poor in interpretability and may contain unnecessary attributes. In this article, we first define a discernibility matrix with fuzzy dominance rough set. With this discernibility matrix, multiple reducts can be found, which provide multiple complementary feature subspaces with original information. Then, diverse ordinal trees can be established from these feature subspaces, and finally, the trees are fused by majority voting. The experimental results show that the proposed fusion method performs significantly better than other fusion methods using dominance rough set or significance measures.

Clustering ensemble has drawn much attention in recent years due to its ability to generate a high quality and robust partition result. Weighted clustering ensemble and selective clustering ensemble are two general ways to further improve the performance of a clustering ensemble method. Existing weighted clustering ensemble methods assign the same weight to each cluster in a partition of the ensemble. Since the qualities of the clusters in a partition are different, the clusters should be weighted differently. To address this issue, this article proposes a new measure to calculate the similarity between a cluster and a partition. Theoretically, this measure is effective in handling two problems in measuring the quality of a cluster, which are defined as the symmetric problem and the context meaning problem. In addition, some properties of the proposed measure are analyzed. This measure can be easily expanded to a clustering performance measure that calculates the similarity between two partitions. As a result of this measure, we propose a novel selective clustering ensemble framework, which considers the differences between the objective of the ensemble selection stage and the object of the ensemble integration stage in the selective clustering ensemble. To verify the performance of the new measure, we compare the performance of the measure with the two existing measures in weighting clusters. The experiments show that the proposed measure is more effective. To verify the performance of the novel framework, four existing state-of-the-art selective clustering ensemble frameworks are employed as references. The experiments show that the proposed framework is statistically better than the others on 17 UCI benchmark datasets, 8 document datasets, and the Olivetti Face Database.

Ordinal classification with a monotonicity constraint is a kind of classification tasks, in which the objects with better attribute values should not be assigned to a worse decision class. Several learning algorithms have been proposed to handle this kind of tasks in recent years. The rank entropy-based monotonic decision tree is very representative thanks to its better robustness and generalization. Ensemble learning is an effective strategy to significantly improve the generalization ability of machine learning systems. The objective of this work is to develop a method of fusing monotonic decision trees. In order to achieve this goal, we take two factors into account: attribute reduction and fusing principle. Through introducing variable dominance rough sets, we firstly propose an attribute reduction approach with rank-preservation for learning base classifiers, which can effectively avoid overfitting and improve classification performance. Then, we establish a fusing principe based on maximal probability through combining the base classifiers, which is used to further improve generalization ability of the learning system. The experimental analysis shows that the proposed fusing method can significantly improve classification performance of the learning system constructed by monotonic decision trees.