Traditional Discriminant analysis (DA) is one of the classical supervised learning algorithms to reduce the dimensionality of data with Gaussian assumption. Since the unique class mean in traditional DA is intractable to estimate the non-Gaussian distrbution of data, some existing DA algorithms based on the clustering criterion focus on learning multiple means in each class so as to address the non-Gaussian issue. The clustering-based DA inevitably involved the constraint optimization problem to learn multiple means, which may lead to the locally optimal solution. To address these issues, inspired by the smooth approximation theory and the concept of Kolmogorov mean, this paper explores an unconstraint function with asymptotic property as an alternative proxy to clustering-based DA algorithms. Thus the derived DA algorithm, i.e., adaptive and asymptotic mean-based subclass discriminant analysis (AASDA), which not only leverages multiple means to represent different subclasses in same class but also adaptively and asymptotically learns the similar mean for each sample in the learned optimal subspace via the gradient-based optimizer. The asymptotic analysis of unconstraint function, the gradient analysis and convergence guarantee of proposed criterion verify the effectiveness of AASDA algorithm. Its merits are thoroughly assessed on a suite of synthetic and real world data experiments.

Principal Component Analysis (PCA) aims to acquire the principal component space containing the essential structure of data, instead of being used for mining and extracting the essential structure of data. In other words, the principal component space contains not only information related to the essential structure of data but also some unrelated information. This frequently occurs when the intrinsic dimensionality of data is unknown or when it has complex distribution characteristics such as multi-modalities, manifolds, etc. Therefore, it is unreasonable to identify noise and useful information based solely on reconstruction error. For this reason, PCA is unsuitable as a preprocessing technique for most applications, especially in noisy environment. To solve this problem, this paper proposes robust PCA based on fuzzy local information reservation (FLIPCA). By analyzing the impact of reconstruction error on sample discriminability, FLIPCA provides a theoretical basis for noise identification and processing. This not only greatly improves its robustness but also extends its applicability and effectiveness as a data preprocessing technique. Meanwhile, FLIPCA maintains consistent mathematical descriptions with traditional PCA while having few adjustable hyperparameters and low algorithmic complexity. Finally, we conducted comprehensive experiments on synthetic and real-world datasets, which substantiated the superiority of our proposed algorithm.

To improve the robustness of SVM models to noise and outliers, fuzzy support vector machine (FSVM) has been proposed. However, many existing FSVM models have limitations such as their dependence on assumptions, limited optimization, and unreasonable handling of noise. To address these problems, we propose a novel approach called curriculum learning-based FSVM. Our approach employs a curriculum-learning strategy, where the model initially learns easy samples to avoid noise interference and obtain a good initial solution, before proceeding to learn all samples, including hard ones. To distinguish between easy and hard samples, we introduce an adaptive density-based clustering model, which is extended to kernel feature space. Moreover, we propose a slack variable-based fuzzy membership function to evaluate the importance of samples. Additionally, our model adaptively adapts the importance of samples based on feedback during the learning process. Finally, our experimental results on popular benchmarks demonstrate that our proposed model outperforms existing competitors in terms of accuracy and robustness.

Feature selection serves as a fundamental technique in machine learning and data analysis, playing a crucial role in extracting valuable features from large-scale and high-dimensional datasets that may contain irrelevant features. To enhance the performance of feature selection, regularizers like${\ell _{1}}$-norm or${\ell _{2,1}}$-norm are commonly utilized to encourage sparsity. Nonetheless, these traditional regularization techniques encounter certain challenges. When correlations exist among features, the sparsity-driven regularization can unfairly diminish weights of correlated features to zero, thus ignoring the feature correlations and lacking group sparsity properties. While a straightforward combination of${\ell _{1}}$-norm and${\ell _{2}}$-norm can uncover feature correlations, it lacks adaptability and effectively balancing sparsity and correlation. To address these challenges, we introduce a novel matrix-based regularization term, called a fuzzy elastic net, in the unsupervised feature selection model. Our model is founded on principal component analysis, a well-established dimensionality reduction technique adept at finding subspaces that retain most information from raw data. The model is enhanced by a fuzzy elastic net, which promotes group or sparsity properties through adaptive parameter tuning. The new regularization term introduces a flexible fuzzy weighted scheme combining the${\ell _{2,2}}$-norm and${\ell _{2,p}}$-norm ($0< p\leq 1$). This approach allows adaptive adjustment based on data characteristics, offering a tunable balance between selecting discriminative features and identifying correlated ones. Consequently, this regularization term equips the model to handle diverse data analysis tasks flexibly, thereby enhancing adaptability and generalization performance. Furthermore, we propose an efficient optimization strategy to solve this model. Extensive experiments conducted on UCI datasets and real-world datasets demonstrate the effectiveness and efficiency of our proposed method.

This paper presents a hierarchical LiDAR simulation framework to address the challenges of accurately simulating LiDAR data in autonomous driving scenarios. The framework utilizes a homology mapping approach to integrate LiDAR responses hierarchically at three levels: the instantaneous power response, environmental optical channel response, and target reflection response of LiDAR. This allows for the dynamic coupling of LiDAR geometric and physical models with varying environmental parameters. By integrating an array of interactions intrinsic to the LiDAR system and its external environment, the proposed model can provide high-fidelity LiDAR point cloud simulations. The effectiveness of the simulated point clouds has been validated through extensive experiments using actual LiDAR data and detection algorithms trained on existing datasets. The experimental results show that the proposed method has the potential to improve the realism of LiDAR simulations and the accumulation of challenging perception data.

Principal component analysis (PCA) is one of the most versatile techniques for unsupervised dimension reduction, which is implemented as a fundamental preprocessing method in multiple tasks of statistics and machine learning research because of its efficiency. Nevertheless, researchers have concentrated on the identification of outliers that do not conform to the low-dimensional approximation through statistical methods, e.g., outlier rejection, without giving insights on each data point with a dynamic ratio of signal-to-noise components in the high-dimensional regimes. To characterize the dynamic nature of the principal component information, we propose a Normalized Robust PCA with Adaptive Reconstruction Error minimization model, which considers both the adaptive normalization technique and flexible weights learning simultaneously. With this configuration, the principal component information constantly adjusts the degree of sparsity for activated samples. In other words, the signal component's discrimination and noise information restriction could work cooperatively. Empirical studies on one synthetic dataset and several benchmarks demonstrate the effectiveness of our proposed method over existing outlier rejection methods.

Recently, several robust principal component analysis (RPCA) models were presented to enhance the robustness of PCA by exploiting the robust norms as their loss functions. But an important problem is that they have no ability to discriminate outliers from correct samples. To solve this problem, we propose a RPCA method based on discriminant information (RPCA-DI). RPCA-DI disentangles the robust PCA with a two-step fashion: the identification and the processing of outliers. To identity outliers, a sample representation model based on entropy regularization is constructed to analyze the membership of data belonging to the principal component space(PC) and its orthogonal complement(OC), the discriminative information of data will be extracted based on measuring the differences of retained information on PC(or OC) of data. By this way, we can discriminate correct samples when we deal with outliers, which is more reasonable for robustness learning respective to previous works. In the noise processing step, in addition to considering the levels of noise, the resistance of the sample points to noise is also considered to prevent overfitting, thereby improving the generalization performance of RPCA-DI. Finally, an iterative algorithm is designed to solve the corresponding model. Compared with some state-of-art RPCA methods on artificial datasets, UCI datasets and face databases that verifies the effectiveness of RPCA-DI.

Robust principal component analysis (RPCA) is a technique that aims to make principal component analysis (PCA) robust to noise samples. The current modeling approaches of RPCA were proposed by analyzing the prior distribution of the reconstruction error terms. However, these methods ignore the influence of samples with large reconstruction errors, as well as the valid information of these samples in principal component space, which will degrade the ability of PCA to extract the principal component of data. In order to solve this problem, Fuzzy sparse deviation regularized robust principal component Analysis (FSD-PCA) is proposed in this paper. First, FSD-PCA learns the principal components by minimizing the square of ℓ2-norm-based reconstruction error. Then, FSD-PCA introduces sparse deviation on reconstruction error term to relax the samples with large bias, thus FSD-PCA can process noise and principal components of samples separately as well as improve the ability of FSD-PCA for retaining the principal component information. Finally, FSD-PCA estimates the prior probability of each sample by fuzzy weighting based on the relaxed reconstruction error, which can improve the robustness of the model. The experimental results indicate that the proposed model performs excellent robustness against different types of noise than the state-of-art algorithms, and the sparse deviation term enables FSD-PCA to process noise information and principal component information separately, so FSD-PCA can filter the noise information of an image and restore the corrupted image.

Locality preserving projections (LPP) method is a classical manifold learning method for dimensionality reduction. However, LPP is sensitive to outliers since squared L2‐norm may exaggerate the distance of outliers. Besides, the normalisation constraint of LPP may impair its robustness during embedding. Motivated by this observation, the authors propose a novel robust LPP using angle‐based adaptive weight (RLPP‐AAW) method. RLPP‐AAW not only considers the distance metric of training samples, but also take the reconstruction error into account, so as to reduce the influence of outliers and noise in the embedding process. In the RLPP‐AAW, based on the angle between distance metric and reconstruction error, a novel way is used to combine them in the objective function. Besides, RLPP‐AAW employs the L21‐norm criterion, which retains rotational invariance and is more robust than squared L2‐norm. An iterative algorithm is presented to solve the objective function of RLPP‐AAW. Experimental results on the benchmark databases illustrate the effectiveness of the proposed algorithm.

Fuzzy c‐means algorithms have been widely utilised in several areas such as image segmentation, pattern recognition and data mining. However, the related studies showed the limitations in facing imbalanced datasets. The maximum fuzzy boundary tends to be located on the largest cluster which is not desirable. The overall fuzzy partition results in false grouping of edge objects and weakens the compactness of cluster. It is important the clusters are delineated by the maximum fuzzy boundary. In this study, a semi‐fuzzy c‐means algorithm is proposed by combining hard partition and soft partition. This study aims to provide an effective partition for the edge objects, such that the compactness of cluster can be improved. The proposed algorithm integrates the semi‐fuzzy c‐means method with the size‐insensitive integrity‐based fuzzy c‐means algorithm. In particular, the latter algorithm has the ability to deal with imbalanced data. With the experiment validation, the proposed algorithm is robust and outperforms the two component algorithms by using synthetic and widely known benchmark datasets.

AdaBoost is a method to improve a given learning algorithm's classification accuracy by combining its hypotheses. Adaptivity, one of the significant advantages of AdaBoost, makes AdaBoost maximize the smallest margin so that AdaBoost has good generalization ability. However, when the samples with large negative margins are noisy or atypical, the maximized margin is actually a “hard margin.” The adaptive feature makes AdaBoost sensitive to the sampling fluctuations, and prone to overfitting. Therefore, the traditional schemes prevent AdaBoost from overfitting by heavily damping the influences of samples with large negative margins. However, the samples with large negative margins are not always noisy or atypical; thus, the traditional schemes of preventing overfitting may not be reasonable. In order to learn a classifier with high generalization performance and prevent overfitting, it is necessary to perform statistical analysis for the margins of training samples. Herein, Hoeffding inequality is adopted as a statistical tool to divide training samples into reliable samples and temporary unreliable samples. A new boosting algorithm, which is named DAdaBoost, is introduced to deal with reliable samples and temporary unreliable samples separately. Since DAdaBoost adjusts weighting scheme dynamically, the loss function of DAdaBoost is not fixed. In fact, it is a series of nonconvex functions that gradually approach the 0-1 function as the algorithm evolves. By defining a virtual classifier, the dynamic adjusted weighting scheme is well unified into the progress of DAdaBoost, and the upper bound of training error is deduced. The experiments on both synthetic and real world data show that DAdaBoost has many merits. Based on the experiments, we conclude that DAdaBoost can effectively prevent AdaBoost from overfitting.