Semi-supervised learning (SSL) based on pseudo-label and consistency has achieved significant success. The core idea behind these methods is to assign sample weights based on pseudo-label probabilities, thereby guiding the model toward biased learning. However, existing research still faces two major challenges in guiding learning: (1) how to evaluate learning states across different classes in the absence of labels, and (2) how to construct an effective sample weight space that provides precise guidance throughout training. To address these challenges, we propose the Bi-Dimensional Sample Weight Guidance algorithm, BidMatch. BidMatch introduces Class Information Entropy (CIE), which captures the learning relationships between classes and reflects the model’s learning state for each class. Additionally, Pseudo-label Probability Redistribution (PPR) is proposed to maintain distribution invariance and sparsity during training, thereby emphasizing differences in sample importance. By leveraging CIE and PPR, BidMatch generates sample weights that account for both class and instance dimensions, effectively guiding the model toward balanced and efficient learning across classes. BidMatch has demonstrated state-of-the-art performance on various SSL datasets. Notably, it achieved a 6.45% error rate on CIFAR-10 with only one label per class, significantly outperforming baseline methods.

The vulnerability of graph-based recommender systems to spurious correlations has become a significant obstacle to their practical deployment, hindering their robustness in out-of-distribution (OOD) scenarios. While existing approaches offer partial solutions, they are limited by fundamental shortcomings: model-centric approaches reliant on predefined causal graphs often suffer from suboptimal performance due to complex and dynamic environmental influences. These methods typically require identifying an environmental label or performing feature decoupling, but hidden environments are often difficult to model. Furthermore, existing general feature decoupling methods fail to account for the unique structural characteristics of graphs. To overcome these challenges, we advocate for a shift towards explicit, geometrically-grounded disentanglement. Hyperbolic geometry is particularly suited for this task due to its capacity to model the inherent hierarchies of user interests. We introduce C-HyPOD : Causal Hyperbolic Representation Learning with Prototype Orthogonal Disentanglement, a novel framework designed for graph-based OOD recommendation. Unlike traditional methods, C-HyPOD transforms disentanglement into a concrete geometric task. It introduces a global interest space by learning a single set of universal interest prototypes. They provides a superior geometric foundation for ensuring these prototypes are well-separated and semantically distinct. To ensure a complete separation and prevent information leakage, a targeted orthogonality constraint is then applied. This constraint purifies the aggregated causal representation by forcing it to be orthogonal to the spurious representation in the tangent space, thereby eliminating their linear correlation. Extensive experiments on four public datasets demonstrate that C-HyPOD significantly improves OOD robustness and recommendation performance, surpassing state-of-the-art methods.

Safe operation is a key concern for industrial robots. However, due to hardware failures and unstable data transmission issues, the multivariate time-series data generated by these axes often contain missing or corrupted signals, which severely hinders downstream tasks such as fault diagnosis. Additionally, the substantial volume of industrial data demands considerable time for training time-series imputation models and subsequent classification models. To address these challenges, this study proposes a multitask approach that serves both the data imputation and fault diagnosis tasks for industrial robots. Specifically, the parameters trained in the imputation model can be transferred to the fault diagnosis model, enhancing its performance and efficiency. A multitask method named Bidirectional-Graph Attention Networks Parallel Encoder (Bi-GATPE) is proposed, which employs a bidirectional graph attention network to capture the spatial dependencies among the various variables of industrial robots. Subsequently, a parallel encoder with Diagonal-Filter Attention is designed to model temporal correlations. This dual approach improves the accuracy and training speed for both the imputation and fault diagnosis tasks. Experimental studies based on real industrial robot datasets demonstrate that by modifying the feature fusion layer of the imputation task and sharing the trained parameters with the fault diagnosis task, the proposed method significantly accelerates the convergence of the fault diagnosis model while also improving diagnostic accuracy. The experiments also indicate that our method shows merits in the imputation and fault diagnosis tasks. The source code of Bi-GATPE is available at:https://github.com/miten073/Bi-GATPE.

Semi-supervised learning can leverage both labeled and unlabeled samples simultaneously to improve performance. However, existing methods often present the following issues: (1) The emphasis of learning is put on either the similarity structures or the regression losses of data, neglecting the interaction between them. (2) The similarity structures among boundary samples might be unreliable, which misleads label propagation and impairs the performance of models on out-of-sample data. (3) They often involve the inverses of high-order matrices, making them inefficient in computation. To overcome these issues, we propose a scalable semi-supervised learning framework with Discriminative Label Propagation and Correction (DLPC), which collaboratively exploits the regression losses and similarity structures of data. Particularly, each sample is projected onto the independent class labels associated with nonnegative adjustment vectors rather than the propagated labels, such that the distances between samples from different classes are naturally enlarged, making regression losses more effective for boundary samples. Benefiting from this, the regression losses can guide the propagation of labels in boundary areas. Thus, the label information is first propagated through dynamically optimized graph structures and then corrected by the regression losses, effectively improving the quality of labels and facilitating feature projection learning. Furthermore, an accelerated solution has been developed to reduce the computational costs of DLPC on sample scales, thereby making it scalable to relatively large-scale problems. Moreover, the proposed DLPC can not only be applied to single-view scenarios but also extended to multi-view tasks. Additionally, an optimization strategy with fast convergence has been presented for DLPC, and extensive experiments demonstrate the effectiveness and superiority of DLPC over state-of-the-art competitors.

Multi-view clustering has been empirically shown to improve learning performance by leveraging the inherent complementary information across multiple views of data. However, in real-world scenarios, collecting strictly aligned views is challenging, and learning from both aligned and unaligned data becomes a more practical solution. Partially View-aligned Clustering (PVC) aims to learn correspondences between misaligned view samples to better exploit the potential consistency and complementarity across views, including both aligned and unaligned data. However, most existing PVC methods fail to leverage unaligned data to capture the shared semantics among samples from the same cluster. Moreover, the inherent heterogeneity of multi-view data induces distributional shifts in representations, leading to inaccuracies in establishing meaningful correspondences between cross-view latent features and, consequently, impairing learning effectiveness. To address these challenges, we propose a Semantic MAtching contRasTive learning model (SMART) for PVC. The main idea of our approach is to alleviate the influence of cross-view distributional shifts, thereby facilitating semantic matching contrastive learning to fully exploit semantic relationships in both aligned and unaligned data. Specifically, we mitigate view distribution shifts by aligning cross-view covariance matrices, which enables the inference of a semantic graph for all data. Guided by the learned semantic graph, we further exploit semantic consistency across views through semantic matching contrastive learning. After the optimization of the above mechanisms, our model smoothly performs semantic matching for different view embeddings instead of the cumbersome view realignment, which enables the learned representations to enjoy richer category-level semantics and stronger robustness. Extensive experiments on eight benchmark datasets demonstrate that our method consistently outperforms existing approaches on the PVC problem. The code is available at https://github.com/THPengL/SMART.

Semi-supervised multiview clustering has garnered considerable attention for its ability to integrate multiview data with limited labeled information. However, existing methods predominantly focus on labeled samples, neglecting abundant unlabeled data, which leads to suboptimal utilization of available prior knowledge. Moreover, existing pairwise constraint propagation-based methods typically follow a two-stage procedure, resulting in unstable clustering outcomes. To address these limitations, we introduce a unified framework that integrates multiview subspace clustering with pairwise constraint propagation, proposing a tensor-based semi-supervised multiview subspace clustering (TSMSC) method with pairwise constraint propagation. Specifically, each view's subspace representation is decomposed into a consensus part and private parts, enabling the consensus representation to better approximate the low-rank structure. Then, a pairwise constraint propagation method is developed for multiview data, which propagates the initial pairwise constraint through a low-rank matrix completion approach. Finally, observing that the ideal multiview consensus representation and the propagated pairwise constraint matrix share the same low-rank structure, we naturally construct them into a third-order tensor to capture high-order correlations via tensor low-rank representation, allowing for joint optimization within a unified framework. Extensive experiments on eight real-world datasets demonstrate the superiority of our method over state-of-the-art approaches.

Cross-modal retrieval facilitates more flexible information access and improves semantic understanding across different modalities. However, traditional cross-modal retrieval models rely on well-aligned datasets, which are often labor-intensive and costly to obtain. In real-world applications, data inevitably includes mismatched pairs, and these semantically inconsistent pairs can significantly degrade retrieval performance. Previous approaches have assumed ideal loss value distributions to optimize models for accurate semantic matching through soft-label estimation. However, the absence of hierarchical semantic correlation learning limits the effectiveness of these models in scenarios involving partial mismatches. To address these challenges, we propose Exploring Hierarchical Cross-Modal Correlation Consistency (EH3C) for cross-modal retrieval under partially mismatched conditions. Specifically, our approach first leverages neighborhood correlation distributions among samples to optimize cross-modal alignment, without assuming ideal distributions. This allows for the measurement of soft matching degrees between cross-modal data pairs and facilitates the effective learning of their positive correlations. Next, we enhance inter-class separability through intra-modal correlation learning by exploiting negative correlations between reliable negative sample pairs, thus enabling a more comprehensive exploration of cross-modal correlations. Finally, to assess the effectiveness and robustness of our approach, we conducted extensive experiments on three benchmark datasets. The results demonstrate that the proposed EH3C significantly improves cross-modal retrieval performance in scenarios involving partial mismatches.

Recommendation systems alleviate the issue of information overload via modeling user preferences from interaction sequences. Although self-attention based sequential models effectively capture long-range dependencies, they are susceptible to noise amplification in sparse sequences and over-smoothing of item representations, which obscures true user intent and reduces sensitivity to fine-grained behavioral changes. To overcome these challenges, we propose SAFA, a sparse sequential recommendation framework comprising: (1) an adaptive sparse attention mechanism that suppresses noisy interactions while preserving embedding diversity; (2) a frequency-aware encoder that decomposes interaction sequences into low-frequency components for long-term preference modeling and high-frequency components for short-term intent dynamics; and (3) a simplified focal loss that removes the class-balancing term while preserving the focusing factor, emphasizing hard-to-predict samples rather than class priors. Experiments on seven benchmark datasets demonstrate that SAFA consistently achieve state-of-the-art performance with average improvements of up to 3.77%, 4.10% and 4.25% in terms of HR@5, HR@10 and HR@20, respectively, and 4.30%, 4.78% and 4.58% in terms of NDCG@5, NDCG@10 and NDCG@20, respectively, over the best competing model. Ablation studies verify the importance of each component, with notable performance degradation upon removing the sparse attention or frequency-aware encoder. Overall, SAFA enhances sequential recommendation by improving robustness and discriminative learning under noisy and sparse conditions.

Anchor-based multi-view clustering has gained increasing attention for its efficiency in approximating similarity structures and scaling to large datasets. To reduce the burden of manual hyper-parameter tuning, recent studies have introduced parameter-free extensions. However, existing methods still face critical challenges: anchors are typically fixed after initialization, limiting adaptability to heterogeneous data; enforcing a shared anchor set across views suppresses view-specific diversity; and heuristic or self-weighted fusion strategies often lack explicit cross-view alignment, resulting in structural inconsistencies. To address these issues, we propose a Parameter-Free Multi-view Clustering framework with Adaptive Anchors for Large-scale Data (FPMCAA). Unlike existing approaches that decouple anchor construction and graph fusion, FPMCAA integrates adaptive anchor learning, anchor graph construction, and explicit cross-view alignment within a unified optimization model. Anchors are iteratively refined to capture complex distributions, while view-specific graphs are aligned toward a consensus structure without sacrificing inherent diversity. The framework avoids manual hyperparameter tuning and achieves linear computational complexity through efficient alternating optimization. Extensive experiments on benchmark datasets demonstrate that FPMCAA consistently outperforms state-of-the-art methods in clustering performance, robustness, and scalability. The source code of FPMCAA is available athttps://github.com/Xuchen2020/FPMCAA.

Active learning can effectively reduce the cost of labeling while enhancing model classification performance. However, prior studies have indicated that imbalanced class distributions adversely impact active learning, leading to diminished model effectiveness. Existing approaches to unbalanced active learning often neglect the multi-class imbalance problem and suffer from low performance and high time consumption. To address these issues, this paper introduces a hybrid active learning with online weighted broad learning system (HAL-OWBLS). Its main advantages include: (1) We optimize the initial labeled instance selection through an approximate query strategy to avoid the cold-start problem and introduce a sample selection strategy based on double uncertainty to enhance the rationality of active learning iterations. (2) A weighted broad learning system (WBLS) is chosen as the classifier, and an improved weighting strategy is adopted for multi-class imbalanced data. (3) We theoretically derive an efficient online updating model for WBLS, which reduces the time cost of active learning iterations by using only newly labeled samples for fast updating. The proposed HAL-OWBLS algorithm has better performance and robustness compared with existing related algorithms on various multi-class imbalanced data sets.

The challenges associated with large-scale user-item interaction graphs have attracted increasing attention in graph-based recommendation systems, primarily due to computational inefficiencies and inadequate information propagation. Existing methods provide partial solutions but suffer from notable limitations: model-centric approaches, such as sampling and aggregation, often struggle with generalization, while data-centric techniques, including graph sparsification and coarsening, lead to information loss and ineffective handling of bipartite graph structures. Recent advances in graph condensation offer a promising direction by reducing graph size while preserving essential information, presenting a novel approach to mitigating these challenges. Inspired by the principles of democracy, we proposeDemoRec, a framework that leverages graph condensation to generate user and item representatives for recommendation tasks. By constructing a compact interaction graph and clustering nodes with shared characteristics from the original graph, DemoRec significantly reduces graph size and computational complexity. Furthermore, it mitigates the over-reliance on high-order information, a critical challenge in large-scale bipartite graphs. Extensive experiments conducted on four public datasets demonstrate the effectiveness of DemoRec, showcasing substantial improvements in recommendation performance, computational efficiency, and robustness compared to SOTA methods.

The processing of continuous data streams in non-stationary environments has gained increasing attention. However, supervised online learning is often limited by label availability. Furthermore, it is crucial to develop a stable and high-performance online method in non-stationary environments. To tackle these issues, we propose a dynamic chunk-based active learning framework (DCAL). This framework includes a dynamic dual-stage query strategy and an enhanced active learning model. Specifically, the proposed query strategy, referred to as DyDQS, evaluates sample value comprehensively by considering local density, uncertainty, and dynamic imbalance ratio. This approach selects samples that are both representative and uncertain, while also enhancing the likelihood of selecting minority class samples. Additionally, we introduce an enhanced active learning model, named eBLS-W, which is based on the broad learning system (BLS). We redesign the update rule of BLS and equip it with a kernel mapping to improve its robustness and performance, enabling it to better handle non-stationary environments. The effectiveness of the DyDQS, eBLS-W, and DCAL was validated through experiments on synthetic datasets with drift and real-world datasets. The results demonstrate that our approach outperforms other advanced methods in terms of robustness and accuracy.

Active learning can significantly reduce the cost of labeling instances while improving model performance. However, similar to other traditional algorithms, active learning encounters the problem of class imbalance and delivers sub-optimal performance. Additionally, existing approaches suffer from poor performance and are time-consuming. To address these issues, we propose an Actively Incrementally Weighted Broad Learning System (AI-WBLS). Firstly, we introduce an active learning framework based on the weighted broad learning system, which employs a double uncertainty sample selection strategy to enhance the value and reasonableness of sample selection in each iteration of active learning. To further improve the model's adaptability during the iterative learning process, an adaptive weighting strategy is designed to adaptively modify the penalty weights according to the changes in the sample labels. Finally, an efficient incremental paradigm is developed to update the model with newly labelled samples instead of re-training, resulting in improved performance and efficiency. Extensive comparative experiments confirm that our approach outperforms other imbalanced active learning methods.

Multi-view clustering has exhibited exceptional performance by harnessing complementary information and extracting shared semantics across diverse views. However, existing methods still encounter substantial obstacles in exploiting high-quality consensus representation: (i) they typically lack effective guidance mechanisms under unsupervised conditions, leading to suboptimal representation learning, and (ii) they overlook the intrinsic cluster structure when forming sample pairs to align representation across views, conflicting with clustering objectives. To address these issues, we propose a contrastive-based method calledPseudo-LabelSimilarityGraph-driven Multi-ViewClustering (PSGVC). Specifically, to uncover the potential semantic relationships between samples, we construct a pseudo-label similarity graph based on the consensus representation. Then, we design a similarity graph-based contrastive loss, which utilizes the pseudo-label similarity graph as a global supervision signal to guide the optimization of cross-view embedded representation, thereby indirectly improving the quality of consensus representation. Additionally, we propose a weighted cluster-aware contrastive loss to align the consensus representation with view-specific representation. It leverages the cluster structure contained in the pseudo-label similarity graph to achieve cluster-level contrastive learning, further enhancing the quality of the consensus representation. Extensive experimental results show that our PSGVC achieves state-of-the-art clustering performance across multiple datasets.

The contactless interaction approach enables robots to identify object attributes via sensors and advanced perceptual technologies, eliminating the need for physical contact. This approach enhances interaction flexibility and safety while offering a richer, more intuitive human-robot experience. Current research focuses on cross-modal retrieval through non-contact feature extraction, enabling material attribute estimation and advancing object perception. However, existing cross-modal material retrieval methods overlook learning common attributes shared within the same material type. To address this, we propose Materials Perception with Self-paced Prototype contrastive Learning (MPSPL). First, attribute prototype contrastive learning extracts shared characteristics within the same material type, aggregates similar samples, and enhances the perception of deep model material attributes. Second, a self-paced learning strategy dynamically adjusts the contrastive loss temperature coefficient, guiding the model to discern discriminative features across materials and progressively recognize inherent attributes. Finally, extensive experiments validate our method under both known and unknown material category conditions. Experiments demonstrate the superiority of our proposed method over state-of-the-art approaches.

No-reference image quality assessment (NR-IQA) plays a pivotal role in computer vision by enabling image quality evaluation without reference images. While recent CNN and Transformer-based methods have advanced feature extraction, they face significant limitations. CNNs exhibit local feature bias, limiting their ability to capture global dependencies and complex structures critical to understanding diverse distortions. Transformers, despite modeling nonlocal dependencies through multihead attention, suffer from quadratic computational complexity with spatial dimensions, hindering efficient multiscale analysis. Moreover, their attention mechanisms frequently overlook critical interchannel dependencies, which are vital for capturing fine details in texture-rich images. Coupled with difficulties in handling high-noise environments and complex textures, this results in limited real-world accuracy and poor generalization across diverse datasets and unknown distortions. To bridge these gaps, we propose WAQNIQA, a novel wavelet-augmented quaternion network for NR-IQA. Distinct from conventional architectures, WAQNIQA integrates two synergistic modules: the wavelet-infused adaptive attention (WIAA) module, which leverages wavelet transforms (WTs) to achieve robust multiscale spatial-frequency analysis with linear complexity, and the quaternion collaborative feature enhancement (QCFE) module, which holistically models interchannel correlations to preserve fine texture details. Furthermore, we introduce PowerGridIQ, the first NR-IQA dataset specifically tailored for power grid scenarios. Extensive experiments demonstrate that WAQNIQA consistently surpasses state-of-the-art CNN and Transformer-based methods on PowerGridIQ and six public benchmarks. Notably, WAQNIQA exhibits superior cross-domain generalization, achieving competitive performance on the AGIQA-1K dataset for AI-generated content (AIGC) without explicit semantic alignment training, thereby validating its robustness against diverse and unknown distortions. Our code is available athttps://github.com/king-huoye/WAQNIQA

Due to its exceptional feature representation capabilities and high computational efficiency, the broad learning system (BLS) has been widely employed in various classification tasks. Nevertheless, BLS encounters considerable challenges in semi-supervised classification tasks involving complex heterogeneous data, given the data’s high-dimensional and noisy nature, coupled with a limited number of available labeled samples. To tackle these challenges, this article introduces a semi-supervised BLS based on distance constraint regularization (DRBLS) and a semi-supervised broad ensemble method (E-DRBLS) for high-dimensional data. Specifically, we present a distance constraint regularization (DR) that utilizes both labeled and unlabeled data to derive an optimal projection matrix, which maximizes the preservation of the original data’s intrinsic distribution structure. DR is designed to minimize intraclass distance, maximize interclass distance, and minimize the distance between neighboring samples. To boost the performance of BLS in semi-supervised classification, we integrate DR and BLS to construct the semi-supervised classifier DRBLS. Finally, we propose a mixed dimensionality reduction space generation (MDRSG) method that generates multiple high-quality and diverse mixed dimensionality reduction spaces (MDRSs). Based on MDRS, an ensemble framework, E-DRBLS, is developed for semi-supervised classification tasks targeting high-dimensional data. Comprehensive experiments confirm the superiority of the proposed methods.

Abstract Multi-view learning is an emerging field that aims to enhance learning performance by leveraging multiple views or sources of data across various domains. By integrating information from diverse perspectives, multi-view learning methods effectively enhance accuracy, robustness, and generalization capabilities. The existing research on multi-view learning can be broadly categorized into four groups in the survey based on the tasks it encompasses, namely multi-view classification approaches, multi-view semi-supervised classification approaches, multi-view clustering approaches, and multi-view semi-supervised clustering approaches. Despite its potential advantages, multi-view learning poses several challenges, including view inconsistency, view complementarity, optimal view fusion, the curse of dimensionality, scalability, limited labels, and generalization across domains. Nevertheless, these challenges have not discouraged researchers from exploring the potential of multiview learning. It continues to be an active and promising research area, capable of effectively addressing complex real-world problems.

Graph-based methods have demonstrated exceptional performance in semi-supervised classification. However, existing graph-based methods typically construct either a predefined graph in the original space or an adaptive graph within the output space, which often limits their ability to fully utilize prior information and capture the optimal intrinsic data distribution, particularly in high-dimensional data with abundant redundant and noisy features. This paper introduces a novel approach: Semi-Supervised Classification with Optimized Graph Construction (SSC-OGC). SSC-OGC leverages both predefined and adaptive graphs to explore intrinsic data distribution and effectively employ prior information. Additionally, a graph constraint regularization term (GCR) and a collaborative constraint regularization term (CCR) are incorporated to further enhance the quality of the adaptive graph structure and the learned subspace, respectively. To eliminate the negative effect of constructing a predefined graph in the original data space, we further propose a Hybrid Subspace Ensemble-enhanced framework based on the proposed Optimized Graph Construction method (HSE-OGC). Specifically, we construct multiple hybrid subspaces, which consist of meticulously chosen features from the original data to achieve high-quality and diverse space representations. Then, HSE-OGC constructs multiple predefined graphs within hybrid subspaces and trains multiple SSC-OGC classifiers to complement each other, significantly improving the overall performance. Experimental results conducted on various high-dimensional datasets demonstrate that HSE-OGC exhibits outstanding performance.

In the field of online learning, handling imbalanced data streams with concept drift has become a substantial challenge. Research interest in the combined challenges of concept drift and class imbalance is steadily growing. When dealing with concept drift, existing methods usually adopt active detection or passive adaptation to deal with concept drift. However, a single adaptation method is relatively limited in dealing with joint concept drift. This paper introduces a novel hybrid ensemble framework (HEF-IDS) specifically designed to tackle drifting data streams while effectively accommodating class imbalance. The framework integrates an active detection module and a passive adaptation module. In the passive adaptation module, we use a hybrid sampling strategy to balance data chunks, construct a sub-ensemble classifier for each balanced chunk, and combine it with a weighted voting scheme to deal with the class imbalance and adapt to the latest concept. In the process of active detection, we design a new sample selection mechanism that stores the most informative samples in the buffer, and uses these samples for training when drift is detected to prevent catastrophic forgetting. Comparative results on artificial and real-world datasets show that the proposed HEF-IDS is superior to other advanced online classification methods.

As data continues to grow exponentially, the importance of online learning across various domains has increased significantly. However, most existing studies assume that the true class label for each incoming data point is readily available, an assumption that is often impractical. To address this issue, this paper introduces a novel algorithm called Density-based Clustering and One-Class Ensemble Active Learning (DCOE-AL). This approach constructs an ensemble model by combining a clustering algorithm with the One-Class Broad Learning System (OCBLS), which represents clusters within the feature space. Furthermore, a new active learning mechanism is developed to enable DCOE-AL to effectively handle challenges associated with concept drift and label scarcity. The proposed method is evaluated on multiple synthetic data streams that exhibit diverse types of concept drift, as well as on several real-world data streams. Comparative evaluations demonstrate that DCOE-AL achieves superior performance while requiring significantly fewer labeled samples.

Time series anomaly detection is a pivotal task in data analysis, yet it poses the challenge of discerning normal and abnormal patterns in label-deficient scenarios. While prior studies have largely employed reconstruction-based approaches, which limit the models’ representational capacities. Moreover, existing deep learning-based methods are not sufficiently lightweight. Addressing these issues, we present PatchAD, our novel, highly efficient multiscale patch-based MLP-Mixer architecture that utilizes contrastive learning for representation extraction and anomaly detection. With its four distinct MLP Mixers and innovative dual project constraint module, PatchAD mitigates potential model degradation and offers a lightweight solution, requiring only0.403 Mparameters. Its efficacy is demonstrated by state-of-the-art results across8datasets sourced from different application scenarios, outperforming over30comparative algorithms. PatchAD significantly improves the classical F1 score by6.84%, the Aff-F1 score by4.27%, and the V-ROC by2.49%. Simultaneously, an in-depth analysis of the mechanisms underlying PatchAD has been conducted from both theoretical and experimental perspectives, validating the design motivations of the model.

Collaborative filtering-based recommendation systems often face challenges such as data sparsity and cold start problem. Knowledge graphs have been leveraged as auxiliary information to mitigate these issues. However, current knowledge graph-based recommendation systems still struggle to accurately capture the latent features of users and items in the embedding space. In this article, we introduce a heterogeneous propagation-based recommendation framework that effectively integrates collaborative patterns derived from user–item interactions with knowledge graph data. Specifically, we first design hierarchical layerwise attention mechanisms (LAMs) tailored to the different layers of the knowledge graph propagation process. These mechanisms assign appropriate attention scores to feature representations at various levels of complexity, thereby enhancing the integration. Additionally, the feature learning is further improved by adding Gaussian noise to the initial latent features of users and items, which are then refined using a denoising autoencoder. This method helps learn more robust and discriminative representations for users and items, enhancing recommendation performance. Extensive experiments were conducted on the MovieLens-1M, Book-Crossing, and Last.FM datasets demonstrate that the proposed algorithm significantly outperforms eighteen state-of-the-art baselines. Furthermore, ablation studies confirm the effectiveness of each module, emphasizing their individual contributions to the overall performance improvement.

Credit card (CC) fraud detection within the realm of financial security faces challenges such as data imbalance, large-scale anonymized transaction datasets, and the need for system-specific model training. Past methods often fail to address these aforementioned issues simultaneously. Utilizing a single model results in a lack of zero-shot capability without adaptation for real-world scenarios. This article introduces LLM40FD, a novel framework that leverages a large language model (LLM) to overcome these obstacles in anonymous zero-shot fraud detection. LLM40FD addresses the aforementioned challenges in CC fraud detection by employing a distribution-based one-class function and the walking embedding, without reliance on labeled data or fine-tuning in downstream. Additionally, LLM40FD enhances the model’s ability to detect fraudulent patterns and define robust decision boundaries. This is achieved through a dual-augmentation strategy and implicit contrastive learning, which generate enriched positive and negative samples. Our experiments demonstrate that LLM40FD not only achieves state-of-the-art (SOTA) performance in the full-shot setting but also exhibits strong zero-shot capability even with limited training data. Furthermore, we conduct additional experiments to validate the effectiveness and working mechanism of LLM40FD.

Multiview Subspace Clustering (MvSC) has demonstrated impressive clustering performance on multiview data. Most existing methods rely on either raw features or reduced-redundancy data for subspace representation learning, followed by spectral clustering to derive the final results. However, these methods maintain a fixed feature space during subspace learning, which limits information propagation and compromises both representation quality and clustering performance. To address this issue, this article proposes an adaptive dictionary learning approach for MvSC (AMvSC), which seamlessly integrates redundancy reduction and representation learning within a unified framework to facilitate mutual information propagation. Specifically, an adaptive dictionary learning strategy is designed to automatically reduce redundancy and noise in the original feature space during the subspace representation learning process. This strategy ensures effective information exchange, thereby enhancing the quality of the learned representations. Additionally, low-rank constraints, combined with smoothness and diversity regularization, are applied to further refine the subspace representations and comprehensively capture complex correlations among samples. Finally, an alternating optimization algorithm is developed to iteratively update the unified learning model. Extensive experiments validate the effectiveness and superiority of the proposed method.

Broad learning system (BLS) is renowned for its excellent generalization and high efficiency in data classification. However, when confronted with class imbalance problems, BLS treats all samples as equally important, resulting in performance degradation. In addition, the presence of noise and outliers in imbalanced data further complicates BLS's ability to handle real-world classification problems. To address these challenges, this article proposes a graph-embedding intuitionistic fuzzy adaptive broad learning system (GEIB). The graph embedding strategy proposed by GEIB leverages the geometric topology of the data and class-specific information, effectively capturing variability among imbalanced samples and improving class separability. Furthermore, we introduce intuitionistic fuzzy theory. The BLS integrated with it considers both the homogeneity and heterogeneity of sample neighborhoods, enabling it to address uncertainty and imprecision in the data. It further differentiates clean samples from noisy ones in imbalanced datasets, thereby enhancing model robustness. To further investigate the prior distribution information of imbalanced data, we design an adaptive class-specific penalty mechanism based on global distribution and local density information. This mechanism accounts for both intraclass and interclass density information and class global distribution. We verify the superiority of our method by conducting a comparison with current approaches using real-world datasets that include both Gaussian noise and noise-free versions.

Since cross-modal hashing requires minimal storage and computation, it is becoming increasingly popular with the exponential growth of multimedia content on the internet. However, the lack of accurate supervisory data has curtailed the effectiveness of unsupervised hashing techniques. Conversely, supervised hashing strategies necessitate considerable human and financial resources for data annotation. To address this limitation, we propose a novel semi-supervised cross-modal hashing method called Enhanced Cross-Modal Hashing via Hybrid Distillation and Structural Refinement (HDSR). Specifically, we first learn the features of inter-modal and inter-instance similarity relationships through pointwise semantic alignment and listwise similarity partial order learning, respectively, to extract refined structural representations from partially labeled data. Secondly, by fusing inter-modal similarity to construct higher-order affinity matrices, we precisely delineate the semantic correlation information across cross-modal data, facilitating stable self-supervised training of unlabeled data through the application of momentum fusion strategies. Finally, the refined structural representation of labeled data is transferred into unlabeled branches through hybrid distillation, enhancing the performance of cross-modal hash learning by generating compact and accurate hash codes. The proposed HDSR is compared with several state-of-the-art deep cross-modal hashing methods on three widely used benchmark databases, and the experimental results verify its efficiency and superiority.

In the swiftly advancing realm of information retrieval, unsupervised cross-modal hashing has emerged as a focal point of research, taking advantage of the inherent advantages of the multifaceted and dynamism inherent in multimedia data. Existing unsupervised cross-modal hashing methods rely mainly on initial pre-trained correlations among cross-modal features, and the inaccurate neighborhood correlations impacts the presentation of common semantics throughout the optimization. To address the aforementioned issues, we proposeEnsemblePrototypeNetworks (EPNet), which delineates class attributes of cross-modal instances through an ensemble clustering methodology. EPNet seeks to extract correlation information between instances by leveraging local correlation aggregation and ensemble clustering from multiple perspectives, aiming to reduce initialization effects and enhance cross-modal representations. Specifically, the local correlation aggregation is first proposed within a batch of semantic affinity relationships to generate a precise and compact hash code among cross-modal instances. Secondly, the ensemble prototype module is employed to discern the class attributes of deep features, thereby aiding the model in extracting more universally applicable feature representations. Thirdly, an early attempt to constrict the representational congruity of local semantic affinity relationships and deep feature ensemble prototype correlations using cross-task consistency loss aims to enhance the representation of cross-modal common semantic features. Finally, EPNet outperforms several state-of-the-art cross-modal retrieval methods on three real-world image-text datasets in extensive experiments.

Time-series anomaly detection has gained considerable prominence in numerous practical applications across various domains. Nonetheless, the scarcity of labels leads to the neglect of anomalous patterns in data, as well as the inherent complexities and variances in the definitions of temporal anomalies, pose significant challenges for insufficient recognition of anomaly patterns. In addition, real-time anomaly detection poses high demands on low computational cost and model robustness, presenting substantial obstacles for unsupervised time-series anomaly detection. In this article, we propose the data-driven spontaneous perturbation based on the sequence-image strategy and temporal anomaly knowledge enhancement strategy based on artificial anomalous data pairs to enhance the cognition of abnormal knowledge in unsupervised scenarios. On this basis, we propose the denoising autoencoder based on the broad learning system (DBLS-AE), which sufficiently learns the anomalous patterns, achieving efficient anomaly detection with low computational costs. To enhance the robustness in handling complex and diverse temporal anomalies, we further propose the progressive diversity denoising autoencoders based on the broad learning system (PddBLS-AE), which gradually prioritizes challenging samples and constructs a diverse ensemble of DBLS-AEs, markedly improving both performance and robustness. By innovatively utilizing the broad learning system (BLS), PddBLS-AE achieves accelerated training compared with advanced deep learning models. Comprehensive evaluations across multiple datasets robustly substantiate the efficacy of PddBLS-AE.

Graph-based multiview clustering (MVC) approaches have demonstrated impressive performance by leveraging the consistency properties of multiview data in an unsupervised manner. However, existing methods for graph learning heavily rely on either Euclidean structures or the manifold topological structures derived from fixed view-specific graphs. Unfortunately, these approaches may not accurately reflect the consensus topological structure in a multiview setting. To address this limitation and enhance the intrinsic graph learning process, an adaptive exploration of a more appropriate consistency topological structure is required. Toward this end, we propose a novel approach called collaborative topological graph learning (CTGL) for MVC. The key idea is to adaptively discover the consistent topological structure to guide intrinsic graph learning. We achieve this by introducing an auxiliary consistency graph that formulates the topological relevance learning function. However, estimating the auxiliary consistency graph is not straightforward, as it is based on the learned view-specific graphs and requires prior availability. To overcome this challenge, we develop a collaborative learning strategy that simultaneously learns both the auxiliary consistency graph and view-specific graphs using tensor learning techniques. This strategy enables the adaptive exploration of the consistency topological structure during graph learning, resulting in more accurate clustering outcomes. Extensive experiments are provided to show the effectiveness of the proposed method. The source code can be found at https://github.com/CLiu272/CTGL.

Time series anomaly detection is the process of identifying anomalies within time series data. The primary challenge of this task lies in the necessity for the model to comprehend the characteristics of time-independent and abnormal data patterns. In this study, a novel algorithm called adaptive memory broad learning system (AdaMemBLS) is proposed for time series anomaly detection. This algorithm leverages the rapid inference capabilities of the broad learning algorithm and the memory bank's capacity to differentiate between normal and abnormal data. Furthermore, an incremental algorithm based on multiple data augmentation techniques is introduced and applied to multiple ensemble learners, thereby enhancing the model's effectiveness in learning the characteristics of time series data. To bolster the model's anomaly detection capabilities, a more diverse ensemble approach and a discriminative anomaly score are recommended. Extensive experiments conducted on various real-world datasets demonstrate that the proposed method exhibits superior inference speed and more accurate anomaly detection compared to the existing competitors. A detailed experimental investigation is presented to elucidate the effectiveness of the proposed method and the underlying reasons for its efficacy.

Despite the prevalence of reconstruction-based deep learning methods, time-series anomaly detection (TSAD) remains a tremendous challenge. Existing approaches often struggle with limited temporal contexts, insufficient representation of normal patterns, and flawed evaluation metrics, all of which hinder their effectiveness in detecting anomalous behavior. To address these issues, we introduce a simple dissimilarity-based approach for time-series anomaly detection (SimAD). Specifically, SimAD first incorporates a patching-based feature extractor capable of processing extended temporal windows and employs the EmbedPatch encoder to fully integrate normal behavioral patterns. Second, we design an innovative ContrastFusion module in SimAD, which strengthens the robustness of anomaly detection by highlighting the distributional differences between normal and abnormal data. Third, we introduce two robust enhanced evaluation metrics, unbiased affiliation (UAff) and normalized affiliation (NAff), designed to overcome the limitations of existing metrics by providing better distinctiveness and semantic clarity. The reliability of these two metrics has been demonstrated by both theoretical and experimental analyses. Experiments conducted on seven diverse time-series datasets clearly demonstrate SimAD's superior performance compared with state-of-the-art (SOTA) methods, achieving relative improvements of 19.85% on ${F}1$ , 4.44% on Aff-F1, 77.79% on NAff-F1, and 9.69% on AUC on six multivariate datasets. Code and pretrained models are available at https://github.com/EmorZz1G/SimAD.

Clustering methods achieve performance improvement by jointly learning representation and cluster assignment. However, they do not consider the confidence of pseudo-labels which are not optimal as supervised information, resulting into error accumulation. To address this issue, we propose a Robust Pseudo-labeling for Semantic Clustering (RPSC) approach, which includes two stages. In the first stage (RPSC-Self), we design a semantic pseudo-labeling scheme by using the consistency of samples, i.e., samples with same semantics should be close to each other in the embedding space. To exploit robust semantic pseudo-labels for self-supervised learning, we propose a soft contrastive loss (SCL) which encourage the model to believe high-confidence sematic pseudo-labels and be less driven by low-confidence pseudo-labels. In the second stage (RPSC-Semi), we first determine the semantic pseudo-label of a sample based on the distance between itself and cluster centers, followed by screening out reliable semantic pseudo-label by exploiting the consistency. These reliable pseudo-labels are used as supervised information in the pseudo-semi-supervised learning algorithm to further improve the performance. Experimental results show that RPSC outperforms 18 competitive clustering algorithms significantly on six challenging image benchmarks. In particular, RPSC achieves an accuracy of 0.688 on ImageNet-Dogs, which is an up to 24% improvement, compared with the second-best method. Meanwhile, we conduct ablation studies to investigate effects of different augmented strategies on RPSC as well as contributions of terms in SCL to clustering performance. Besides, experimental results indicate that SCL can be easily integrated into existing clustering methods and bring performance improvement.

In the current era of big data, securing computer networks and preventing cyberattacks have become a major challenge. In this regard, we present a novel approach for addressing the imbalanced problem of network traffic data. Specifically, the proposed method MC-OCBLS, leverages the maximum correntropy criterion (MCC) to develop a one-class broad learning system. The robustness of the one-class classification model is enhanced by maximizing the correlation entropy. Moreover, considering the limited generalization capability of individual models, we propose the boosting ensemble model SMC-OCBLS, which assigns higher weights to misclassified samples and classifiers with lower error rates. To increase the variety of classifiers in the ensemble framework, feature columns are randomly disrupted during training to construct different feature spaces. Training multiple hybrid classifiers using various feature spaces further enhances the accuracy and generalizability of the model. The experimental results demonstrate the superiority of the proposed model over other advanced approaches.

Feature mining is a challenging work in the field of multivariate time series (MTS) data mining. Traditional methods suffer from three major issues. 1) Learned shapelets may seriously diverge from original subsequences since learning methods do not restrain the learned ones similar to raw sequences, which reduces interpretability. 2) Existing rule mining methods just generate association rules based on feature combination of different variables without considering temporal relations among features, which could not adequately express the essential characteristics of MTS data. 3) Most deep learning methods only mine global and high-level features of MTS data, which affects interpretability. To address these issues, we propose a temporal association rule mining method based on Generative Adversarial Network (GAN) called TAR-GAN. First, a shapelet mining method based on GAN (SGAN) is advanced to discover dataset-level and sample-level shapelets of all variables in MTS data. Second, a Temporal Graph based Rule Mining method (TGRM) is introduced to discover temporal association rules based on the temporal relationships among shapelets of different variables. Meanwhile, a Fast Convolution-based Similarity Measure methods(FCSM) is introduced to measure the similarity between MTS samples and temporal association rules. Furthermore, an adversarial training strategy is introduced to ensure the effectiveness and stability of generated temporal association rules, which could reflect the essential characteristics of MTS data. Extensive experiments on 12 datasets show the effectiveness and efficiency of our method.

Recently, rule-based classification on multivariate time series (MTS) data has gained lots of attention, which could improve the interpretability of classification. However, state-of-the-art approaches suffer from three major issues. 1) few existing studies consider temporal relations among features in a rule, which could not adequately express the essential characteristics of MTS data. 2) due to the concept drift and time warping of MTS data, traditional methods could not mine essential characteristics of MTS data. 3) existing online learning algorithms could not effectively update shapelet-based temporal association rules of MTS data due to its temporal relationships among features of different variables. To handle these issues, we propose an online learning method for temporal association rule on dynamically collected MTS data (OTARL). First, a new type of rule named temporal association rule is defined and mined to represent temporal relationships among features in a rule. Second, an online learning mechanism with a probability correlation-based evaluation criterion is proposed to realize the online learning of temporal association rules on dynamically collected MTS data. Finally, an ensemble classification approach based on maximum-likelihood estimation is advanced to further enhance the classification performance. We conduct experiments on ten real-world datasets to verify the effectiveness and efficiency of our approach.

Feature selection is a highly regarded research area in the field of data mining, as it significantly enhances the efficiency and performance of high-dimensional data analysis by eliminating redundant and irrelevant features. Despite the ease of data acquisition, labeling data remains a laborious and expensive task. To leverage the abundance of unlabeled data, researchers have proposed various feature selection methods that operate with limited labels, including semi-supervised feature selection and unsupervised feature selection. However, a comprehensive review encompassing a thorough overview of feature selection algorithms with limited labels is lacking. To bridge this gap, this paper conducts a comprehensive exploration of feature selection methods specifically tailored to limited-label scenarios. These methods are systematically classified into two primary categories: semi-supervised and unsupervised feature selection. Additionally, by introducing a novel taxonomy and discussing future challenges, this survey aims to provide researchers with a comprehensive and in-depth understanding of feature selection in limited-label scenarios. Moreover, it aims to offer valuable insights that can guide further research and development in this domain.

Incomplete multi-view clustering (IMVC) presents a significant challenge due to the need for effectively exploring complementary and consistent information within the context of missing views. One promising strategy to tackle this challenge is to recover missing views by inferring the missing samples. However, such approaches often fail to fully utilize discriminative structural information or adequately address consistency, as it requires such information to be known or learnable in advance, which contradicts the incomplete data setting. In this study, we propose a novel approach calledLatentStructure-Aware view recovery (LaSA) for the IMVC task. Our objective is to recover missing views through discriminative latent representations by leveraging structural information. Specifically, our method offers a unified closed-form formulation that simultaneously performs missing data inference and latent representation learning, using a learned intrinsic graph as structural information. This formulation, incorporating graph structure information, enhances the inference of missing data while facilitating discriminative feature learning. Even when intrinsic graph is initially unknown due to incomplete data, our formulation allows for effective view recovery and intrinsic graph learning through an iterative optimization process. To further enhance performance, we introduce an iterative consistency diffusion process, which effectively leverages the consistency and complementary information across multiple views. Extensive experiments demonstrate the effectiveness of the proposed method compared to state-of-the-art approaches.

Imbalanced data biases the classifier towards the majority class. Accompanied with high-dimensional characteristics, classification performance is further degraded. Existing researches for skewed data mainly involve resampling, cost-sensitive learning, and classifier ensemble. However, these approaches have some limitations: 1) resampling suffers from noisy and redundant features in high-dimensional skewed data; 2) cost-sensitive learning is hard to construct an optimal cost matrix for sample misclassification; 3) ensemble with random feature subspace easily leads to information loss; 4) ensemble with sample subspace on small-size data easily leads to insufficient description of sample space and suffers from negative impacts of high-dimensional data. This paper proposes an improved contraction-expansion subspace ensemble (ICESE) for high-dimensional imbalanced data classification. First, a contraction-expansion subspace optimization (CESO) is designed to perform subspace selection and transformation, which is beneficial for enhancing the discrimination and diversity of subspace. Then, to strengthen classification capabilities, a CESO-based multilayer optimization structure is developed to construct the improved subspace. Finally, to mitigate the effects of skewed data, ICESE performs a resampling scheme on the improved subspace for constructing a rebalanced subset to base classifier. Experimental results on 24 high-dimensional imbalanced data sets demonstrate that our ICESE outperforms different mainstream ensemble systems in terms of F-score and G-mean.

Imbalanced data poses a substantial challenge to conventional classification methods, which often disproportionately favor samples from the majority class. To mitigate this issue, various oversampling techniques have been deployed, but opportunities for optimizing data distributions remain underexplored. By exploiting the ability of metric learning to refine the sample distribution, we propose a novel approach, Imbalance Large Margin Nearest Neighbor (ILMNN). Initially, ILMNN is applied to establish a latent feature space, pulling intra-class samples closer and distancing inter-class samples, thereby amplifying the efficacy of oversampling techniques. Subsequently, we allocate varying weights to samples contingent upon their local distribution and relative class frequency, thereby equalizing contributions from minority and majority class samples. Lastly, we employ Kullback-Leibler (KL) divergence as a safeguard to maintain distributional similarity to the original dataset, mitigating severe intra-class imbalances. Comparative experiments on various class-imbalanced datasets verify that our ILMNN approach yields superior results.

Broad learning system (BLS) is a simple yet efficient learning algorithm that only needs to train a three-layer feedforward neural network. Although various BLS variants have been designed for supervised learning, none have been used for unsupervised learning. This paper proposes BLS-AE, a novel data clustering scheme that seamlessly combines BLS and auto-encoder. Then, graph regularization is introduced into BLS-AE to increase the capability of learning intrinsic structures in data and adaptation to various data simultaneously, which is termed BLSg-AE. Moreover, different concatenation styles of feature and enhancement nodes are investigated for reusing the learned features, followed by designing two special strategies (i.e., pruning optimization and incremental learning) to reduce the parameter scale significantly and improve performance, which is termed xBLSg-AE. To address the performance instability issue caused by random subspace in a single xBLSg-AE, the x-cascade broad learning system graph regularization multi-auto-encoder (xBLSg-MAE) algorithm is proposed. Extensive experiments are conducted on multiple real data sets to demonstrate that the proposed methods are more effective and robust than competing approaches.

Due to the advantages of fast training speed and competitive performance, Broad Learning System (BLS) has been widely used for classification tasks across various domains. However, the random weight generation mechanism in BLS makes the model unstable, and the performance of BLS may be limited when dealing with some complex datasets. On the other hand, the instability of BLS brings diversity to ensemble learning, and ensemble methods can also reduce the variance and bias of the single BLS. Therefore, we propose an ensemble learning algorithm based on BLS, which includes three modules. To improve the stability and generalization ability of BLS, we utilize BLS as the base classifier in an AdaBoost framework first. Taking advantage of the incremental learning mechanism of BLS, we then propose a selective ensemble method to raise the accuracy and diversity of the BLS ensemble method. In addition, based on the former selective Adaboost framework, we suggest a hierarchical ensemble algorithm, which combines sample and feature dimensions to further improve the fitting ability of the ensemble BLS. Extensive experiments have demonstrated that the proposed method performs better than the original BLS and other state-of-the-art models, proving the effectiveness and versatility of our proposed approaches.

In this work, we study a more realistic challenging scenario in multiview clustering (MVC), referred to as incomplete MVC (IMVC) where some instances in certain views are missing. The key to IMVC is how to adequately exploit complementary and consistency information under the incompleteness of data. However, most existing methods address the incompleteness problem at the instance level and they require sufficient information to perform data recovery. In this work, we develop a new approach to facilitate IMVC based on the graph propagation perspective. Specifically, a partial graph is used to describe the similarity of samples for incomplete views, such that the issue of missing instances can be translated into the missing entries of the partial graph. In this way, a common graph can be adaptively learned to self-guide the propagation process by exploiting the consistency information, and the propagated graph of each view is in turn used to refine the common self-guided graph in an iterative manner. Thus, the associated missing entries can be inferred through graph propagation by exploiting the consistency information across all views. On the other hand, existing approaches focus on the consistency structure only, and the complementary information has not been sufficiently exploited due to the data incompleteness issue. By contrast, under the proposed graph propagation framework, an exclusive regularization term can be naturally adopted to exploit the complementary information in our method. Extensive experiments demonstrate the effectiveness of the proposed method in comparison with state-of-the-art methods. The source code of our method is available at the https://github.com/CLiu272/TNNLS-PGP.

Consensus clustering can derive a more promising and robust clustering result by integrating multiple partitions strategically. However, there are several limitations in the existing approaches: 1) most of the methods compute the ensemble-information matrix heuristically and lack of sufficient optimization; 2) the information from the original dataset is rarely considered; and 3) the noise in both label space and feature space is ignored. To address these issues, we proposed a novel consensus clustering method with co-association matrix optimization (CC-CMO), which aims at improving the co-association matrix by taking abundant information from both label space and feature space into consideration. In label space, CC-CMO derives a weighted partition matrix capturing the intercluster correlation and further designs a least squares regression (LSR) model to explore the global structure of data. In feature space, CC-CMO minimizes the reconstruction error with doubly stochastic normalization in the projective subspace to eliminate noise features as well as learn the local affinity of data. To improve the co-association matrix by jointly considering the subspace representation, global structure, and local affinity of data, we explicitly propose a unified optimization framework and design an alternating optimization algorithm for the optimal co-association matrix. Extensive experiments on a variety of real-world datasets demonstrate the superior performance of CC-CMO to the state-of-the-art consensus clustering approaches.

High-dimensional class imbalanced data have plagued the performance of classification algorithms seriously. Because of a large number of redundant/invalid features and the class imbalanced issue, it is difficult to construct an optimal classifier for high-dimensional imbalanced data. Classifier ensemble has attracted intensive attention since it can achieve better performance than an individual classifier. In this work, we propose a multiview optimization (MVO) to learn more effective and robust features from high-dimensional imbalanced data, based on which an accurate and robust ensemble system is designed. Specifically, an optimized subview generation (OSG) in MVO is first proposed to generate multiple optimized subviews from different scenarios, which can strengthen the classification ability of features and increase the diversity of ensemble members simultaneously. Second, a new evaluation criterion that considers the distribution of data in each optimized subview is developed based on which a selective ensemble of optimized subviews (SEOS) is designed to perform the subview selective ensemble. Finally, an oversampling approach is executed on the optimized view to obtain a new class rebalanced subset for the classifier. Experimental results on 25 high-dimensional class imbalanced datasets indicate that the proposed method outperforms other mainstream classifier ensemble methods.

Great progress has been made in StyleGAN-based image editing. To associate with preset attributes, most existing approaches focus on supervised learning for semantically meaningful latent space traversal directions, and each manipulation step is typically determined for an individual attribute. To address this limitation, we propose a Text-guided Unsupervised StyleGAN Latent Transformation (TUSLT) model, which adaptively infers a single transformation step in the latent space of StyleGAN to simultaneously manipulate multiple attributes on a given input image. Specifically, we adopt a two-stage architecture for a latent mapping network to break down the transformation process into two manageable steps. Our network first learns a diverse set of semantic directions tailored to an input image, and later nonlinearly fuses the ones associated with the target attributes to infer a residual vector. The resulting tightly interlinked two-stage architecture delivers the flexibility to handle diverse attribute combinations. By leveraging the cross-modal text-image representation of CLIP, we can perform pseudo annotations based on the semantic similarity between preset attribute text descriptions and training images, and further jointly train an auxiliary attribute classifier with the latent mapping network to provide semantic guidance. We perform extensive experiments to demonstrate that the adopted strategies contribute to the superior performance of TUSLT.

Face retouching aims to remove facial blemishes, while at the same time maintaining the textual details of a given input image. The main challenge lies in distinguishing blemishes from the facial characteristics, such as moles. Training an image-to-image translation network with pixel-wise supervision suffers from the problem of expensive paired training data, since professional retouching needs specialized experience and is time-consuming. In this paper, we propose a Blemish-aware and Progressive Face Retouching model, which is referred to as BPFRe. Our framework can be partitioned into two manageable stages to perform progressive blemish removal. Specifically, an encoder-decoder-based module learns to coarsely remove the blemishes at the first stage, and the resulting intermediate features are injected into a generator to enrich local detail at the second stage. We find that explicitly suppressing the blemishes can contribute to an effective collaboration among the components. Toward this end, we incorporate an attention module, which learns to infer a blemish-aware map and further determine the corresponding weights, which are then used to refine the intermediate features transferred from the encoder to the decoder, and from the decoder to the generator. Therefore, BPFRe is able to deliver significant performance gains on a wide range of face retouching tasks. It is worth noting that we reduce the dependence of BPFRe on paired training samples by imposing effective regularization on unpaired ones.

Consensus clustering methods provide better performance by fusing multiple clustering solutions in terms of accuracy, robustness and stability. However, most current methods suffer from different challenges: i) the high-dimensional problem; ii) limitations of single clustering method; iii) the optimal number of clusters selecting for a certain validity measure; iv) redundant clustering candidate attributes. To overcome the above limitations, we propose a hybrid clustering solutions fusion method based on gated three-way decision (HCFG) for data analysis. By integrating multiple clustering solutions and executing information fusion, HCFG enjoys four properties: (1) multiple random subspace generation strategy is utilized to generate diverse low-dimensional subspaces effectively; (2) a fusion framework that considers characteristics of both the soft clustering and hard clustering methods is designed, in which potential boundary of feature attribute sets is explored; (3) the optimal number of clusters is set by utilizing multiple clustering validity indices; (4) clustering solutions is considered as attributes and a gated three-way decision method is proposed to adaptively conduct attribute reductions. Extensive comparative experiments on 24 real-world data sets demonstrates the effectiveness and superiority of HCFG. Moreover, nonparametric tests are conducted to compare HCFG with multiple consensus clustering methods.

With the vigorous development of Industry 4.0, industrial Big Data has turned into the core element of the Industrial Internet of Things. As one of the most fundamental and indispensable components in industrial cyber-physical systems (CPS), intelligent anomaly detection is still an essential and challenging issue. However, with the development of the network, there may exist unknown types of attacks, which are difficult to collect. Facing one-class industrial intrusion detection scenario that the collected training data only includes normal state, the one-class broad learning system (OCBLS) and the stacked OCBLS (ST-OCBLS) algorithms are developed. Benefiting from the characteristics of BLS, our proposed approaches retain the advantage of efficient training process. Moreover, the high-level hidden features of the network traffic data can be learned through the progressive encoding and decoding mechanism in ST-OCBLS. Extensive comparative experiments on several real-world intrusion detection tasks are carried out to demonstrate that our proposed methods have competitive performance and high efficiency in the face of complex network data and diversified types of intrusions. Overall, this article provides a new alternative solution for network intrusion detection in Industry 4.0.

Cluster ensemble improves the robustness and stability of clustering performances by utilizing multiple solutions. Although traditional cluster ensemble methods have achieved promising performances, they are not adaptive enough to cope with data sets that have multiple levels of complexities. Besides, these methods may contain noisy and redundancy members which have negative effects. To mitigate the above issues, in this paper, we propose a multi-objective filter refinement scheme (MOFRS). First, we perform various clustering methods on different representations of data to generate diverse solutions. Second, we propose a solution filter to select a proper method and reduce the number of initial partitions for a given data set. Third, four stability indices are designed to split instances into stable and unstable groups. Fourth, objective functions based on diversity and quality are utilized to quantify the goodness of base clustering solutions. Finally, we design an improvement oriented multi-objective evolutionary algorithm to optimize these objective functions. Extensive experimental results conducted on 27 real-world data sets show that MOFRS outperforms most cluster ensemble selection methods, and achieves statistically significant improvements, compared with full ensemble methods.

Ensemble clustering has an advantage in producing a more promising and robust clustering result by combining multiple partitions strategically. The quality of both base partitions and co-association matrix plays an essential role in improving the consensus partition. However, the current ensemble clustering methods have several limitations: 1) The noise in high-dimensional feature space is ignored; 2) The independent base partition generation process does not pay attention to ambiguous samples; 3) The co-association matrix and the weights of base partitions commonly lack of theoretical optimization. In order to address these issues, we propose an adaptive ensemble clustering framework with boosting BLS-based autoencoder (BoostAEC). In the generation step, a boosting BLS-based autoencoder (BoostBLSAE) is designed to generate base partitions sequentially, which learns compressed feature subspaces for ambiguous samples and adaptively evaluates the corresponding weights of reliability. In the integration step, we construct a fuzzy membership function to capture the inter-cluster correlation and explicitly propose a consensus objective function to optimize the unified co-association matrix by considering the weighted base partitions. Extensive experiments on various real-world datasets demonstrate the superior performance of BoostAEC to the state-of-the-art ensemble clustering methods.

High-dimensional small-size data seriously affects the performance of classifiers. By combining classifiers, ensemble learning obtains higher accuracy and more robust predictions. However, these classifier ensemble methods suffer from several limitations: 1) ensemble with sample space suffers from noise and redundant features; 2) constructing sample subspace on small-size data leads to an insufficient description of sample space; 3) ensemble with feature space leads to information loss, which will degrade performance of classifiers. To overcome the above limitations, a new classifier ensemble method based on subspace enhancement (CESE) is proposed for high-dimensional data classification. First, a superior subspace enhancement scheme (SSE) is designed to effectively implement feature selection and transformation for high-dimensional data, followed by generating multiple superior feature subspaces with diversity and discrimination, which enhances the representative ability of features. Second, we develop a mixed space enhancement process (MSE) based on multiscale rotation reconstruction and various subspace enhanced features of SSE. Furthermore, to improve the capacity of our method, we design various feature combination strategies for enhanced features from both SSE and MSE. Comparative results on 33 high-dimensional data sets indicate that our approach CESE outperforms different mainstream integrated system

With effective performance and fast training speed, broad learning system (BLS) has been widely developed in recent years, which provides a new way for network training. However, the randomly generated feature nodes and enhancement nodes in the BLS network may have redundant and inefficient features, which will affect the subsequent classification performance. In response to the above issues, we propose a series of self-encoding networks based on BLS from the perspective of unsupervised feature extraction. These include the single hidden layer autoencoder built on the basis of BLS(BLS-AE), the stacked BLS-based autoencoder (ST-BLS), the sparse BLS-based autoencoder (SP-BLS), and the stacked sparse BLS-based autoencoder(SS-BLS). The proposed BLS-based self-encoding networks retain the advantage of efficient BLS model training, and overcome the time-consuming defect of iterative parameter optimization in traditional self-encoding networks. In addition, the higher-level abstract features of the input data can be learned through the progressive encoding and decoding process. Combining$L_1$regularization to train the parameters can further enhance the robustness of the extracted features. Extensive comparative experiments on real-world data sets demonstrate the superiority of the proposed methods in terms of both effectiveness and efficiency.

It is hard to construct an optimal classifier for high-dimensional imbalanced data, on which the performance of classifiers is seriously affected and becomes poor. Although many approaches, such as resampling, cost-sensitive, and ensemble learning methods, have been proposed to deal with the skewed data, they are constrained by high-dimensional data with noise and redundancy. In this study, we propose an adaptive subspace optimization ensemble method (ASOEM) for high-dimensional imbalanced data classification to overcome the above limitations. To construct accurate and diverse base classifiers, a novel adaptive subspace optimization (ASO) method based on adaptive subspace generation (ASG) process and rotated subspace optimization (RSO) process is designed to generate multiple robust and discriminative subspaces. Then a resampling scheme is applied on the optimized subspace to build a class-balanced data for each base classifier. To verify the effectiveness, our ASOEM is implemented based on different resampling strategies on 24 real-world high-dimensional imbalanced datasets. Experimental results demonstrate that our proposed methods outperform other mainstream imbalance learning approaches and classifier ensemble methods.

Generative Adversarial Network (GAN)-based models have made remarkable progress in high-fidelity image synthesis. However, the performance of class-conditional image synthesis may significantly deteriorate for the case where a limited number of labeled training samples are available. To reduce the dependence on labeled data, we propose a semi-supervised GAN with Extended Distribution Matching, and our model is referred to as EDM-GAN. To prevent a class-conditional discriminator from overfitting the limited labeled data, we perform a transformation of random regional replacement on both real and synthesized samples. By matching the extended distributions, the discriminator is encouraged to focus more on the spatial regions that contain certain objects, while at the same time a class-conditional generator is induced to capture precise class semantics. The adversarial training process can be effectively stabilized and converges to a better solution. Our experimental results on multiple standard benchmarks demonstrate consistent performance gains in synthesis quality and class-semantic accuracy.

Identifying cancer subtypes by integration of multi-omic data is beneficial to improve the understanding of disease progression, and provides more precise treatment for patients. Cancer subtypes identification is usually accomplished by clustering patients with unsupervised learning approaches. Thus, most existing integrative cancer subtyping methods are performed in an entirely unsupervised way. An integrative cancer subtyping approach can be improved to discover clinically more relevant cancer subtypes when considering the clinical survival response variables. In this study, we propose a Survival Supervised Graph Clustering (S2GC)for cancer subtyping by taking into consideration survival information. Specifically, we use a graph to represent similarity of patients, and develop a multi-omic survival analysis embedding with patient-to-patient similarity graph learning for cancer subtype identification. The multi-view (omic)survival analysis model and graph of patients are jointly learned in a unified way. The learned optimal graph can be unitized to cluster cancer subtypes directly. In the proposed model, the survival analysis model and adaptive graph learning could positively reinforce each other. Consequently, the survival time can be considered as supervised information to improve the quality of the similarity graph and explore clinically more relevant subgroups of patients. Experiments on several representative multi-omic cancer datasets demonstrate that the proposed method achieves better results than a number of state-of-the-art methods. The results also suggest that our method is able to identify biologically meaningful subgroups for different cancer types. (Our Matlab source code is available online at github: https://github.com/CLiu272/S2GC).

As an effective method for clustering applications, the clustering ensemble algorithm integrates different clustering solutions into a final one, thus improving the clustering efficiency. The key to designing the clustering ensemble algorithm is to improve the diversities of base learners and optimize the ensemble strategies. To address these problems, we propose a clustering ensemble framework that consists of three parts. First, three view transformation methods, including random principal component analysis, random nearest neighbor, and modified fuzzy extension model, are used as base learners to learn different clustering views. A random transformation and hybrid multiview learning-based clustering ensemble method (RTHMC) is then designed to synthesize the multiview clustering results. Second, a new random subspace transformation is integrated into RTHMC to enhance its performance. Finally, a view-based self-evolutionary strategy is developed to further improve the proposed method by optimizing random subspace sets. Experiments and comparisons demonstrate the effectiveness and superiority of the proposed method for clustering different kinds of data.

Despite the tremendous success in computer vision, deep convolutional networks suffer from serious computation costs and redundancies. Although previous works address that by enhancing the diversities of filters, they have not considered the complementarity and the completeness of the internal convolutional structure. To respond to this problem, we propose a novel inner-imaging (InI) architecture, which allows relationships between channels to meet the above requirement. Specifically, we organize the channel signal points in groups using convolutional kernels to model both the intragroup and intergroup relationships simultaneously. A convolutional filter is a powerful tool for modeling spatial relations and organizing grouped signals, so the proposed methods map the channel signals onto a pseudoimage, like putting a lens into the internal convolution structure. Consequently, not only is the diversity of channels increased but also the complementarity and completeness can be explicitly enhanced. The proposed architecture is lightweight and easy to be implement. It provides an efficient self-organization strategy for convolutional networks to improve their performance. Extensive experiments are conducted on multiple benchmark datasets, including CIFAR, SVHN, and ImageNet. Experimental results verify the effectiveness of the InI mechanism with the most popular convolutional networks as the backbones.

Text classification has been widely explored in natural language processing. In this article, we propose a novel adaptive dense ensemble model (AdaDEM) for text classification, which includes local ensemble stage (LES) and global dense ensemble stage (GDES). To strengthen the classification ability and robustness of the enhanced layer, we propose a selective ensemble model based on enhanced attention convolutional neural networks (EnCNNs). To increase the diversity of the ensemble system, these EnCNNs are generated by using two manners: 1) different sample subsets and 2) different granularity kernels. Then, an evaluation criterion that considers both accuracy and diversity is proposed in LES to obtain effective integration results. Furthermore, to make better use of information flow, we develop an adaptive dense ensemble structure with multiple enhanced layers in GDES to mitigate the issue that there may be redundant or invalid enhanced layers in the cascade structure. We conducted extensive experiments against state-of-the-art methods on multiple real-world datasets, including long and short texts, which has verified the effectiveness and generality of our method.

The broad learning system (BLS) is an algorithm that facilitates feature representation learning and data classification. Although weights of BLS are obtained by analytical computation, which brings better generalization and higher efficiency, BLS suffers from two drawbacks: 1) the performance depends on the number of hidden nodes, which requires manual tuning, and 2) double random mappings bring about the uncertainty, which leads to poor resistance to noise data, as well as unpredictable effects on performance. To address these issues, a kernel-based BLS (KBLS) method is proposed by projecting feature nodes obtained from the first random mapping into kernel space. This manipulation reduces the uncertainty, which contributes to performance improvements with the fixed number of hidden nodes, and indicates that manually tuning is no longer needed. Moreover, to further improve the stability and noise resistance of KBLS, a progressive ensemble framework is proposed, in which the residual of the previous base classifiers is used to train the following base classifier. We conduct comparative experiments against the existing state-of-the-art hierarchical learning methods on multiple noisy real-world datasets. The experimental results indicate our approaches achieve the best or at least comparable performance in terms of accuracy.

Ensemble learning has many successful applications because of its effectiveness in boosting the predictive performance of classification models. In this article, we propose a semisupervised multiple choice learning (SemiMCL) approach to jointly train a network ensemble on partially labeled data. Our model mainly focuses on improving a labeled data assignment among the constituent networks and exploiting unlabeled data to capture domain-specific information, such that semisupervised classification can be effectively facilitated. Different from conventional multiple choice learning models, the constituent networks learn multiple tasks in the training process. Specifically, an auxiliary reconstruction task is included to learn domain-specific representation. For the purpose of performing implicit labeling on reliable unlabeled samples, we adopt a negative$\ell _{1}$-norm regularization when minimizing the conditional entropy with respect to the posterior probability distribution. Extensive experiments on multiple real-world datasets are conducted to verify the effectiveness and superiority of the proposed SemiMCL model.

Broad learning system (BLS) is a novel and efficient model, which facilitates representation learning and classification by concatenating feature nodes and enhancement nodes. In spite of the efficient properties, BLS is still suboptimal when facing with imbalance problem. Besides, outliers and noises in imbalanced data remain a challenge for BLS. To address the above issues, in this paper we first propose a weighted BLS, which assigns a weight to each training sample, and adopt a general weighting scheme, which augments the weight of samples from the minority class. To further explore the prior distribution of original data, we design a density based weight generation mechanism to guide the specific weight matrix generation and propose the adaptive weighted broad learning system (AWBLS). This mechanism considers the inter-class and intra-class distance simultaneously in the density calculation. Finally, we propose the incremental weighted ensemble broad learning system (IWEB) by utilizing a progressive mechanism to further improve the stability and robustness of AWBLS. Extensive comparative experiments on 38 real-world data sets verfy that IWEB outperforms most of the imbalance ensemble classification methods.

In this paper, we propose two GAN-based enhanced deep subspace clustering approaches: deep subspace clustering via dual adversarial generative networks (DSC-DAG) and self-supervised deep subspace clustering with adversarial generative networks ($S^2 DSC-AG$). In DSC-DAG, the distributions of both the inputs and corresponding latent representations are learning via adversarial training simultaneously. Besides, there are two kinds of synthetical representations to facilitate the fine-tuning of the encoder module: the combinations of latent representations with certain random combination coefficients and the representations of real-like inputs derived from noise variables. In$S^DSC-AG$, a self-supervised information learning module substitutes for adversarial learning in the latent space, since both of them play the same role in learning discriminative latent representations. We analyze the connections between these methods and demonstrate their equivalences. We conduct extensive experiments on multiple real-world data sets against state-of-the-art subspace clustering methods in terms of accuracy, normalized mutual information and purity. Experimental results demonstrate the effectiveness and superiority of our proposed methods.

Recently, multitask learning has been successfully applied to survival analysis problems. A critical challenge in real-world survival analysis tasks is that not all instances and tasks are equally learnable. A survival analysis model can be improved when considering the complexities of instances and tasks during the model training. To this end, we propose an asymmetric graph-guided multitask learning approach with self-paced learning for survival analysis applications. The proposed model is able to improve the learning performance by identifying the complex structure among tasks and considering the complexities of training instances and tasks during the model training. Especially, by incorporating the self-paced learning strategy and asymmetric graph-guided regularization, the proposed model is able to learn the model in a progressive way from "easy" to "hard" loss function items. In addition, together with the self-paced learning function, the asymmetric graph-guided regularization allows the related knowledge transfer from one task to another in an asymmetric way. Consequently, the knowledge acquired from those earlier learned tasks can help to solve complex tasks effectively. The experimental results on both synthetic and real-world TCGA data suggest that the proposed method is indeed useful for improving survival analysis and achieves higher prediction accuracies than the previous state-of-the-art methods.

Graph-based methods have achieved impressive performance on semisupervised classification (SSC). Traditional graph-based methods have two main drawbacks. First, the graph is predefined before training a classifier, which does not leverage the interactions between the classifier training and similarity matrix learning. Second, when handling high-dimensional data with noisy or redundant features, the graph constructed in the original input space is actually unsuitable and may lead to poor performance. In this article, we propose an SSC method with novel graph construction (SSC-NGC), in which the similarity matrix is optimized in both label space and an additional subspace to get a better and more robust result than in original data space. Furthermore, to obtain a high-quality subspace, we learn the projection matrix of the additional subspace by preserving the local and global structure of the data. Finally, we intergrade the classifier training, the graph construction, and the subspace learning into a unified framework. With this framework, the classifier parameters, similarity matrix, and projection matrix of subspace are adaptively learned in an iterative scheme to obtain an optimal joint result. We conduct extensive comparative experiments against state-of-the-art methods over multiple real-world data sets. Experimental results demonstrate the superiority of the proposed method over other state-of-the-art algorithms.

The class imbalance problem has posed a leading challenge in real-world applications. Traditional methods focus on either the data level or algorithm level to solve the binary classification problem on imbalanced data, and seldom consider searching an effective transformation for classification. Besides, the undersampling process adopted in them is always subjective and unilateral. To address the above issues, we first propose a hybrid classifier ensemble (HCE) framework to conduct binary imbalanced data classification, which mainly includes a metric-based data space transformation (MDST) and an adaptive two-stage undersampling process (ATUP). The MDST aims to find a more appropriate embedding space for original imbalance data sets, and the ATUP considers both informative and representative samples to generate balanced data sets. Furthermore, we design a progressive HCE (PHCE) framework to improve the performance of HCE by utilizing a progressive mechanism with local and global evaluation criteria to select ensemble members. Extensive comparative experiments conducted on 28 real-world data sets exhibit that our method PHCE outperforms the majority of imbalance ensemble classification approaches.

Semi-supervised generative learning (SSGL) makes use of unlabeled data to achieve a trade-off between the data collection/annotation effort and generation performance, when adequate labeled data are not available. Learning precise class semantics is crucial for class-conditional image synthesis with limited supervision. Toward this end, we propose a semi-supervised Generative Adversarial Network with a Mask-Embedded Discriminator, which is referred to as MED-GAN. By incorporating a mask embedding module, the discriminator features are associated with spatial information, such that the focus of the discriminator can be limited in the specified regions when distinguishing between real and synthesized images. A generator is enforced to synthesize the instances holding more precise class semantics in order to deceive the enhanced discriminator. Also benefiting from mask embedding, region-based semantic regularization is imposed on the discriminator feature space, and the degree of separation between real and fake classes and among object categories can thus be increased. This eventually improves class-conditional distribution matching between real and synthesized data. In the experiments, the superior performance of MED-GAN demonstrates the effectiveness of mask embedding and associated regularizers in facilitating SSGL.

Data interpolation is typically used to explore and understand the latent representation learnt by a deep network. Naive linear interpolation may induce mismatch between the interpolated data and the underlying manifold of the original data. In this paper, we propose an Adversarial Adaptive Interpolation (AdvAI) approach for facilitating representation learning and image synthesis in autoencoders. To determine an interpolation path that stays on the manifold, we incorporate an interpolation correction module, which learns to offset the deviation from the manifold. Further, we perform matching with a prior distribution to control the characteristics of the representation. The data synthesized from random codes along with interpolation-based regularization are in turn used to constrain the representation learning process. In the experiments, the superior performance of the proposed approach demonstrates the effectiveness of AdvAI and associated regularizers in a variety of downstream tasks.

In this work, we propose an unsupervised ensemble learning method via network generation, referred to as UELNG. Specifically, we first generate weights of clustering ensemble models by adopting HyperGAN, and obtain diverse partitions for data. With these partitions, we can easily identify high-confident pseudo-labels as supervised information to predict low-entropy labels for unlabeled augmented data, thereby enhancing the quality of pseudo-labels and clustering accuracy. We conduct experiments on multiple data sets. Experimental results indicate that our method outperforms state-of-the-art methods by 0.3%, 1.8%, 5.7%, 3.2% and 2.4% on MNIST, STL-10, CIFAR-10, Reuters and 20News, respectively, which demonstrates the effectiveness of our proposed UELNG.

Learning and classification of imbalanced data is a quite common challenge in machine learning. In order to generate more high-quality minority samples in imbalanced data and help solve the classification process, this paper proposes a novel local tangent generative adversarial network (LT-GAN). In LT-GAN, a local tangent based generator is designed to generate realistic and diverse minority class samples by learning the local tangent space of original minority samples. Meanwhile, a two-function discriminator is explored to judge the authenticity of samples and distinguish the majority samples from the minority samples. With the synthesis of minority class samples, the generator and discriminator are trained together by using adversarial learning. Experiments and comparisons show that our proposed LT-GAN outperforms other techniques and significantly improves the classification performance of imbalanced data.

The newly proposed Broad Learning System (BLS) offers an alternative way to deep learning which saves a time-consuming training process and powerful computing resources. However, the randomly generated feature nodes and lots of enhancement nodes in BLS may reduce the performance of the final classifier. Aiming at the problems in randomly generated feature nodes which suffer from unpredictability and need guidance, this paper proposes an adaptive feature nodes evolutionary algorithm (AFNE) to extract better features; While broad learning neural network often requires a large number of enhancement nodes parameters to ensure its performance, which easily leads to the redundancy or dependency between features, as well as the performance degradation of the final model. Therefore, this article also proposes a new criterion based on node sensitivity to prune the network enhancement layer nodes to remove redundant nodes, reduce the network scale, and increase the generalization ability. The proposed algorithm ADP-BLS can improve the accuracy and generalization performance of the final classifier through the evolution of feature nodes and the pruning of enhancement nodes. Extensive comparative experiments on real world data sets verify the effectiveness of the proposed ADP-BLS. At the same time, when verifying the module validity of the innovative algorithms added in this article, experiments also show that the AFNE and pruning method integrated into BLS can improve the model to a certain extent.

Healthcare question answering (HQA) system plays a vital role in encouraging patients to inquire for professional consultation. However, there are some challenging factors in learning and representing the question corpus of HQA datasets, such as high dimensionality, sparseness, noise, nonprofessional expression, etc. To address these issues, we propose an inception convolutional autoencoder model for Chinese healthcare question clustering (ICAHC). First, we select a set of kernels with different sizes using convolutional autoencoder networks to explore both the diversity and quality in the clustering ensemble. Thus, these kernels encourage to capture diverse representations. Second, we design four ensemble operators to merge representations based on whether they are independent, and input them into the encoder using different skip connections. Third, it maps features from the encoder into a lower-dimensional space, followed by clustering. We conduct comparative experiments against other clustering algorithms on a Chinese healthcare dataset. Experimental results show the effectiveness of ICAHC in discovering better clustering solutions. The results can be used in the prediction of patients' conditions and the development of an automatic HQA system.

The tongue image provides important physical information of humans. It is of great importance for diagnoses and treatments in clinical medicine. Herbal prescriptions are simple, noninvasive, and have low side effects. Thus, they are widely applied in China. Studies on the automatic construction technology of herbal prescriptions based on tongue images have great significance for deep learning to explore the relevance of tongue images for herbal prescriptions, it can be applied to healthcare services in mobile medical systems. In order to adapt to the tongue image in a variety of photographic environments and construct herbal prescriptions, a neural network framework for prescription construction is designed. It includes single/double convolution channels and fully connected layers. Furthermore, it proposes the auxiliary therapy topic loss mechanism to model the therapy of Chinese doctors and alleviate the interference of sparse output labels on the diversity of results. The experiment use the real-world tongue images and the corresponding prescriptions and the results can generate prescriptions that are close to the real samples, which verifies the feasibility of the proposed method for the automatic construction of herbal prescriptions from tongue images. Also, it provides a reference for automatic herbal prescription construction from more physical information.

As efficient recurrent neural network (RNN) models, echo state networks (ESNs) have attracted widespread attention and been applied in many application domains in the last decade. Although they have achieved great success in modeling time series, a single ESN may have difficulty in capturing the multitimescale structures that naturally exist in temporal data. In this paper, we propose the convolutional multitimescale ESN (ConvMESN), which is a novel training-efficient model for capturing multitimescale structures and multiscale temporal dependencies of temporal data. In particular, a multitimescale memory encoder is constructed with a multireservoir structure, in which different reservoirs have recurrent connections with different skip lengths (or time spans). By collecting all past echo states in each reservoir, this multireservoir structure encodes the history of a time series as nonlinear multitimescale echo state representations (MESRs). Our visualization analysis verifies that the MESRs provide better discriminative features for time series. Finally, multiscale temporal dependencies of MESRs are learned by a convolutional layer. By leveraging the multitimescale reservoirs followed by a convolutional learner, the ConvMESN has not only efficient memory encoding ability for temporal data with multitimescale structures but also strong learning ability for complex temporal dependencies. Furthermore, the training-free reservoirs and the single convolutional layer provide high-computational efficiency for the ConvMESN to model complex temporal data. Extensive experiments on 18 multivariate time series (MTS) benchmark datasets and 3 skeleton-based action recognition datasets demonstrate that the ConvMESN captures multitimescale dynamics and outperforms existing methods.

Clustering is a fundamental data exploration task which aims at discovering the hidden grouping structure in the data. The traditional clustering methods typically compute a single partition. However, there often exist different and equally meaningful clusterings in complex data. To solve this issue, multiple clustering approaches have emerged with the goal of exploring alternative clusterings from different perspectives. Existing solutions to this problem mainly focus on one-way clustering, that is, they cluster either the samples or the features. However, for many practical tasks, it is meaningful and desirable to explore alternative two-way clusterings (or co-clusterings), which capture not only the sample cluster structure but also the feature cluster structure. To tackle this interesting and unresolved task, we introduce an approach, called multiple co-clusterings (MultiCCs), to generate multiple alternative co-clusterings at the same time. MultiCC takes advantage of matrix tri-factorization to seek the co-clustering indicator matrices for samples and features and defines the row and column redundancy quantification terms to enforce diversity among co-clusterings based on these indicator matrices. After that, it integrates matrix tri-factorization and two nonredundancy terms into a unified objective function and gives an alternative optimization procedure to optimize the objective function. Extensive experimental results demonstrate that MultiCC performs significantly better than the existing multiple clustering methods. In addition, MultiCC can find out interesting co-clusters, which cannot be made by those comparing methods.

Classifying high-dimensional small-size data is challenging in the field of pattern recognition. Traditional ensemble learning methods have several limitations: 1) sample-space based methods are easily affected by noise and redundant features; 2) feature-space based methods cannot excavate the essential characteristics of features; 3) feature subspaces cause information loss, which leads to a decline in accuracy; 4) most selective ensemble methods only consider the diversity and performance of sub-classifiers and ignore the impact on integration systems. To address the above limitations, we propose an adaptive classifier ensemble learning method (AdaSPEL) based on spatial perception for high-dimensional data. First, we design a local-space perception method for feature transformation, which encourages both high performance and diversity of the ensemble members. Second, we design a cross-space perception method based on the distribution of samples to obtain the cross-space enhanced features to provide a macro analysis for the characteristics of data. Furthermore, an adaptive selective ensemble method based on local and global evaluation mechanisms is proposed, which considers the impact of sub-classifiers on integrated systems. Experimental results on 33 high-dimensional data sets verify that our method outperforms mainstream ensemble learning methods based on feature space and sample space, and neural network-based algorithms.

Semisupervised clustering methods improve performance by randomly selecting pairwise constraints, which may lead to redundancy and instability. In this context, active clustering is proposed to maximize the efficacy of annotations by effectively using pairwise constraints. However, existing methods lack an overall consideration of the querying criteria and repeatedly run semisupervised clustering to update labels. In this work, we first propose an active density peak (ADP) clustering algorithm that considers both representativeness and informativeness. Representative instances are selected to capture data patterns, while informative instances are queried to reduce the uncertainty of clustering results. Meanwhile, we design a fast-update-strategy to update labels efficiently. In addition, we propose an active clustering ensemble framework that combines local and global uncertainties to query the most ambiguous instances for better separation between the clusters. A weighted voting consensus method is introduced for better integration of clustering results. We conducted experiments by comparing our methods with state-of-the-art methods on real-world data sets. Experimental results demonstrate the effectiveness of our methods.

Large-scale data clustering is an essential key for big data problem. However, no current existing approach is “optimal” for big data due to high complexity, which remains it a great challenge. In this article, a simple but fast approximate DBSCAN, namely, KNN-BLOCK DBSCAN, is proposed based on two findings: 1) the problem of identifying whether a point is a core point or not is, in fact, a kNN problem and 2) a point has a similar density distribution to its neighbors, and neighbor points are highly possible to be the same type (core point, border point, or noise). KNN-BLOCK DBSCAN uses a fast approximate kNN algorithm, namely, FLANN, to detect core-blocks (CBs), noncore-blocks, and noise-blocks within which all points have the same type, then a fast algorithm for merging CBs and assigning noncore points to proper clusters is also invented to speedup the clustering process. The experimental results show that KNN-BLOCK DBSCAN is an effective approximate DBSCAN algorithm with high accuracy, and outperforms other current variants of DBSCAN, including ρ-approximate DBSCAN and AnyDBC.

Semi-supervised generative learning aims to learn the underlying class-conditional distribution of partially labeled data. Generative Adversarial Networks (GANs) have led to promising progress in this task. However, it still needs to further explore the issue of imbalance between real labeled data and fake data in the adversarial learning process. To address this issue, we propose a regularization technique based on Random Regional Replacement (R3-regularization) to facilitate the generative learning process. Specifically, we construct two types of between-class instances: cross-category ones and real-fake ones. These instances could be closer to the decision boundaries and are important for regularizing the classification and discriminative networks in our class-conditional GANs, which we refer to as R3-CGAN. Better guidance from these two networks makes the generative network produce instances with class-specific information and high fidelity. We experiment with multiple standard benchmarks, and demonstrate that the R3-regularization can lead to significant improvement in both classification and class-conditional image synthesis.

Multitask feature selection (MTFS) methods have become more important for many real world applications, especially in a high-dimensional setting. The most widely used assumption is that all tasks share the same features, and the l2,1 regularization method is usually applied. However, this assumption may not hold when the correlations among tasks are not obvious. Learning with unrelated tasks together may result in negative transfer and degrade the performance. In this paper, we present a flexible MTFS by graph-clustered feature sharing approach. To avoid the above limitation, we adopt a graph to represent the relevance among tasks instead of adopting a hard task set partition. Furthermore, we propose a graph-guided regularization approach such that the sparsity of the solution can be achieved on both the task level and the feature level, and a variant of the smooth proximal gradient method is developed to solve the corresponding optimization problem. An evaluation of the proposed method on multitask regression and multitask binary classification problem has been performed. Extensive experiments on synthetic datasets and real-world datasets demonstrate the effectiveness of the proposed approach to capture task structure.

Time series with missing values (incomplete time series) are ubiquitous in real life on account of noise or malfunctioning sensors. Time-series imputation (replacing missing data) remains a challenge due to the potential for nonlinear dependence on concurrent and previous values of the time series. In this paper, we propose a novel framework for modeling incomplete time series, called a linear memory vector recurrent neural network (LIME-RNN), a recurrent neural network (RNN) with a learned linear combination of previous history states. The technique bears some similarity to residual networks and graph-based temporal dependency imputation. In particular, we introduce a linear memory vector [called the residual sum vector (RSV)] that integrates over previous hidden states of the RNN, and is used to fill in missing values. A new loss function is developed to train our model with time series in the presence of missing values in an end-to-end way. Our framework can handle imputation of both missing-at-random and consecutive missing inputs. Moreover, when conducting time-series prediction with missing values, LIME-RNN allows imputation and prediction simultaneously. We demonstrate the efficacy of the model via extensive experimental evaluation on univariate and multivariate time series, achieving state-of-the-art performance on synthetic and real-world data. The statistical results show that our model is significantly better than most existing time-series univariate or multivariate imputation methods.

Clustering ensemble (CE) takes multiple clustering solutions into consideration in order to effectively improve the accuracy and robustness of the final result. To reduce redundancy as well as noise, a CE selection (CES) step is added to further enhance performance. Quality and diversity are two important metrics of CES. However, most of the CES strategies adopt heuristic selection methods or a threshold parameter setting to achieve tradeoff between quality and diversity. In this paper, we propose a transfer CES (TCES) algorithm which makes use of the relationship between quality and diversity in a source dataset, and transfers it into a target dataset based on three objective functions. Furthermore, a multiobjective self-evolutionary process is designed to optimize these three objective functions. Finally, we construct a transfer CE framework (TCE-TCES) based on TCES to obtain better clustering results. The experimental results on 12 transfer clustering tasks obtained from the 20newsgroups dataset show that TCE-TCES can find a better tradeoff between quality and diversity, as well as obtaining more desirable clustering results.

Using an ensemble of neural networks with consistency regularization is effective for improving performance and stability of deep learning, compared to the case of a single network. In this paper, we present a semi-supervised Deep Coupled Ensemble (DCE) model, which contributes to ensemble learning and classification landmark exploration for better locating the final decision boundaries in the learnt latent space. First, multiple complementary consistency regularizations are integrated into our DCE model to enable the ensemble members to learn from each other and themselves, such that training experience from different sources can be shared and utilized during training. Second, in view of the possibility of producing incorrect predictions on a number of difficult instances, we adopt class-wise mean feature matching to explore important unlabeled instances as classification landmarks, on which the model predictions are more reliable. Minimizing the weighted conditional entropy on unlabeled data is able to force the final decision boundaries to move away from important training data points, which facilitates semi-supervised learning. Ensemble members could eventually have similar performance due to consistency regularization, and thus only one of these members is needed during the test stage, such that the efficiency of our model is the same as the non-ensemble case. Extensive experimental results demonstrate the superiority of our proposed DCE model over existing state-of-the-art semi-supervised learning methods.

In this paper, we explore how to leverage readily available unlabeled data to improve semi-supervised human detection performance. For this purpose, we specifically modify the region proposal network (RPN) for learning on a partially labeled dataset. Based on commonly observed false positive types, a verification module is developed to assess foreground human objects in the candidate regions to provide an important cue for filtering the RPN's proposals. The remaining proposals with high confidence scores are then used as pseudo annotations for re-training our detection model. To reduce the risk of error propagation in the training process, we adopt a self-paced training strategy to progressively include more pseudo annotations generated by the previous model over multiple training rounds. The resulting detector re-trained on the augmented data can be expected to have better detection performance. The effectiveness of the main components of this framework is verified through extensive experiments, and the proposed approach achieves state-of-the-art detection results on multiple scene-specific human detection benchmarks in the semi-supervised setting.

The class imbalance problem has become a leading challenge. Although conventional imbalance learning methods are proposed to tackle this problem, they have some limitations: 1) undersampling methods suffer from losing important information and 2) cost-sensitive methods are sensitive to outliers and noise. To address these issues, we propose a hybrid optimal ensemble classifier framework that combines density-based undersampling and cost-effective methods through exploring state-of-the-art solutions using multi-objective optimization algorithm. Specifically, we first develop a density-based undersampling method to select informative samples from the original training data with probability-based data transformation, which enables to obtain multiple subsets following a balanced distribution across classes. Second, we exploit the cost-sensitive classification method to address the incompleteness of information problem via modifying weights of misclassified minority samples rather than the majority ones. Finally, we introduce a multi-objective optimization procedure and utilize connections between samples to self-modify the classification result using an ensemble classifier framework. Extensive comparative experiments conducted on real-world data sets demonstrate that our method outperforms the majority of imbalance and ensemble classification approaches.

For unsupervised domain adaptation, the process of learning domain-invariant representations could be dominated by the labeled source data, such that the specific characteristics of the target domain may be ignored. In order to improve the performance in inferring target labels, we propose a targetspecific network which is capable of learning collaboratively with a domain adaptation network, instead of directly minimizing domain discrepancy. A clustering regularization is also utilized to improve the generalization capability of the target-specific network by forcing target data points to be close to accumulated class centers. As this network learns and specializes to the target domain, its performance in inferring target labels improves, which in turn facilitates the learning process of the adaptation network. Therefore, there is a mutually beneficial relationship between these two networks. We perform extensive experiments on multiple digit and object datasets, and the effectiveness and superiority of the proposed approach is presented and verified on multiple visual adaptation benchmarks, e.g., we improve the state-ofthe-art on the task of MNIST→SVHN from 76.5% to 84.9% without specific augmentation.

Learning class-conditional data distributions is crucial for Generative Adversarial Networks (GAN) in semi-supervised learning. To improve both instance synthesis and classification in this setting, we propose an enhanced TripleGAN (EnhancedTGAN) model in this work. We follow the adversarial training scheme of the original TripleGAN, but completely re-design the training targets of the generator and classifier. Specifically, we adopt feature-semantics matching to enhance the generator in learning class-conditional distributions from both the aspects of statistics in the latent space and semantics consistency with respect to the generator and classifier. Since a limited amount of labeled data is not sufficient to determine satisfactory decision boundaries, we include two classifiers, and incorporate collaborative learning into our model to provide better guidance for generator training. The synthesized high-fidelity data can in turn be used for improving classifier training. In the experiments, the superior performance of our approach on multiple benchmark datasets demonstrates the effectiveness of the mutual reinforcement between the generator and classifiers in facilitating semi-supervised instance synthesis and classification.

Deep mutual learning jointly trains multiple essential networks having similar properties to improve semi-supervised classification. However, the commonly used consistency regularization between the outputs of the networks may not fully leverage the difference between them. In this paper, we explore how to capture the complementary information to enhance mutual learning. For this purpose, we propose a complementary correction network (CCN), built on top of the essential networks, to learn the mapping from the output of one essential network to the ground truth label, conditioned on the features learnt by another. To make the second essential network increasingly complementary to the first one, this network is supervised by the corrected predictions. As a result, minimizing the prediction divergence between the two complementary networks can lead to significant performance gains in semi-supervised learning. Our experimental results demonstrate that the proposed approach clearly improves mutual learning between essential networks, and achieves state-of-the-art results on multiple semi-supervised classification benchmarks. In particular, the test error rates are reduced from previous 21.23% and 14.65% to 12.05% and 10.37% on CIFAR-10 with 1000 and 2000 labels, respectively.

Classification of high-dimensional data with very limited labels is a challenging task in the field of data mining and machine learning. In this paper, we propose the multiobjective semisupervised classifier ensemble (MOSSCE) approach to address this challenge. Specifically, a multiobjective subspace selection process (MOSSP) in MOSSCE is first designed to generate the optimal combination of feature subspaces. Three objective functions are then proposed for MOSSP, which include the relevance of features, the redundancy between features, and the data reconstruction error. Then, MOSSCE generates an auxiliary training set based on the sample confidence to improve the performance of the classifier ensemble. Finally, the training set, combined with the auxiliary training set, is used to select the optimal combination of basic classifiers in the ensemble, train the classifier ensemble, and generate the final result. In addition, diversity analysis of the ensemble learning process is applied, and a set of nonparametric statistical tests is adopted for the comparison of semisupervised classification approaches on multiple datasets. The experiments on 12 gene expression datasets and two large image datasets show that MOSSCE has a better performance than other state-of-the-art semisupervised classifiers on high-dimensional data.

In the era of data science, a huge amount of data has emerged in the form of tensors. In many applications, the collected tensor data are incomplete with missing entries, which affects the analysis process. In this paper, we investigate a new method for tensor completion, in which a low-rank tensor approximation is used to exploit the global structure of data, and sparse coding is used for elucidating the local patterns of data. Regarding the characterization of low-rank structures, a weighted nuclear norm for the tensor is introduced. Meanwhile, an orthogonal dictionary learning process is incorporated into sparse coding for more effective discovery of the local details of data. By simultaneously using the global patterns and local cues, the proposed method can effectively and efficiently recover the lost information of incomplete tensor data. The capability of the proposed method is demonstrated with several experiments on recovering MRI data and visual data, and the experimental results have shown the excellent performance of the proposed method in comparison with recent related methods.

Traditional ensemble learning approaches explore the feature space and the sample space, respectively, which will prevent them to construct more powerful learning models for noisy real-world dataset classification. The random subspace method only search for the selection of features. Meanwhile, the bagging approach only search for the selection of samples. To overcome these limitations, we propose the hybrid incremental ensemble learning (HIEL) approach which takes into consideration the feature space and the sample space simultaneously to handle noisy dataset. Specifically, HIEL first adopts the bagging technique and linear discriminant analysis to remove noisy attributes, and generates a set of bootstraps and the corresponding ensemble members in the subspaces. Then, the classifiers are selected incrementally based on a classifier-specific criterion function and an ensemble criterion function. The corresponding weights for the classifiers are assigned during the same process. Finally, the final label is summarized by a weighted voting scheme, which serves as the final result of the classification. We also explore various classifier-specific criterion functions based on different newly proposed similarity measures, which will alleviate the effect of noisy samples on the distance functions. In addition, the computational cost of HIEL is analyzed theoretically. A set of nonparametric tests are adopted to compare HIEL and other algorithms over several datasets. The experiment results show that HIEL performs well on the noisy datasets. HIEL outperforms most of the compared classifier ensemble methods on 14 out of 24 noisy real-world UCI and KEEL datasets.

High dimensional data classification with very limited labeled training data is a challenging task in the area of data mining. In order to tackle this task, we first propose a feature selection-based semi-supervised classifier ensemble framework (FSCE) to perform high dimensional data classification. Then, we design an adaptive semi-supervised classifier ensemble framework (ASCE) to improve the performance of FSCE. When compared with FSCE, ASCE is characterized by an adaptive feature selection process, an adaptive weighting process (AWP), and an auxiliary training set generation process (ATSGP). The adaptive feature selection process generates a set of compact subspaces based on the selected attributes obtained by the feature selection algorithms, while the AWP associates each basic semi-supervised classifier in the ensemble with a weight value. The ATSGP enlarges the training set with unlabeled samples. In addition, a set of nonparametric tests are adopted to compare multiple semi-supervised classifier ensemble (SSCE)approaches over different datasets. The experiments on 20 high dimensional real-world datasets show that: 1) the two adaptive processes in ASCE are useful for improving the performance of the SSCE approach and 2) ASCE works well on high dimensional datasets with very limited labeled training data, and outperforms most state-of-the-art SSCE approaches.

Multitask clustering methods are proposed to improve performances of related tasks concurrently, because they explore the relationship among tasks via exploiting the coefficient matrix or the shared feature matrix. However, divergent effects of features in learning this relationship are seldom considered. To further improve performances, we propose a new multitask clustering approach through exploring correlations among tasks, clusters, and features based on effects of features on clusters. First, a Feature-Cluster (FeaCluster) matrix is introduced to capture the similarity and the distinct task-feature information simultaneously for each task. With the FeaCluster matrix, two affinities are calculated to constitute the interdependencies among tasks: the former is the graphical affinity based on feature-task and task-cluster correlations, while the latter is the reconstructive affinity. Here, the feature-task correlation considers effects of features on tasks, and the task-cluster correlation considers the overall effects of features on clusters. The reconstructive affinity is obtained by minimizing the reconstruction error when representing the FeaCluster matrix for a given task with a linear combination of others. The interdependencies among tasks allow transferring asymmetric shared information, exploring significant features and preserving key information when mapping data into the subspace. The experimental results on multiple data sets reveal that the proposed approach outperforms the state-of-the-art clustering methods in terms of accuracy and normal mutual information.

The goal of multiple clusterings is to discover multiple independent ways of organizing a dataset into clusters. Current approaches to this problem just focus on one-way clustering. In many real-world applications, though, it's meaningful and desirable to explore alternative two-way clustering (or co-clusterings), where both samples and features are clustered. To tackle this challenge and unexplored problem, in this paper we introduce an approach, called Multiple Co-Clusterings (MultiCC), to discover non-redundant alternative co-clusterings. MultiCC makes use of matrix tri-factorization to optimize the sample-wise and feature-wise co-clustering indicator matrices, and introduces two non-redundancy terms to enforce diversity among co-clusterings. We then combine the objective of matrix tri-factorization and two non-redundancy terms into a unified objective function and introduce an iterative solution to optimize the function. Experimental results show that MultiCC outperforms existing multiple clustering methods, and it can find interesting co-clusters which cannot be discovered by current solutions.

The objective of cluster analysis is to partition a set of data points into several groups based on a suitable distance measure. We first propose a model called local gravitation among data points. In this model, each data point is viewed as an object with mass, and associated with a local resultant force (LRF) generated by its neighbors. The motivation of this paper is that there exist distinct differences between the LRFs (including magnitudes and directions) of the data points close to the cluster centers and at the boundary of the clusters. To capture these differences efficiently, two new local measures named centrality and coordination are further investigated. Based on empirical observations, two new clustering methods called local gravitation clustering and communication with local agents are designed, and several test cases are conducted to verify their effectiveness. The experiments on synthetic data sets and real-world data sets indicate that both clustering approaches achieve good performance on most of the data sets.

Semisupervised learning methods are often adopted to handle datasets with very small number of labeled samples. However, conventional semisupervised ensemble learning approaches have two limitations: 1) most of them cannot obtain satisfactory results on high dimensional datasets with limited labels and 2) they usually do not consider how to use an optimization process to enlarge the training set. In this paper, we propose the progressive semisupervised ensemble learning approach (PSEMISEL) to address the above limitations and handle datasets with very small number of labeled samples. When compared with traditional semisupervised ensemble learning approaches, PSEMISEL is characterized by two properties: 1) it adopts the random subspace technique to investigate the structure of the dataset in the subspaces and 2) a progressive training set generation process and a self evolutionary sample selection process are proposed to enlarge the training set. We also use a set of nonparametric tests to compare different semisupervised ensemble learning methods over multiple datasets. The experimental results on 18 real-world datasets from the University of California, Irvine machine learning repository show that PSEMISEL works well on most of the real-world datasets, and outperforms other state-of-the-art approaches on 10 out of 18 datasets.

Traditional cluster ensemble approaches have several limitations. (1) Few make use of prior knowledge provided by experts. (2) It is difficult to achieve good performance in high-dimensional datasets. (3) All of the weight values of the ensemble members are equal, which ignores different contributions from different ensemble members. (4) Not all pairwise constraints contribute to the final result. In the face of this situation, we propose double weighting semi-supervised ensemble clustering based on selected constraint projection(DCECP) which applies constraint weighting and ensemble member weighting to address these limitations. Specifically, DCECP first adopts the random subspace technique in combination with the constraint projection procedure to handle high-dimensional datasets. Second, it treats prior knowledge of experts as pairwise constraints, and assigns different subsets of pairwise constraints to different ensemble members. An adaptive ensemble member weighting process is designed to associate different weight values with different ensemble members. Third, the weighted normalized cut algorithm is adopted to summarize clustering solutions and generate the final result. Finally, nonparametric statistical tests are used to compare multiple algorithms on real-world datasets. Our experiments on 15 high-dimensional datasets show that DCECP performs better than most clustering algorithms.

In a semi-supervised setting, direct training of a deep discriminative model on partially labeled images often suffers from overfitting and poor performance, because only a small number of labeled images are available, and errors in label propagation are, in many cases, inevitable. In this paper, we introduce an auxiliary clustering task to explore the structure of the image data, and judiciously weigh unlabeled data to alleviate the influence of ambiguous data on model training. For this purpose, we propose a cross-task network composed of two streams to jointly learn two tasks: classification and clustering. Based on the model predictions, a large number of pairwise constraints can be generated from unlabeled images, and are fed to the clustering stream. Since pairwise constraints encode weak supervision information, the clustering is tolerant of errors in labeling. Unlabeled images are weighted according to the distances to the clusters discovered, and a better discriminative model is trained on the classification stream associated with a weighted softmax loss. Furthermore, a self-paced learning paradigm is adopted to gradually train our deep model from easy examples to difficult ones. Experimental results on widely used image classification datasets confirm the effectiveness and superiority of the proposed approach.

Taxi dispatch is a critical issue for taxi company to consider in modern life. This paper formulates the problem into a taxi-passenger matching model and proposes a parallel ant colony optimization algorithm to optimize the model. As the search space is large, we develop a region-dependent decomposition strategy to divide and conquer the problem. To keep the global performance, a critical region is defined to deal with the communications and interactions between the subregions. The experimental results verify that the proposed algorithm is effective, efficient, and extensible, which outperforms the traditional global perspective greedy algorithm in terms of both accuracy and efficiency.

Virtual network embedding (VNE), which is a significant problem in cloud computing, has gained much attention among many researchers recently. Due to the NP-hardness of VNE, the existing solvers are either inefficient or inaccurate. This paper develops a novel algorithm based on the ant colony system (ACS). To solve the VNE problem, the algorithm structure concentrates on the link mapping from virtual network to substrate network. Particularly, for a specific virtual network request, we first sort the embedding sequence of virtual nodes according to their link resources. Then, ACS is used to embed the virtual nodes onto substrate nodes according the sorted sequence, while the virtual links are mapped via a shortest path strategy for the embedded nodes. For the first time, we propose a link resource heuristic information and incorporate it into the search process of ACS. The link resource heuristic information has two significant effects, one is to make virtual nodes tend to be embedded on the substrate nodes that cost less bandwidth, and the other is to confirm the connectivity of the substrate nodes that embed the virtual nodes. The proposed algorithm improves the optimization performance of VNE when compared with a few existing algorithms, while it substantially reduces the cost of time.

Clustering ensemble approaches usually have more accurate, robust and stable results than traditional single clustering approaches. However, clustering ensemble can still be improved in the following aspects: (1) improve the diversity of subspaces; (2) employ probabilistic latent clustering; (3) adopt the internal latent factor analysis before the consensus function. Therefore, we propose a new clustering ensemble framework. Specifically, we analysis the original data via Jensen-Shannon divergence distribution, and then soft subspaces are generated according to the corresponding fuzzy matrix. Next, the probabilistic latent semantic analysis clustering algorithm is employed to perform clustering in each soft subspace. The final clustering performance is improved due to the reason that the latent factor model is applied to fusion matrix. Compared with traditional single and ensemble clustering algorithms, our framework achieves superior performances on 12 real-world datasets.

Nonparametric statistical analysis, such as the Friedman test (FT), is gaining more and more attention due to its useful applications in a lot of experimental studies. However, traditional FT for the comparison of multiple learning algorithms on different datasets adopts the naive ranking approach. The ranking is based on the average accuracy values obtained by the set of learning algorithms on the datasets, which neither considers the differences of the results obtained by the learning algorithms on each dataset nor takes into account the performance of the learning algorithms in each run. In this paper, we will first propose three kinds of ranking approaches, which are the weighted ranking approach, the global ranking approach (GRA), and the weighted GRA. Then, a theoretical analysis is performed to explore the properties of the proposed ranking approaches. Next, a set of the modified FTs based on the proposed ranking approaches are designed for the comparison of the learning algorithms. Finally, the modified FTs are evaluated through six classifier ensemble approaches on 34 real-world datasets. The experiments show the effectiveness of the modified FTs.

The objective of cluster structure ensemble is to find a unified cluster structure from multiple cluster structures obtained from different datasets. Unfortunately, not all the cluster structures contribute to the unified cluster structure. This paper investigates the problem of how to select the suitable cluster structures in the ensemble which will be summarized to a more representative cluster structure. Specifically, the cluster structure is first represented by a mixture of Gaussian distributions, the parameters of which are estimated using the expectation-maximization algorithm. Then, several distribution-based distance functions are designed to evaluate the similarity between two cluster structures. Based on the similarity comparison results, we propose a new approach, which is referred to as the distribution-based cluster structure ensemble (DCSE) framework, to find the most representative unified cluster structure. We then design a new technique, the distribution-based cluster structure selection strategy (DCSSS), to select a subset of cluster structures. Finally, we propose using a distribution-based normalized hypergraph cut algorithm to generate the final result. In our experiments, a nonparametric test is adopted to evaluate the difference between DCSE and its competitors. We adopt 20 real-world datasets obtained from the University of California, Irvine and knowledge extraction based on evolutionary learning repositories, and a number of cancer gene expression profiles to evaluate the performance of the proposed methods. The experimental results show that: 1) DCSE works well on the real-world datasets and 2) DCSE based on DCSSS can further improve the performance of the algorithm.

Conventional semi-supervised clustering approaches have several shortcomings, such as (1) not fully utilizing all useful must-link and cannot-link constraints, (2) not considering how to deal with high dimensional data with noise, and (3) not fully addressing the need to use an adaptive process to further improve the performance of the algorithm. In this paper, we first propose the transitive closure based constraint propagation approach, which makes use of the transitive closure operator and the affinity propagation to address the first limitation. Then, the random subspace based semi-supervised clustering ensemble framework with a set of proposed confidence factors is designed to address the second limitation and provide more stable, robust, and accurate results. Next, the adaptive semi-supervised clustering ensemble framework is proposed to address the third limitation, which adopts a newly designed adaptive process to search for the optimal subspace set. Finally, we adopt a set of nonparametric tests to compare different semi-supervised clustering ensemble approaches over multiple datasets. The experimental results on 20 real high dimensional cancer datasets with noisy genes and 10 datasets from UCI datasets and KEEL datasets show that (1) The proposed approaches work well on most of the real-world datasets. (2) It outperforms other state-of-the-art approaches on 12 out of 20 cancer datasets, and 8 out of 10 UCI machine learning datasets.

Differential evolution (DE) is a simple and efficient evolutionary algorithm for global optimization. In distributed differential evolution (DDE), the population is divided into several sub-populations and each sub-population evolves independently for enhancing population diversity as well as algorithmic performance. Sub-populations in DDE share their elite individuals with neighborhood through a predefined migration topology. However, the construction of traditional migration topologies does not consider the position information of sub-populations in the search space. The position information is helpful in controlling the degree of diversity between the sub-populations and their migrated individuals. A proper degree of diversity could promote the balance between exploration and exploitation for DDE algorithms. To achieve this target, a dynamic space-driven migration topology is proposed in this paper. The proposed topology is constructed and updated according to the distances between sub-populations. Based on this proposed topology, some sub-populations receive diverse individuals from neighborhood far away while others communicate with neighborhood nearby. Numerical experiments have been performed on 13 diverse test functions. Results verify the advantage of DDE with the proposed migration topology compared to those with classic topologies.

Many real-world optimization problems encounter the presence of uncertainties. Dynamic optimization is a class of problems whose fitness functions vary through time. For these problems, evolutionary algorithm is expected to adapt to the changing environments immediately and find the best solution accurately. Besides, most of the environmental changes may not be too drastic in real-world applications, which indicates the evolutionary information in the past may be helpful for finding the optimum solution in the new environment. As a result, effective reuse of historical evolutionary information is necessary, since it leads a faster convergence after a change has occurred. This paper develops an orthogonal learning particle swarm optimization (OLPSO) with the variable relocation strategy (VRS) to solve the dynamic optimization problem (DOP). The proposed OLPSO-VRS algorithm has two advantages as follows. First and foremost, the orthogonal learning strategy guides particles to fly in better directions by constructing a much promising and efficient exemplar, which can achieve a balance between fast convergence and population diversity. Furthermore, the VRS collects historical information in the stable search stage and then reuse such information to guide the particle variable relocation once the search environment has changed. This operation enables the algorithm to quickly shift to the new promising regions in the changing fitness landscape. We evaluated OLPSO-VRS on several dynamic benchmark problems and compared with several state-of-the-art dynamic algorithms. The results show that OLPSO-VRS obtains very competitive results and enjoys a statistically superior performance on most problems.

Cluster ensemble, as one of the important research directions in the ensemble learning area, is gaining more and more attention, due to its powerful capability to integrate multiple clustering solutions and provide a more accurate, stable and robust result. Cluster ensemble has a lot of useful applications in a large number of areas. Although most of traditional cluster ensemble approaches obtain good results, few of them consider how to achieve good performance for noisy datasets. Some noisy datasets have a number of noisy attributes which may degrade the performance of conventional cluster ensemble approaches. Some noisy datasets which contain noisy samples will affect the final results. Other noisy datasets may be sensitive to distance functions.

Recently, cluster ensemble approaches have gained more and more attention [1]–[2], due to useful applications in the areas of pattern recognition, data mining, bioinformatics, and so on. When compared with traditional single clustering algorithms, cluster ensemble approaches are able to integrate multiple clustering solutions obtained from different data sources into a unified solution, and provide a more robust, stable and accurate final result.

Conventional k -nearest neighbor (KNN) classification approaches have several limitations when dealing with some problems caused by the special datasets, such as the sparse problem, the imbalance problem, and the noise problem. In this paper, we first perform a brief survey on the recent progress of the KNN classification approaches. Then, the hybrid KNN (HBKNN) classification approach, which takes into account the local and global information of the query sample, is designed to address the problems raised from the special datasets. In the following, the random subspace ensemble framework based on HBKNN (RS-HBKNN) classifier is proposed to perform classification on the datasets with noisy attributes in the high-dimensional space. Finally, the nonparametric tests are proposed to be adopted to compare the proposed method with other classification approaches over multiple datasets. The experiments on the real-world datasets from the Knowledge Extraction based on Evolutionary Learning dataset repository demonstrate that RS-HBKNN works well on real datasets, and outperforms most of the state-of-the-art classification approaches.

Traditional cluster ensemble approaches have three limitations: (1) They do not make use of prior knowledge of the datasets given by experts. (2) Most of the conventional cluster ensemble methods cannot obtain satisfactory results when handling high dimensional data. (3) All the ensemble members are considered, even the ones without positive contributions. In order to address the limitations of conventional cluster ensemble approaches, we first propose an incremental semi-supervised clustering ensemble framework (ISSCE) which makes use of the advantage of the random subspace technique, the constraint propagation approach, the proposed incremental ensemble member selection process, and the normalized cut algorithm to perform high dimensional data clustering. The random subspace technique is effective for handling high dimensional data, while the constraint propagation approach is useful for incorporating prior knowledge. The incremental ensemble member selection process is newly designed to judiciously remove redundant ensemble members based on a newly proposed local cost function and a global cost function, and the normalized cut algorithm is adopted to serve as the consensus function for providing more stable, robust, and accurate results. Then, a measure is proposed to quantify the similarity between two sets of attributes, and is used for computing the local cost function in ISSCE. Next, we analyze the time complexity of ISSCE theoretically. Finally, a set of nonparametric tests are adopted to compare multiple semisupervised clustering ensemble approaches over different datasets. The experiments on 18 real-world datasets, which include six UCI datasets and 12 cancer gene expression profiles, confirm that ISSCE works well on datasets with very high dimensionality, and outperforms the state-of-the-art semi-supervised clustering ensemble approaches.

Performing clustering analysis is one of the important research topics in cancer discovery using gene expression profiles, which is crucial in facilitating the successful diagnosis and treatment of cancer. While there are quite a number of research works which perform tumor clustering, few of them considers how to incorporate fuzzy theory together with an optimization process into a consensus clustering framework to improve the performance of clustering analysis. In this paper, we first propose a random double clustering based cluster ensemble framework (RDCCE) to perform tumor clustering based on gene expression data. Specifically, RDCCE generates a set of representative features using a randomly selected clustering algorithm in the ensemble, and then assigns samples to their corresponding clusters based on the grouping results. In addition, we also introduce the random double clustering based fuzzy cluster ensemble framework (RDCFCE), which is designed to improve the performance of RDCCE by integrating the newly proposed fuzzy extension model into the ensemble framework. RDCFCE adopts the normalized cut algorithm as the consensus function to summarize the fuzzy matrices generated by the fuzzy extension models, partition the consensus matrix, and obtain the final result. Finally, adaptive RDCFCE (A-RDCFCE) is proposed to optimize RDCFCE and improve the performance of RDCFCE further by adopting a self-evolutionary process (SEPP) for the parameter set. Experiments on real cancer gene expression profiles indicate that RDCFCE and A-RDCFCE works well on these data sets, and outperform most of the state-of-the-art tumor clustering algorithms.

Traditional random subspace-based classifier ensemble approaches (RSCE) have several limitations, such as viewing the same importance for the base classifiers trained in different subspaces, not considering how to find the optimal random subspace set. In this paper, we design a general hybrid adaptive ensemble learning framework (HAEL), and apply it to address the limitations of RSCE. As compared with RSCE, HAEL consists of two adaptive processes, i.e., base classifier competition and classifier ensemble interaction, so as to adjust the weights of the base classifiers in each ensemble and to explore the optimal random subspace set simultaneously. The experiments on the real-world datasets from the KEEL dataset repository for the classification task and the cancer gene expression profiles show that: 1) HAEL works well on both the real-world KEEL datasets and the cancer gene expression profiles and 2) it outperforms most of the state-of-the-art classifier ensemble approaches on 28 out of 36 KEEL datasets and 6 out of 6 cancer datasets.

Cluster ensemble is one of the main branches in the ensemble learning area which is an important research focus in recent years. The objective of cluster ensemble is to combine multiple clustering solutions in a suitable way to improve the quality of the clustering result. In this paper, we design a new noise immune cluster ensemble framework named as AP2CE to tackle the challenges raised by noisy datasets. AP2CE not only takes advantage of the affinity propagation algorithm (AP) and the normalized cut algorithm (Ncut), but also possesses the characteristics of cluster ensemble. Compared with traditional cluster ensemble approaches, AP2CE is characterized by several properties. (1) It adopts multiple distance functions instead of a single Euclidean distance function to avoid the noise related to the distance function. (2) AP2CE applies AP to prune noisy attributes and generate a set of new datasets in the subspaces consists of representative attributes obtained by AP. (3) It avoids the explicit specification of the number of clusters. (4) AP2CE adopts the normalized cut algorithm as the consensus function to partition the consensus matrix and obtain the final result. In order to improve the performance of AP2CE, the adaptive AP2CE is designed, which makes use of an adaptive process to optimize a newly designed objective function. The experiments on both synthetic and real datasets show that (1) AP2CE works well on most of the datasets, in particular the noisy datasets; (2) AP2CE is a better choice for most of the datasets when compared with other cluster ensemble approaches; (3) AP2CE has the capability to provide more accurate, stable and robust results.

Multi-view learning, one of the important sub-fields in the area of machine learning, has gained more and more attention in class prediction of gene expression datasets. In this paper, we propose a new classifier ensemble framework, named as multi-view based Ad-a boost classifier ensemble framework (MV-ACE), which not only utilizes a random view generation technique to regulate different views and applies adaboost to adjust the training set, but also designs an adaptive process which explores the feasible combination of multiple views through an optimization process. Traditional multi-view learning focuses on exploring diverse views and the best integration of multiple views in a straight-forward manner, such as the linear combination of different views. Our proposed model, however, additionally applies a progressive training approach to improve the accuracies of the base classifiers. Moreover, we investigate the assembly of views at the model level, and employ an adaptive process to optimize the multi-view learning model to improve its performance. Our experiments on 12 cancer gene data sets for the classification task show that(i) MV-ACE works well on a diverse class of cancer gene expression profiles. (ii) It outperforms most of the state-of-the-art classifier ensemble approaches on these datasets.

Tumor clustering is one of the important techniques for tumor discovery from cancer gene expression profiles, which is useful for the diagnosis and treatment of cancer. While different algorithms have been proposed for tumor clustering, few make use of the expert's knowledge to better the performance of tumor discovery. In this paper, we first view the expert's knowledge as constraints in the process of clustering, and propose a feature selection based semi-supervised cluster ensemble framework (FS-SSCE) for tumor clustering from bio-molecular data. Compared with traditional tumor clustering approaches, the proposed framework FS-SSCE is featured by two properties: (1) The adoption of feature selection techniques to dispel the effect of noisy genes. (2) The employment of the binate constraint based K-means algorithm to take into account the effect of experts' knowledge. Then, a double selection based semi-supervised cluster ensemble framework (DS-SSCE) which not only applies the feature selection technique to perform gene selection on the gene dimension, but also selects an optimal subset of representative clustering solutions in the ensemble and improve the performance of tumor clustering using the normalized cut algorithm. DS-SSCE also introduces a confidence factor into the process of constructing the consensus matrix by considering the prior knowledge of the data set. Finally, we design a modified double selection based semi-supervised cluster ensemble framework (MDS-SSCE) which adopts multiple clustering solution selection strategies and an aggregated solution selection function to choose an optimal subset of clustering solutions. The results in the experiments on cancer gene expression profiles show that (i) FS-SSCE, DS-SSCE and MDS-SSCE are suitable for performing tumor clustering from bio-molecular data. (ii) MDS-SSCE outperforms a number of state-of-the-art tumor clustering approaches on most of the data sets.

The authors of "Protein Function Prediction Using Multilabel Ensemble Classification" [ibid., vol. 10, no. 4, pp. 1045-1057, July/Aug. 2013] point out a typo that occurred in the Acknowledgments section. The acknowledgment should instead read: "The authors would like to thank the anonymous reviewers and editors for their valuable comments. This paper was partially supported by grants from the US National Science Foundation (NSF) (no. IIS-0905117), the NSF Career Award (no. IIS-1252318), Natural Science Foundation of China (project nos. 61070090, 61003174, 61101234, 61372138), the Natural Science Foundation of Guangdong Province (S2012010009961), Specialized Research Fund for the Doctoral Program of Higher Education (20110172120027), Fundamental Research Funds for the Central Universities of China (project nos. XDJK2010B002, XDJK2013C026, XDJK2013C123, XDJK2014C044), Doctoral Fund of Southwest University (project nos. SWU110063 and SWU113034) and China Scholarship Council (CSC)."

Automated protein function prediction is one of the grand challenges in computational biology. Multi-label learning is widely used to predict functions of proteins. Most of multi-label learning methods make prediction for unlabeled proteins under the assumption that the labeled proteins are completely annotated, i.e., without any missing functions. However, in practice, we may have a subset of the ground-truth functions for a protein, and whether the protein has other functions is unknown. To predict protein functions with incomplete annotations, we propose a Protein Function Prediction method with Weak-label Learning (ProWL) and its variant ProWL-IF. Both ProWL and ProWL-IF can replenish the missing functions of proteins. In addition, ProWL-IF makes use of the knowledge that a protein cannot have certain functions, which can further boost the performance of protein function prediction. Our experimental results on protein-protein interaction networks and gene expression benchmarks validate the effectiveness of both ProWL and ProWL-IF.

People naturally escape from a place when unexpected events happen. Based on this observation, efficient detection of crowd escape behavior in surveillance videos is a promising way to perform timely detection of anomalous situations. In this paper, we propose a Bayesian framework for escape detection by directly modeling crowd motion in both the presence and absence of escape events. Specifically, we introduce the concepts of potential destinations and divergent centers to characterize crowd motion in the above two cases respectively, and construct the corresponding class-conditional probability density functions of optical flow. Escape detection is finally performed based on the proposed Bayesian framework. Although only data associated with nonescape behavior are included in the training set, the density functions associated with the case of escape can also be adaptively updated using observed data. In addition, the identified divergent centers indicate possible locations at which the unexpected events occur. The performance of our proposed method is validated in a number of experiments on crowd escape detection in various scenarios.

Precise cancer classification is essential to the successful diagnosis and treatment of cancers. Although semisupervised dimensionality reduction approaches perform very well on clean datasets, the topology of the neighborhood constructed with most existing approaches is unstable in the presence of high-dimensional data with noise. In order to solve this problem, a novel local and global preserving semisupervised dimensionality reduction based on random subspace algorithm marked as RSLGSSDR, which utilizes random subspace for semisupervised dimensionality reduction, is proposed. The algorithm first designs multiple diverse graphs on different random subspace of datasets and then fuses these graphs into a mixture graph on which dimensionality reduction is performed. As themixture graph is constructed in lower dimensionality, it can ease the issues on graph construction on highdimensional samples such that it can hold complicated geometric distribution of datasets as the diversity of random subspaces. Experimental results on public gene expression datasets demonstrate that the proposed RSLGSSDR not only has superior recognition performance to competitive methods, but also is robust against a wide range of values of input parameters.

High-throughput experimental techniques produce several kinds of heterogeneous proteomic and genomic data sets. To computationally annotate proteins, it is necessary and promising to integrate these heterogeneous data sources. Some methods transform these data sources into different kernels or feature representations. Next, these kernels are linearly (or nonlinearly) combined into a composite kernel. The composite kernel is utilized to develop a predictive model to infer the function of proteins. A protein can have multiple roles and functions (or labels). Therefore, multilabel learning methods are also adapted for protein function prediction. We develop a transductive multilabel classifier (TMC) to predict multiple functions of proteins using several unlabeled proteins. We also propose a method called transductive multilabel ensemble classifier (TMEC) for integrating the different data sources using an ensemble approach. The TMEC trains a graph-based multilabel classifier on each single data source, and then combines the predictions of the individual classifiers. We use a directed birelational graph to capture the relationships between pairs of proteins, between pairs of functions, and between proteins and functions. We evaluate the effectiveness of the TMC and TMEC to predict the functions of proteins on three benchmarks. We show that our approaches perform better than recently proposed protein function prediction methods on composite and multiple kernels. The code, data sets used in this paper and supplemental material are available at https://sites.google.com/site/guoxian85/tmec.

Advances in biotechnology have made available multitudes of heterogeneous proteomic and genomic data. Integrating these heterogeneous data sources, to automatically infer the function of proteins, is a fundamental challenge in computational biology. Several approaches represent each data source with a kernel (similarity) function. The resulting kernels are then integrated to determine a composite kernel, which is used for developing a function prediction model. Proteins are also found to have multiple roles and functions. As such, several approaches cast the protein function prediction problem within a multi-label learning framework. In our work we develop an approach that takes advantage of several unlabeled proteins, along with multiple data sources and multiple functions of proteins. We develop a graph-based transductive multi-label classifier (TMC) that is evaluated on a composite kernel, and also propose a method for data integration using the ensemble framework, called transductive multi-label ensemble classifier (TMEC). The TMEC approach trains a graph-based multi-label classifier for each individual kernel, and then combines the predictions of the individual models. Our contribution is the use of a bi-relational directed graph that captures relationships between pairs of proteins, between pairs of functions, and between proteins and functions. We evaluate the ability of TMC and TMEC to predict the functions of proteins by using two yeast datasets. We show that our approach performs better than recently proposed protein function prediction methods on composite and multiple kernels.

In order to perform successful diagnosis and treatment of cancer, discovering, and classifying cancer types correctly is essential. One of the challenging properties of class discovery from cancer data sets is that cancer gene expression profiles not only include a large number of genes, but also contains a lot of noisy genes. In order to reduce the effect of noisy genes in cancer gene expression profiles, we propose two new consensus clustering frameworks, named as triple spectral clustering-based consensus clustering (SC3) and double spectral clustering-based consensus clustering (SC2Ncut) in this paper, for cancer discovery from gene expression profiles. SC3 integrates the spectral clustering (SC) algorithm multiple times into the ensemble framework to process gene expression profiles. Specifically, spectral clustering is applied to perform clustering on the gene dimension and the cancer sample dimension, and also used as the consensus function to partition the consensus matrix constructed from multiple clustering solutions.Compared with SC3, SC2Ncut adopts the normalized cut algorithm, instead of spectral clustering, as the consensus function.Experiments on both synthetic data sets and real cancer gene expression profiles illustrate that the proposed approaches not only achieve good performance on gene expression profiles, but also outperforms most of the existing approaches in the process of class discovery from these profiles.

MOTIVATION: RNA 3D motifs are recurrent substructures in an RNA subunit and are building blocks of the RNA architecture. They play an important role in binding proteins and consolidating RNA tertiary structures. RNA 3D motif searching consists of two steps: candidate generation and candidate filtering. We proposed a novel method, known as Feature-based RNA Motif Filtering (FRMF), for identifying motifs based on a set of moment invariants and the Earth Mover's Distance in the second step. RESULTS: A positive set of RNA motifs belonging to six characteristic types, with eight subtypes occurring in HM 50S, is compiled by us. The proposed method is validated on this representative set. FRMF successfully finds most of the positive fragments. Besides the proposed new method and the compiled positive set, we also recognize some new motifs, in particular a π-turn and some non-standard A-minor motifs are found. These newly discovered motifs provide more information about RNA structure conformation. AVAILABILITY: Matlab code can be downloaded from www.cs.cityu.edu.hk/~yingshen/FRMF.html CONTACT: [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Although there are a variety of approaches to solve multiobjective optimization problems, few of them makes systematic use of the neighborhood relationship between the candidate solutions observed during the search process to improve the final results. In this paper, a new evolutionary algorithm, referred to as the neighborhood knowledge-based evolutionary algorithm (NKEA), is proposed to solve the multiobjective optimization problem. NKEA not only takes into account the advantages of NSGA-II, and JGGA, such as the fast nondominated sorting algorithm and horizontal transmission of information in a candidate solution, but also exploits systematically the neighborhood knowledge acquired during the search process. Specifically, NKEA consists of three major stages: the direction learning stage, the mutual adaptation stage, and the self adaptation stage. NKEA not only uses the fast nondominated sorting algorithm to find the Pareto optimal solutions, but also adopts the elitist strategy to maintain the best individuals for the next generation based on this strategy. Two adaptive control functions in the mutual adaptation stage and the self adaptation stage are designed to adjust the respective contributions of coarse local search and fine local search, which allows NKEA to perform a more thorough local search. Finally, we introduce a new notion, known as the measure space, which integrates multiple measures, such as the convergence metric and the diversity metric, to evaluate the performance of the algorithm. The results of our experiments show that NKEA not only achieves good performance in a large number of multiobjective optimization problems, but also outperforms most of the state-of-the-art approaches in these problems.

In this paper, we propose a novel 3-D model retrieval framework, which is referred to as hybrid 3-D model associative retrieval. Unlike the conventional 3-D model similarity retrieval approach, the query model and the models obtained by 3-D model hybrid associative retrieval have the following properties: They belong to different model classes and have different shape characteristics in general but are semantically related and preassembled in a certain associative group. For instance, given a furniture associative group { desk, chair, bed}, we may probably like to use a desk as a query model to search for a list of matching models, which belong to the chair or bed class. We consider the following possibilities: 1) there can be more than two classes in an association group and 2) different association groups might have different numbers of classes. The hybrid associative retrieval is performed in two stages: 1) to establish the relationship between different 3-D model categories with semantic associations, we propose three approaches based on neural network learning and 2) to address the aforementioned two conditions, we use a cyclic-shift scheme to partition different associative groups into two-class pairwise associative groups and then adopt two different strategies to combine the final retrieval results. Experiments by using different data sets demonstrate the effectiveness and efficiency of our proposed framework on the new hybrid associative retrieval task.

In this paper, we propose a new evolutionary algorithm called nearest neighbor evolutionary algorithm (NNE) to solve the unconstrained optimization problem. Specifically, NNE consists of two major steps: coarse nearest neighbor evolutionary and fine nearest neighbor evolutionary. The coarse nearest neighbor evolutionary step pays more attention to searching the optimal solutions in the global way, while the fine nearest neighbor evolutionary step focuses on searching the best solutions in the local way. NNE repeats two major steps until the terminate condition is reached. NNE not only adopts the elitist strategy and maintains the best individuals for the next generation, but also considers the knowledge obtained in the searching process. The experiments demonstrate that (1) NNE achieves good performance in most of numerical optimization problems; (2) NNE outperforms most of state-of-art evolutionary algorithms, such as traditional genetic algorithm (GA), the jumping gene genetic algorithm (JGGA).

Although there exist a lot of approaches to solve constrained optimization problem, few of them makes use of the knowledge obtained in the searching process. In the paper, a new algorithm called nearest neighbor evolutionary algorithm (NNE) is proposed to solve the constrained optimization problem. NNE not only performs global search and local search in the searching process, but also considers the knowledge obtained in the searching process. NNE also avail itself of the elitist strategy and keeps the best individuals for the next generation. The results in the experiments show that NNE not only achieves good performance in a lot of constrained optimization problems, but also outperforms most of state-of-art approaches in most of constrained optimization problems, such as ASCHEA and SEMS.

In this paper, we propose a novel 3D human motion sequence retrieval method based on the similarity of the motion data distribution. First, for each motion sequence in the database, the Self-Organizing Maps (SOM) clustering algorithm is adopted to partition the frames into different classes to get the associated class reference vectors. Then given a query motion, Probabilistic Principal Component Analysis (PPCA) is applied to estimate the distribution of its data. We adopt two different approaches to model the query data. In the first one, we directly estimate the distribution of the original data. For the other one, we estimate the class reference vector’s distribution after training by SOM, instead of that of the original data. Both of these approaches model the data using a Gaussian distribution. Finally the similarity between the query example and the motion sequence in a database is measured using the Mahalanobis distance. Experimental results on the CMU database demonstrate that the proposed method achieves good performance.

A novel approach for 3D motion capture data retrieval based on the Hierarchical Self Organizing Map (HSOM) is proposed. Given a query motion sequence, our goal is to search for all the similar motions from a database. Specifically, a feature vector based on the distribution of the human motion data is first extracted from each motion sequence in the database. Then, Singular Value Decomposition (SVD) is applied to reduce the dimensionality of the feature vector. To improve the retrieval efficiency, a two-level indexing scheme based on the HSOM is constructed, in which the motion sequences are first partitioned with the reduced feature vectors at the top level and then at the lower level, the original feature vectors are adopted to classify the cluster associated with a parent node at the first level into sub-clusters. Finally, fuzzy search is implemented to traverse the index structure to search for similar motions. Experimental results show that our method can achieve good performance in terms of retrieval accuracy and efficiency.

In order to enable more effective image retrieval via keywords, automatic image annotation and categorization becomes an important problem in computer vision and content based image retrieval. Unfortunately, there exists a semantic gap between the low-level feature vectors and the high-level semantics or concepts. In this paper, we design a basic concept repertory to bridge this semantic gap. Specifically, a basic concept repertory, in the form of a dictionary, is first designed to store a set of classifiers, with each of these representing a concept and a set of rules which is used to distinguish concepts with similar characteristics. We propose a new classifier based on our proposed bagging-adaboosting ensemble (BAE) approach. The experiments demonstrate the good performance of our approaches.

Protein phosphorylation is an important step in many biological processes, such as cell cycles, membrane transport, apoptosis, and so on. We design a new classifier ensemble approach called Bagging-Adaboost Ensemble (BAE) for the prediction of eukaryotic protein phosphorylation sites, which incorporates the bagging technique and the adaboost technique into the classifier framework to improve the accuracy, stability and robustness of the final result. To our knowledge, this is the first time in which the ensemble approach is applied to predict phosphorylation sites. Our prediction system based on BAE focuses on five kinase families: CDK, CK2, MAPK, PKA, and PKC. BAE achieves good performance in six families, and the accuracies of the prediction system for these families are 84:7%, 87:4%, 85:5%, 85:2%, and 82:3% respectively.

Recently, researchers are paying more attention to 3D model classification due to its useful applications in multimedia, computer graphics, and so on. Although there exist a number of approaches to classify 3D models, few of them consider the prior knowledge during the process of 3D model classification. In this paper, we propose a new framework called knowledge based cluster ensemble which incorporates the prior knowledge of the dataset into the cluster ensemble framework to classify 3D models. The experiments show that knowledge based cluster ensemble framework works well on 3D human head model database.

In this paper, we propose a new algorithm called fuzzy cluster ensemble algorithm (FCEA) which integrates the fuzzy logic theory and traditional cluster ensembles for 3D head model classification. Specifically, FCEA consists of two parts: (i) data processing on the distributed locations and (ii) data fusion on the centralized location. In the distributed locations, data processing includes (i) extracting feature vectors from 3D head models, (ii) performing basic fuzzy clustering algorithm to obtain fuzzy membership matrix, while data fusion on the centralized location contains (i) creating a fuzzy cluster ensemble constructor by integrating different fuzzy membership matrices from the distributed locations, and (ii) obtaining the final results of 3D head model classification based on the fuzzy logic theory and the fuzzy cluster ensemble constructor. The experiments show that FCEA works well on 3D head model database.

Although there exist a lot of cluster ensemble approaches, few of them consider the prior knowledge of the datasets. In this paper, we propose a new cluster ensemble approach called knowledge based cluster ensemble (KCE) which incorporates the prior knowledge of the dataset into the cluster ensemble framework. Specifically, the prior knowledge of the dataset is first represented by the side information which is encoded as pairwise constraints. Then, KCE generates a set of cluster solutions by the basic clustering algorithm. Next, KCE transforms the pairwise constraints to the confidence factor of the cluster solutions. In the following, the new data matrix is constructed by considering all the cluster solutions and their corresponding confidence factor. Finally, the results are obtained by partitioning the consensus matrix. The experiments illustrate that (1) KCE works well on the real datasets; (2) KCE outperforms most of the state-of-art cluster ensemble approaches.

Due to the rapid development of motion capture technology, more and more human motion databases appear. In order to effectively and efficiently manage human motion database, human motion classification is necessary. In this paper, we propose an Ensemble based Human Motion Classification Approach (EHMCA). Specifically, EHMCA first extracts the descriptors from human motion sequences. Then, singular value decomposition (SVD) is adopted to reduce the dimensionality of all the feature vectors. In the following step, a cluster ensemble approach is designed to construct the consensus matrix from the feature vectors. Finally, the normalized cut algorithm is applied to partition the consensus matrix and assign the feature vectors into the corresponding clusters. Experiments on the CMU database illustrate that the proposed approach achieves good performance.

Pattern mining gains more and more attention due to its useful applications in many areas, such as machine learning, database, multimedia, biology, and so on. Though there exist a lot of approaches for pattern mining, few of them consider the local distribution of the data. In the paper, we not only design six challenge datasets related to the local patterns, but also propose a new pattern mining algorithm based on local distribution. Unlike traditional pattern mining algorithms, our new algorithm first creates a local distribution for each data point by a random approach. Then, the distribution curve of each data point is simulated by the sum of low frequency curves obtained by the wavelet approach. In the third step, the coefficients of these low frequency curves for each data point are clustered by the normalized cut approach. Finally, the patterns of the datasets are obtained by the new pattern mining algorithm. The experiments show that our new algorithm outperforms traditional unsupervised learning approaches, such as K-means, EM, spectral clustering algorithm (SCA), and so on, on these six new datasets.

Support vector machine (SVM) is a hot topic in many areas, such as machine learning, computer vision, data mining, and so on, due to its powerful ability to perform classification. Though there exist a lot of approaches to improve the accuracy and the efficiency of the models of SVM, few of them address how to eliminate the redundant data from the input training vectors. As it is known, most of support vectors distributes in the boundary of the class, which means the vectors in the center of the class are useless. In the paper, we propose a new approach based on Gaussian model to preserve the training vectors in the boundary of the class and eliminate the training vectors in the center of the class. The experiments show that our approach can reduce most of the input training vectors and preserve the support vectors at the same time, which leads to a significant reduction in the computational cost and maintains the accuracy.

3D point based head model classification gains more and more attention due to its useful application in mechanical engineering, computer vision and biology. In this paper, we propose a new approach to perform 3D point based head model classification. Each head model is represented by a Gaussian distribution with the constrained boundary. The constrained boundary is the minimal enclosing ellipsoid of 3D head model. In order to estimate the parameter of the Gaussian distribution, we propose the generalized minimax algorithm. The generalized minimax algorithm for parameter estimation saves a lot of computational cost when comparing with traditional minimax algorithm. The Mahalanobis distance is applied to measure the similarity between two Gaussian distributions with respect to two head models. The experiments show that the new approach works well during the process of 3D point based head model classification.

MOTIVATION: Consensus clustering, also known as cluster ensemble, is one of the important techniques for microarray data analysis, and is particularly useful for class discovery from microarray data. Compared with traditional clustering algorithms, consensus clustering approaches have the ability to integrate multiple partitions from different cluster solutions to improve the robustness, stability, scalability and parallelization of the clustering algorithms. By consensus clustering, one can discover the underlying classes of the samples in gene expression data. RESULTS: In addition to exploring a graph-based consensus clustering (GCC) algorithm to estimate the underlying classes of the samples in microarray data, we also design a new validation index to determine the number of classes in microarray data. To our knowledge, this is the first time in which GCC is applied to class discovery for microarray data. Given a pre specified maximum number of classes (denoted as K(max) in this article), our algorithm can discover the true number of classes for the samples in microarray data according to a new cluster validation index called the Modified Rand Index. Experiments on gene expression data indicate that our new algorithm can (i) outperform most of the existing algorithms, (ii) identify the number of classes correctly in real cancer datasets, and (iii) discover the classes of samples with biological meaning. AVAILABILITY: Matlab source code for the GCC algorithm is available upon request from Zhiwen Yu.

In this paper, a novel 3D head model retrieval approach is proposed, in which only a single 2D face view query is required. The proposed approach will be important for multimedia application areas such as virtual world construction and game design, in which 3D virtual characters with a given set of facial features can be rapidly constructed based on 2D view queries, instead of having to generate each model anew. To achieve this objective, we construct an adaptive mapping through which each 2D view feature vector is associated with its corresponding 3D model feature vector. Given this estimated 3D model feature vector, similarity matching can then be performed in the 3D model feature space. To avoid the explicit specification of the complex relationship between the 2D and 3D feature spaces, a neural network approach is adopted in which the required mapping is implicitly specified through a set of training examples. In addition, for efficient feature representation, principal component analysis (PCA) is adopted to achieve dimensionality reduction for facilitating both the mapping construction and the similarity matching process. Since the linear nature of the original PCA formulation may not be adequate to capture the complex characteristics of 3D models, we also consider the adoption of its nonlinear counterpart, i.e., the so-called kernel PCA approach, in this work. Experimental results show that the proposed approach is capable of successfully retrieving the set of 3D models which are similar in appearance to a given 2D face view.

Recently, the detection of visual attention regions (VAR) is becoming more important due to its useful application in the area of multimedia. Although there exist a lot of approaches to detect visual attention regions, few of them consider the semantic gap between the visual attention regions and high-level semantics. In this paper, we propose a rule based technique for the extraction of visual attention regions at the object level based on real-time clustering, such that VAR detection can be performed in a very efficient way. The proposed technique consists of four stages: 1) a fast segmentation technique which is called the real time clustering algorithm (RTCA); 2) a refined specification of VAR which is known as the hierarchical visual attention regions (HVAR); 3) a new algorithm known as the rule based detection algorithm (RADA) to obtain the set of HVARs in real time, and 4) a new adaptive image display module and the corresponding adaptation operations using HVAR. We also define a new background measure which combines both feature contrast and the geometric property of the region to identify the background region, and a confidence factor which is used to extract the set of hierarchical visual attention regions. Compared with existing techniques, our approach has two advantages: 1) the approach detects the visual attention region at the object level, which bridges the gap between traditional visual attention regions and high-level semantics; 2) our approach is efficient and easy to implement

EM is one of the popular algorithms which can be applied to skin segmentation. Due to the high computational cost of EM, traditional EM is difficult to apply to a large skin database. Inspired by the idea of subsampling, we integrate EM with incremental clustering and hierarchical clustering to estimate the parameters of mixture models. The algorithm first selects the samples by the incremental clustering approach and hierarchical clustering approach. Then, EM is applied to the sample set. The experiments show that the new EM algorithm works well in the skin database.

Given a time horizon parameter h and an object set O, predictive collision detection finds all the object pairsi, oj, ti> which will collide in the future time interval [t, t+h] (where 1lesi, jlesn, tiisin[t, t+h]). Although there are a number of state-of-the-an approaches to solve collision detection problems, predictive collision detection is addressed for the first time. In this paper, we propose a grid-based predictive collision detection algorithm (GPCD), which is a general technique for the efficient detection of the collision of object pairs in a future time interval. GPCD first determines a candidate list which stores the object pairs having a non-zero probability to collide in a future time. Then, GPCD achieves low running time based on two pruning strategies: (i) space intersection test and (ii) time intersection test. These two pruning strategies eliminate most of the false collision cases in an initial filtering phase. In the refinement phase, a bounding-volume tree is applied to refine the detection results. Our experiments show that GPCD works well for the purpose of predictive collision detection

EM algorithm is an important unsupervised clustering algorithm, but the algorithm has several limitations. In this paper, we propose a fast EM algorithm (FEMA) to address the limitations of EM and enhance its efficiency. FEMA achieves low running time by combining principal component analysis (PCA), a grid cell expansion algorithm (GCEA) and a hierarchical cluster tree. PCA and multi-dimensional grid are applied to find a set of "good" initial parameters for the EM algorithm, while the hierarchical cluster tree deals with the case where the cluster is concave by making use of a merging algorithm. The experiments indicate that FEMA outperforms EM by reducing 45% of the CPU time

Given a point-based 3D computer graphics model which is defined by a point set P(P={piisinR3}) and a desired reduced number of output samples Ns, the simplification approach finds a point set Pswhich (i) satisfies |Ps|=Ns(|Ps| is the cardinality of Ps) and (ii) minimizes the difference of the corresponding surface Ss(defined by Ps) and the original surface S(defined by P). Although a number of previous approaches have been proposed for simplification, most of them (i) do not focus on point-based 3D models, (ii) do not consider efficiency, quality and generality together. In this paper, we introduce an adaptive simplification method (ASM) which is an efficient technique for simplifying point-based complex 3D model. ASM achieves low running time by clustering the points locally based on the preservation of geometric characteristics. Finally, we analyze the performance of ASM and show that it outperforms most of the current state-of-the-art methods in terms of efficiency, quality and generality

In this paper, we propose a new approach for similarity matching in image retrieval based on approximate query processing in a hierarchical SOM. We first map the high dimensional input vectors to a low dimensional grid by a local membership function which preserves the relationships between the input vectors and their neighboring weight vectors. Then, we use a hierarchical tree to reduce the computation cost for finding the best match unit. Finally, we retrieve the k nearest neighbors of the query vector by an approximate query processing approach. The experiments show that the proposed approach works well on both synthetic datasets and image databases.

In this paper, we introduce a novel shape signature, called Volumetric Extended Gaussian Image (VEGI). It captures the volumetric distribution of a 3D mesh model along the latitude-longitude direction without conventional pose normalization and is translation and scaling invariant. Rotation invariance is accomplished by further calculating the spherical harmonic transform of this directional distribution. Due to the completeness and orthonormality properties of spherical harmonics, the VEGI also provides multi-resolution description of a model so that a multi-level indexing scheme based on Hierarchical Self Organizing Map (HSOM) can be established to improve retrieval efficiency. Experimental results show that our retrieval architecture has high discriminative power and outperforms many existing methods.

Estimating the selectivity of range query when applied to image database is an important stage for optimizing a image database query. In this paper, we propose a new approach to approximate the selectivity of a range query based on a probabilistic framework. Specifically, each image is represented as a sequence of states in a multi-dimensional space, where each dimension corresponds to a particular time stamp. We then associate a histogram with each dimension, and the histogram bins in each dimension correspond to specific states. A range query in the proposed probabilistic framework is the sum of the probabilities of observing a sequence of states within a hyper-sphere. A minimum bounding box and a maximum enclosed box are used to determine the upper bound and the lower bound of the selectivity of the range query. The experiments show that this probabilistic framework works well for an image database.