Unsupervised cell type identification is crucial for uncovering and characterizing heterogeneous populations in single cell omics studies. Although a range of clustering methods have been developed, most focus exclusively on intrinsic cellular structure and ignore the pivotal role of cell-gene associations, which limits their ability to distinguish closely related cell types. To this end, we propose a Refinement Contrastive Learning framework (scRCL) that explicitly incorporates cell-gene interactions to derive more informative representations. Specifically, we introduce two contrastive distribution alignment components that reveal reliable intrinsic cellular structures by effectively exploiting cell-cell structural relationships. Additionally, we develop a refinement module that integrates gene-correlation structure learning to enhance cell embeddings by capturing underlying cell-gene associations. This module strengthens connections between cells and their associated genes, refining the representation learning to exploiting biologically meaningful relationships. Extensive experiments on several single-cell RNA-seq and spatial transcriptomics benchmark datasets demonstrate that our method consistently outperforms state-of-the-art baselines in cell-type identification accuracy. Moreover, downstream biological analyses confirm that the recovered cell populations exhibit coherent gene-expression signatures, further validating the biological relevance of our approach.

MOTIVATION: For spot-based spatial transcriptomics (ST), accurate cell-type deconvolution is essential for downstream analysis since each spot captures mixtures of multiple cell types. Meanwhile, spatial niches define distinct micro-environmental contexts, also salient for biological interpretation. However, existing deconvolution methods usually rely on fixed reference signatures or mapping single cells onto ST spots, without incorporating niche priors or modeling niche-dependent shifts. Consequently, existing methods remain focused on spot-level proportion estimation, with limited ability to support functional analysis of niche-associated molecular programs. RESULTS: We present NicheDeSig for niche-aware deconvolution. NicheDeSig models each cell type through adaptive signatures, enabling spot deconvolution under context-dependent signatures and supporting niche-aware analysis of cell-state variation across spatial micro-environments. Our method achieves strong deconvolution performance across the simulated benchmark datasets and improves spatial fidelity in the simulated colon dataset. The learned signatures recover laminar and white-matter-associated programs in the human dorsolateral prefrontal cortex (DLPFC), domain-stratified tumor microenvironment patterns in breast cancer (BRCA), and region-associated signatures in pancreatic ductal adenocarcinoma sample A (PDAC-A) and colorectal liver metastasis analyses. AVAILABILITY AND IMPLEMENTATION: Source code and the archived code snapshot are available at https://github.com/Davidcoach/NicheDeSig and https://doi.org/10.5281/zenodo.20685597.

Multi-view clustering has been empirically shown to improve learning performance by leveraging the inherent complementary information across multiple views of data. However, in real-world scenarios, collecting strictly aligned views is challenging, and learning from both aligned and unaligned data becomes a more practical solution. Partially View-aligned Clustering (PVC) aims to learn correspondences between misaligned view samples to better exploit the potential consistency and complementarity across views, including both aligned and unaligned data. However, most existing PVC methods fail to leverage unaligned data to capture the shared semantics among samples from the same cluster. Moreover, the inherent heterogeneity of multi-view data induces distributional shifts in representations, leading to inaccuracies in establishing meaningful correspondences between cross-view latent features and, consequently, impairing learning effectiveness. To address these challenges, we propose a Semantic MAtching contRasTive learning model (SMART) for PVC. The main idea of our approach is to alleviate the influence of cross-view distributional shifts, thereby facilitating semantic matching contrastive learning to fully exploit semantic relationships in both aligned and unaligned data. Specifically, we mitigate view distribution shifts by aligning cross-view covariance matrices, which enables the inference of a semantic graph for all data. Guided by the learned semantic graph, we further exploit semantic consistency across views through semantic matching contrastive learning. After the optimization of the above mechanisms, our model smoothly performs semantic matching for different view embeddings instead of the cumbersome view realignment, which enables the learned representations to enjoy richer category-level semantics and stronger robustness. Extensive experiments on eight benchmark datasets demonstrate that our method consistently outperforms existing approaches on the PVC problem. The code is available at https://github.com/THPengL/SMART.

Text-to-image (T2I) diffusion models aim to generate images that are both visually realistic and aligned with open-domain textual prompts. However, the leading T2I models often fall short in capturing semantic details, especially for fine-grained object categories. We find that simply expanding original prompts cannot effectively guide the model to generate the desired content, since T2I diffusion models pre-trained on generic datasets lack a mechanism to boost class semantics. In this work, we present ClassBooth, a flexible framework that improves pre-trained T2I diffusion model in rendering class-specific content, while preserving the open-domain generation capability. Toward this end, we introduce an auxiliary class-specific semantic booster conditioned on learnable prompts, which are associated with specific classes to encode fine-grained conditioning information. To enable the T2I model to synthesize fine-grained details of specific categories, we perform bidirectional fusion on the features conditioned on different information, and this design is beneficial for class-specific semantic expression, thereby synthesizing high-fidelity data encapsulating precise class-specific details. We validate ClassBooth across multiple benchmarks, demonstrating its superiority over existing methods through comprehensive quantitative and qualitative evaluations.

The advent of Spatial Transcriptomics (ST) has revolutionized understanding of tissue architecture by creating high-resolution maps of gene expression patterns. However, the low capture rate of ST leads to significant sparsity. The aim of imputation is to recover biological signals by imputing the dropouts in ST data to approximate the true expression values. In this paper, we introduce a Spatial Gene Expression Imputation Diffusion model to facilitate ST data imputation, and our model is referred to as SpotDiff. Specifically, we incorporate a spot-gene prompt learning module to capture the association between spots and genes. Further, SpotDiff integrates single-cell RNA sequencing data to impute gene expression at each spot. The proposed approach is able to reduce the uncertainty in the imputation process, since the aggregation of multiple single-cell measurements yield a stable representation of the corresponding spot expression profile. Extensive experiments have been performed to demonstrate that SpotDiff outperforms existing imputation methods across multiple benchmarks in terms of yielding more accurate and biologically relevant gene expression profiles, particularly in highly sparse scenarios.

Blind face video restoration aims to restore high-fidelity details from videos subjected to complex and unknown degradations. This task poses a significant challenge of managing temporal heterogeneity while at the same time maintaining stable face attributes. In this paper, we introduce a Discrete Prior-based Temporal-Coherent content prediction transformer to address the challenge, and our model is referred to as DP-TempCoh. Specifically, we incorporate a spatial-temporal-aware content prediction module to synthesize high-quality content from discrete visual priors, conditioned on degraded video tokens. To further enhance the temporal coherence of the predicted content, a motion statistics modulation module is designed to adjust the content, based on discrete motion priors in terms of cross-frame mean and variance. As a result, the statistics of the predicted content can match with that of real videos over time. By performing extensive experiments, we verify the effectiveness of the design elements and demonstrate the superior performance of our DP-TempCoh in both synthetically and naturally degraded video restoration.

Face retouching aims to remove facial imperfections from image and videos while at the same time preserving face attributes. The existing methods are designed to perform non-interactive end-to-end retouching, while the ability to interact with users is highly demanded in downstream applications. In this paper, we propose RetouchGPT, a novel framework that leverages Large Language Models (LLMs) to guide the interactive retouching process. Towards this end, we design an instruction-driven imperfection prediction module to accurately identify imperfections by integrating textual and visual features. To learn imperfection prompts, we further incorporate a LLM-based embedding module to fuse multi-modal conditioning information. The prompt-based feature modification is performed in each transformer block, such that the imperfection features are suppressed and replaced with the features of normal skin progressively. Extensive experiments have been performed to verify effectiveness of our design elements and demonstrate that RetouchGPT is a useful tool for interactive face retouching and achieves superior performance over state-of-the-arts.

The goal of visual grounding is to establish connections between target objects and textual descriptions. Large Language Models (LLMs) have demonstrated strong comprehension abilities across a variety of visual tasks. To establish precise associations between the text and the corresponding visual region, we propose a Task-aware Crossmodal feature Refinement Transformer with LLMs for visual grounding, and our model is referred to as TCRT. To enable the LLM trained solely on text to understand images, we introduce an LLM adaptation module that extracts textrelated visual features to bridge the domain discrepancy between the textual and visual modalities. We feed the text and visual features into the LLM to obtain task-aware priors. To enable the priors to guide the feature fusion process, we further incorporate a cross-modal feature fusion module, which allows task-aware embeddings to refine visual features and facilitate information interaction between the Referring Expression Comprehension (REC) and Referring Expression Segmentation (RES) tasks. We have performed extensive experiments to verify the effectiveness of the main components and demonstrate the superior performance of the proposed TCRT over state-of-the-art end-to-end visual grounding methods on RefCOCO, RefCOCOg, RefCOCO+ and ReferItGame.

Blind Face Video Restoration (BFVR) focuses on reconstructing high-quality facial image sequences from degraded video inputs. The main challenge is address unknown degradations, while maintaining temporal consistency across frames. Current blind face restoration methods are primarily designed for images, and directly applying these approaches to BFVR will encounter a significant drop in restoration performance. In this work, we proposed Dynamic Content Prediction with Motion-aware Priors, referred to as DCP-MP. We develop a motion-aware semantic dictionary by encoding the semantic information of high-quality videos into discrete elements, and capturing the motion information in terms of element relationships, which are derived from the dynamic temporal changes within videos. For the purpose of utilizing dictionary to represent the degraded video, we train a temporal-aware element predictor, conditioned on degraded content, to learn the prediction of discrete elements in dictionary. The predicted elements will be refined, conditioned on motion information captured by the motion-aware semantic dictionary, to enhance temporal coherence. To alleviate deviation from the original structure information, we propose a conditional structure feature correction module that corrects the features flowing from the encoder to the generator. Through extensive experiments, we validate the effectiveness of our design components and demonstrate the superior performance of DCP-MP in synthesizing high-quality video.

Pre-trained language models (PrLMs) demonstrate impressive performance on the sentiment analysis task. However, the large number of trainable parameters brings about heavy computational costs, which become more serious in multi-domain scenarios. In this paper, we propose to extract multi-layer features from the PrLM for efficient training since the training process is independent to its large backbone. Meanwhile, compared with the conventional feature extraction, we leverage prompts to induce PrLM for generating sentiment-aware features which lead to significant improvement on the sentiment analysis. In addition, most previous methods adopted a domain alignment paradigm for multi-domain learning, which becomes cumbersome when the number of domains is large. Therefore, we propose a novel prompt-augmented cross-domain contrastive learning for generalizable performance, which clusters samples with the same label under different prompts or domains. Our method is evaluated on two public multi-domain sentiment analysis benchmarks, which significantly outperforms recent state-of-the-art methods. Extensive ablation studies also verify the effectiveness of each proposed component.

Multi-view clustering (MVC) has shown that leveraging both consistency and complementary information across views enhances clustering performance. However, most existing methods focus on aligning features into the same dimension, often neglecting cross-view heterogeneity and introducing discrepancies. To address this, we propose a novel multi-view clustering framework that combines cross-view neighborhood contrastive learning with a cross-attention view-mixup feature learning mechanism. Specifically, the cross-attention view-mixup module learns view-invariant feature representations by capturing complementary and consistent information, while the neighborhood contrastive learning module uncovers semantic structures across views based on the learned mixup features. By implicitly performing feature mixup across views and effectively integrating cross-view neighborhood contrastive learning, our method alleviates cross-view discrepancies and enables more effective integration of complementary and consistent information, ultimately enhancing clustering performance. Experiments conducted on several real datasets demonstrate the effectiveness of our proposed method in comparision with several representative MVC approaches.

In-context learning (ICL) has garnered significant attention for its ability to grasp functions/tasks from demonstrations. Recent studies suggest the presence of a latent task/function vector in LLMs during ICL. Merullo et al. (2024) showed that LLMs leverage this vector alongside the residual stream for Word2Vec-like vector arithmetic, solving factual-recall ICL tasks. Additionally, recent work empirically highlighted the key role of Question-Answer data in enhancing factual-recall capabilities. Despite these insights, a theoretical explanation remains elusive. To move one step forward, we propose a theoretical framework building on empirically grounded hierarchical concept modeling. We develop an optimization theory, showing how nonlinear residual transformers trained via gradient descent on cross-entropy loss perform factual-recall ICL tasks via vector arithmetic. We prove 0-1 loss convergence and show the strong generalization, including robustness to concept recombination and distribution shifts. These results elucidate the advantages of transformers over static embedding predecessors. Empirical simulations corroborate our theoretical insights.

MOTIVATION: Single-cell RNA sequencing (scRNA-seq) enables high-throughput transcriptomic profiling at single-cell resolution. The inherent spatial location is crucial for understanding how single cells orchestrate multicellular functions and drive diseases. However, spatial information is often lost during tissue dissociation. Spatial transcriptomic (ST) technologies can provide precise spatial gene expression atlas, while their practicality is constrained by the number of genes they can assay or the associated costs at a larger scale and the fine-grained cell-type annotation. By transferring knowledge between scRNA-seq and ST data through cell correspondence learning, it is possible to recover the spatial properties inherent in scRNA-seq datasets. RESULTS: In this study, we introduce COME, a COntrastive Mapping lEarning approach that learns mapping between ST and scRNA-seq data to recover the spatial information of scRNA-seq data. Extensive experiments demonstrate that the proposed COME method effectively captures precise cell-spot relationships and outperforms previous methods in recovering spatial location for scRNA-seq data. More importantly, our method is capable of precisely identifying biologically meaningful information within the data, such as the spatial structure of missing genes, spatial hierarchical patterns, and the cell-type compositions for each spot. These results indicate that the proposed COME method can help to understand the heterogeneity and activities among cells within tissue environments. AVAILABILITY AND IMPLEMENTATION: The COME is freely available in GitHub (https://github.com/cindyway/COME).

Single-cell RNA sequencing (scRNA-seq) technology has transformed gene expression studies by enabling analysis at the individual cell level, offering unprecedented insights into cellular heterogeneity. A key challenge in scRNA-seq data analysis is cell type identification, which requires grouping cells with similar gene expression profiles using unsupervised clustering methods. However, the high dimensionality, inherent noise, and significant sparsity of scRNA-seq data present substantial obstacles to accurately determining relationships among cell samples. To address these challenges, we propose a novel deep subspace clustering approach for cell type identification that captures a more reliable subspace structure from scRNA-seq data. Our method leverages a robust self-representation learning framework to effectively characterize and learn the underlying cluster structure. This framework is optimized through an integrated strategy combining a structure-guided approach with an optimal transport algorithm, enhancing the robustness of the subspace clustering process. By mitigating the effects of noise and sparsity in scRNA-seq data, this approach enables more accurate cell clustering. Experimental results on 18 real scRNA-seq datasets demonstrate that our method outperforms several state-of-the-art clustering approaches tailored for scRNA-seq data, excelling in both accuracy and interpretability.

Typically, deep network-based full-reference image quality assessment (FR-IQA) models compare deep features from reference and distorted images pairwise, overlooking correlations among features from the same source. We propose a dual-branch framework to capture the joint degradation effect among deep network features. The first branch uses kernel representation similarity analysis (KRSA), which compares feature self-similarity matrices via the mean absolute error (MAE). The second branch conducts pairwise comparisons via the MAE, and a training-free logarithmic summation of both branches derives the final score. Our approach contributes in three ways. First, integrating the KRSA with pairwise comparisons enhances the model's perceptual awareness. Second, our approach is adaptable to diverse network architectures. Third, our approach can guide perceptual image enhancement. Extensive experiments on 10 datasets validate our method's efficacy, demonstrating that perceptual deformation widely exists in diverse IQA scenarios and that measuring the joint degradation effect can discern appealing content deformations.

Ensemble learning improves the capability of convolutional neural network (CNN)-based discriminators, whose performance is crucial to the quality of generated samples in generative adversarial network (GAN). However, this learning strategy results in a significant increase in the number of parameters along with computational overhead. Meanwhile, the suitable number of discriminators required to enhance GAN performance is still being investigated. To mitigate these issues, we propose an evidential discriminator for GAN (EviD-GAN)-code is available at https://github.com/Tohokantche/EviD-GAN-to learn both the model (epistemic) and data (aleatoric) uncertainties. Specifically, by analyzing three GAN models, the relation between the distribution of discriminator's output and the generator performance has been discovered yielding a general formulation of GAN framework. With the above analysis, the evidential discriminator learns the degree of aleatoric and epistemic uncertainties via imposing a higher order distribution constraint over the likelihood as expressed in the discriminator's output. This constraint can learn an ensemble of likelihood functions corresponding to an infinite set of discriminators. Thus, EviD-GAN aggregates knowledge through the ensemble learning of discriminator that allows the generator to benefit from an informative gradient flow at a negligible computational cost. Furthermore, inspired by the gradient direction in maximum mean discrepancy (MMD)-repulsive GAN, we design an asymmetric regularization scheme for EviD-GAN. Unlike MMD-repulsive GAN that performs at the distribution level, our regularization scheme is based on a pairwise loss function, performs at the sample level, and is characterized by an asymmetric behavior during the training of generator and discriminator. Experimental results show that the proposed evidential discriminator is cost-effective, consistently improves GAN in terms of Frechet inception distance (FID) and inception score (IS), and performs better than other competing models that use multiple discriminators.

Graph-based multiview clustering (MVC) approaches have demonstrated impressive performance by leveraging the consistency properties of multiview data in an unsupervised manner. However, existing methods for graph learning heavily rely on either Euclidean structures or the manifold topological structures derived from fixed view-specific graphs. Unfortunately, these approaches may not accurately reflect the consensus topological structure in a multiview setting. To address this limitation and enhance the intrinsic graph learning process, an adaptive exploration of a more appropriate consistency topological structure is required. Toward this end, we propose a novel approach called collaborative topological graph learning (CTGL) for MVC. The key idea is to adaptively discover the consistent topological structure to guide intrinsic graph learning. We achieve this by introducing an auxiliary consistency graph that formulates the topological relevance learning function. However, estimating the auxiliary consistency graph is not straightforward, as it is based on the learned view-specific graphs and requires prior availability. To overcome this challenge, we develop a collaborative learning strategy that simultaneously learns both the auxiliary consistency graph and view-specific graphs using tensor learning techniques. This strategy enables the adaptive exploration of the consistency topological structure during graph learning, resulting in more accurate clustering outcomes. Extensive experiments are provided to show the effectiveness of the proposed method. The source code can be found at https://github.com/CLiu272/CTGL.

Face retouching is to beautify a face image, while preserving the image content as much as possible. It is a promising yet challenging task to remove face imperfections and fill with normal skin. Generic image enhancement methods are hampered by the lack of imperfection localization, which often results in incomplete removal of blemishes at large scales. To address this issue, we propose a transformer-based approach, RetouchFormer, which simultaneously identify imperfections and synthesize realistic content in the corresponding regions. Specifically, we learn a latent dictionary to capture the clean face priors, and predict the imperfection regions via a reconstruction-oriented localization module. Also based on this, we can realize face retouching by explicitly suppressing imperfections in our selective self-attention computation, such that local content will be synthesized from normal skin. On the other hand, multi-scale feature tokens lead to increased flexibility in dealing with the imperfections at various scales. The design elements bring greater effectiveness and efficiency. RetouchFormer outperforms the advanced face retouching methods and synthesizes clean face images with high fidelity in our list of extensive experiments performed.

Oriented object detection has witnessed significant progress in recent years. However, the impressive performance of oriented object detectors is at the huge cost of labor-intensive annotations, and deteriorates once the an-notated data becomes limited. Semi-supervised learning, in which sufficient unannotated data are utilized to enhance the base detector, is a promising method to address the annotation deficiency problem. Motivated by weakly supervised learning, we introduce annotation-efficient point annotations for unannotated images and propose a weakly semi-supervised method for oriented object detection to balance the detection performance and annotation cost. Specifically, we propose a Rotation-Modulated Relational Graph Matching method to match relations of proposals centered on an-notated points between the teacher and student models to alleviate the ambiguity of point annotations in depicting the oriented object. In addition, we further propose a Relational Rank Distribution Matching method to align the rank distribution on classification and regression between different models. Finally, to handle the difficult annotated points that both models are confused about, we introduce weakly supervised learning to impose positive signals for difficult point-induced clusters to the base model, and focus the base model on the occupancy between the predictions and an-notated points. We perform extensive experiments on chal-lenging datasets to demonstrate the effectiveness of our proposed weakly semi-supervised method in leveraging point-annotated data for significant performance improvement.

Blind face restoration focuses on restoring high-fidelity details from images subjected to complex and unknown degradations, while preserving identity information. In this paper, we present a Prior-based Latent Transformation approach (PLTrans), which is specifically designed to learn a degradation-unaware representation, thereby allowing the restoration network to effectively generalize to real-world degradation. Toward this end, PLTrans learns a degradation-unaware query via a latent diffusion-based regularization module. Furthermore, conditioned on the features of a degraded face image, a latent dictionary that captures the priors of HQ face images is leveraged to refine the features by mapping the top-d nearest elements. The refined version will be used to build key and value for the cross-attention computation, which is tailored to each degraded image and exhibits reduced sensitivity to different degradation factors. Conditioned on the resulting representation, we train a decoding network that synthesizes face images with authentic details and identity preservation. Through extensive experiments, we verify the effectiveness of the design elements and demonstrate the generalization ability of our proposed approach for both synthetic and unknown degradations. We finally demonstrate the applicability of PLTrans in other vision tasks.

With the advent of generative models and vision-language pre-training, significant improvement has been made in text-driven face manipulation. The text embedding can be used as target supervision for expression control. However, it is non-trivial to associate with its 3D attributes, i.e., pose and illumination. To address these issues, we propose a Text-conditional Attribute aLignment approach for 3D controllable face image synthesis, and our model is referred to as TcALign. Specifically, since the 3D rendered image can be precisely controlled with the 3D face representation, we first propose a Text-conditional 3D Editor to produce the target face representation to realize text-driven manipulation in the 3D space. An attribute embedding space spanned by the target-related attributes embeddings is also introduced to infer the disentangled task-specific direction. Next, we train a cross-modal latent mapping network conditioned on the derived difference of 3D representation to infer a correct vector in the latent space of Style-GAN. This correction vector learning design can accurately transfer the attribute manipulation on 3D images to 2D images. We show that the proposed method delivers more precise text-driven multi-attribute manipulation for 3D controllable face image synthesis. Extensive qualitative and quantitative experiments verify the effectiveness and superiority of our method over the other competing methods.

Face Video Retouching is a complex task that often requires labor-intensive manual editing. Conventional image retouching methods perform less satisfactorily in terms of generalization performance and stability when applied to videos without exploiting the correlation among frames. To address this issue, we propose a Video Retouching transformEr to remove facial imperfections in videos, which is referred to as VRetouchEr. Specifically, we estimate the apparent motion of imperfections between two consecutive frames, and the resulting displacement vectors are used to refine the imperfection map, which is synthesized from the current frame together with the corresponding encoder features. The flow-based imperfection refinement is critical for precise and stable retouching across frames. To leverage the temporal contextual information, we inject the refined imperfection map into each transformer block for multi-frame masked attention computation, such that we can capture the interdependence between the current frame and multiple reference frames. As a result, the imperfection regions can be replaced with normal skin with high fidelity, while at the same time keeping the other regions unchanged. Extensive experiments are performed to verify the superiority of VRetouchEr over state-of-the-art image retouching methods in terms of fidelity and stability.

Neural Network-based active learning (NAL) is a cost-effective data selection technique that utilizes neural networks to select and train on a small subset of samples. While existing work successfully develops various effective or theory-justified NAL algorithms, the understanding of the two commonly used query criteria of NAL: uncertainty-based and diversity-based, remains in its infancy. In this work, we try to move one step forward by offering a unified explanation for the success of both query criteria-based NAL from a feature learning view. Specifically, we consider a feature-noise data model comprising easy-to-learn or hard-to-learn features disrupted by noise, and conduct analysis over 2-layer NN-based NALs in the pool-based scenario. We provably show that both uncertainty-based and diversity-based NAL are inherently amenable to one and the same principle, i.e., striving to prioritize samples that contain yet-to-be-learned features. We further prove that this shared principle is the key to their success-achieve small test error within a small labeled set. Contrastingly, the strategy-free passive learning exhibits a large test error due to the inadequate learning of yet-to-be-learned features, necessitating resort to a significantly larger label complexity for a sufficient test error reduction. Experimental results validate our findings.

The prevalence of multimedia applications has led to increased concerns and demand for auto face retouching. Face retouching aims to enhance portrait quality by removing blemishes. However, the existing auto-retouching methods rely heavily on a large amount of paired training samples, and perform less satisfactorily when handling complex and unusual blemishes. To address this issue, we propose a Language-guided Blemish Removal Transformer for automatically retouching face images, while at the same time reducing the dependency of the model on paired training data. Our model is referred to as LangBRT, which leverages vision-language pre-training for precise facial blemish removal. Specifically, we design a text-prompted blemish detection module that indicates the regions to be edited. The priors not only enable the transformer network to handle specific blemishes in certain areas, but also reduce the reliance on retouching training data. Further, we adopt a target-aware cross attention mechanism, such that the blemish-like regions are edited accurately while at the same time maintaining the normal skin regions unchanged. Finally, we adopt a regularization approach to encourage the semantic consistency between the synthesized image and the text description of the desired retouching outcome. Extensive experiments are performed to demonstrate the superior performance of LangBRT over competing auto-retouching methods in terms of dependency on training data, blemish detection accuracy and synthesis quality.

Partially View-aligned Clustering (PVC) presents a challenge as it requires a comprehensive exploration of complementary and consistent information in the presence of partial alignment of view data. Existing PVC methods typically learn view correspondence based on latent features that are expected to contain common semantic information. However, latent features obtained from heterogeneous spaces, along with the enforcement of alignment into the same feature dimension, can introduce cross-view discrepancies. In particular, partially view-aligned data lacks sufficient shared correspondences for the critical common semantic feature learning, resulting in inaccuracies in establishing meaningful correspondences between latent features across different views. While feature representations may differ across views, instance relationships within each view could potentially encode consistent common semantics across views. Motivated by this, our aim is to learn view correspondence based on graph distribution metrics that capture semantic view-invariant instance relationships. To achieve this, we utilize similarity graphs to depict instance relationships and learn view correspondence by aligning semantic similarity graphs through optimal transport with graph distribution. This facilitates the precise learning of view alignments, even in the presence of heterogeneous view-specific feature distortions. Furthermore, leveraging well-established cross-view correspondence, we introduce a cross-view contrastive learning to learn semantic features by exploiting consistency information. The resulting meaningful semantic features effectively isolate shared latent patterns, avoiding the inclusion of irrelevant private information. We conduct extensive experiments on several real datasets, demonstrating the effectiveness of our proposed method for the PVC task.

Transformer-based large language models (LLMs) have displayed remarkable creative prowess and emergence capabilities. Existing empirical studies have revealed a strong connection between these LLMs' impressive emergence abilities and their in-context learning (ICL) capacity, allowing them to solve new tasks using only task-specific prompts without further fine-tuning. On the other hand, existing empirical and theoretical studies also show that there is a linear regularity of the multi-concept encoded semantic representation behind transformer-based LLMs. However, existing theoretical work fail to build up an understanding of the connection between this regularity and the innovative power of ICL. Additionally, prior work often focuses on simplified, unrealistic scenarios involving linear transformers or unrealistic loss functions, and they achieve only linear or sub-linear convergence rates. In contrast, this work provides a fine-grained mathematical analysis to show how transformers leverage the multi-concept semantics of words to enable powerful ICL and excellent out-of-distribution ICL abilities, offering insights into how transformers innovate solutions for certain unseen tasks encoded with multiple cross-concept semantics. Inspired by empirical studies on the linear latent geometry of LLMs, the analysis is based on a concept-based low-noise sparse coding prompt model. Leveraging advanced techniques, this work showcases the exponential 0-1 loss convergence over the highly non-convex training dynamics, which pioneeringly incorporates the challenges of softmax self-attention, ReLU-activated MLPs, and cross-entropy loss. Empirical simulations corroborate the theoretical findings.

The spatial reconstruction of single-cell RNA sequencing (scRNA-seq) data into spatial transcriptomics (ST) is a rapidly evolving field that addresses the significant challenge of aligning gene expression profiles to their spatial origins within tissues. This task is complicated by the inherent batch effects and the need for precise gene expression characterization to accurately reflect spatial information. To address these challenges, we developed SELF-Former, a transformer-based framework that utilizes multi-scale structures to learn gene representations, while designing spatial correlation constraints for the reconstruction of corresponding ST data. SELF-Former excels in recovering the spatial information of ST data and effectively mitigates batch effects between scRNA-seq and ST data. A novel aspect of SELF-Former is the introduction of a gene filtration module, which significantly enhances the spatial reconstruction task by selecting genes that are crucial for accurate spatial positioning and reconstruction. The superior performance and effectiveness of SELF-Former's modules have been validated across four benchmark datasets, establishing it as a robust and effective method for spatial reconstruction tasks. SELF-Former demonstrates its capability to extract meaningful gene expression information from scRNA-seq data and accurately map it to the spatial context of real ST data. Our method represents a significant advancement in the field, offering a reliable approach for spatial reconstruction.

Adopting deep learning in early fundus screening images benefits ocular disease recognition and helps patients avoid blindness in recent years. The robust data representation capability of deep learning is derived from numerous data and annotations. However, the fundus images collected from hospitals or institutes have privacy issues and obvious domain gaps, which greatly influence multi-site learning performance. In this work, a learning system with differential privacy and unsupervised domain regularizer is proposed for ocular disease recognition. First, a Laplace randomized mechanism is introduced to keep the privacy of local models and a global model is constructed via a weighted sum process. Second, an unsupervised domain regularizer, which converts the last fully-connected layer into two sub-layers and then adopts an MMD loss in the element-wise layers of source and target domains, is proposed for unsupervised domain adaptation. Numerous experiments, including four different settings, verify the performance in a multi-disease ocular dataset.

Oriented object detection, which aims at detecting objects with orientation property, shows great potential for visual analysis in complex scenarios, such as aerial images. However, the powerful detection performance relies on abundant and accurate annotations, and deteriorates once the annotations become insufficient. Semi-supervised learning, which utilizes unannotated data to improve the target model, is a promising method to address the problem of annotation deficiency. In this work, we propose Pseudo-Siamese Teacher (PST), a new semi-supervised learning framework for oriented object detection. In this architecture, two teacher models, updated from the same student model with different optimizations, inspect the predictions of each other and collaborate to generate high-quality pseudo annotations. To reduce the unreliability of pseudo annotations on the localization, scale and orientation, we propose to model the oriented object as a Gaussian distribution, and apply a symmetric and bounded Jensen–Shannon divergence (JSD) to evaluate the divergence between predictions of different teacher models, the results of which serve as an indicator to remove confusing pseudo annotations without consistent regression estimation of teacher models. Scale invariance is also an important challenge in oriented object detection, which we address by proposing a scale-adaptive knowledge distillation to align information between the feature maps from the student model on images with flexible scales and the feature maps, interpolated from adjacent feature maps with scales closest to that of the down-sampled images, from the teacher models. We perform extensive experiments to demonstrate the effectiveness of our proposed method in leveraging unannotated data for performance improvement.

As one of the effective ways of ocular disease recognition, early fundus screening can help patients avoid unrecoverable blindness. Although deep learning is powerful for image-based ocular disease recognition, the performance mainly benefits from a large number of labeled data. For ocular disease, data collection and annotation in a single site usually take a lot of time. If multi-site data are obtained, there are two main issues: 1) the data privacy is easy to be leaked; 2) the domain gap among sites will influence the recognition performance. Inspired by the above, first, a Gaussian randomized mechanism is adopted in local sites, which are then engaged in a global model to preserve the data privacy of local sites and models. Second, to bridge the domain gap among different sites, a two-step domain adaptation method is introduced, which consists of a domain confusion module and a multi-expert learning strategy. Based on the above, a privacy-preserving federated learning framework with domain adaptation is constructed. In the experimental part, a multi-disease early fundus screening dataset, including a detailed ablation study and four experimental settings, is used to show the stepwise performance, which verifies the efficiency of our proposed framework.

Incomplete multi-view clustering (IMVC) presents a significant challenge due to the need for effectively exploring complementary and consistent information within the context of missing views. One promising strategy to tackle this challenge is to recover missing views by inferring the missing samples. However, such approaches often fail to fully utilize discriminative structural information or adequately address consistency, as it requires such information to be known or learnable in advance, which contradicts the incomplete data setting. In this study, we propose a novel approach calledLatentStructure-Aware view recovery (LaSA) for the IMVC task. Our objective is to recover missing views through discriminative latent representations by leveraging structural information. Specifically, our method offers a unified closed-form formulation that simultaneously performs missing data inference and latent representation learning, using a learned intrinsic graph as structural information. This formulation, incorporating graph structure information, enhances the inference of missing data while facilitating discriminative feature learning. Even when intrinsic graph is initially unknown due to incomplete data, our formulation allows for effective view recovery and intrinsic graph learning through an iterative optimization process. To further enhance performance, we introduce an iterative consistency diffusion process, which effectively leverages the consistency and complementary information across multiple views. Extensive experiments demonstrate the effectiveness of the proposed method compared to state-of-the-art approaches.

Multi-attribute editing aims to synthesize new facial images with multiple desired attributes while at the same time preserving other contents. Generative Adversarial Networks (GANs) with encoder-decoder-based generators are typically applied to this task, while the co-occurrence nature of attributes is overlooked by generic discriminators when identifying real and synthesized instances. To address the issue, we focus on precisely capturing semantics associated with target attributes in this work, and propose a Graph-based Discriminator architecture for a GAN model, which is referred to as GD-GAN, for explicitly modeling and leveraging the attribute dependencies. Specifically, the co-occurrence ratio between attributes is used to build a correlation matrix, which captures inter-attribute relationships. We design a discriminator with a Graph Convolutional Network (GCN) to integrate knowledge about the attribute dependencies into the adversarial training process. Different from the existing methods that identify the synthesized data conditioned on the attributes individually, we leverage the attribute correlations by performing feature propagation over the graph of attributes, which leads to interdependent representations for real-fake instance identification. Incorporating the relationships of attributes eventually induces the generator to capture precise semantics associated with the attributes. Empirical results on multiple benchmarks demonstrate the superior performance of GD-GAN in high-quality semantic manipulation.

In this work, we study a more realistic challenging scenario in multiview clustering (MVC), referred to as incomplete MVC (IMVC) where some instances in certain views are missing. The key to IMVC is how to adequately exploit complementary and consistency information under the incompleteness of data. However, most existing methods address the incompleteness problem at the instance level and they require sufficient information to perform data recovery. In this work, we develop a new approach to facilitate IMVC based on the graph propagation perspective. Specifically, a partial graph is used to describe the similarity of samples for incomplete views, such that the issue of missing instances can be translated into the missing entries of the partial graph. In this way, a common graph can be adaptively learned to self-guide the propagation process by exploiting the consistency information, and the propagated graph of each view is in turn used to refine the common self-guided graph in an iterative manner. Thus, the associated missing entries can be inferred through graph propagation by exploiting the consistency information across all views. On the other hand, existing approaches focus on the consistency structure only, and the complementary information has not been sufficiently exploited due to the data incompleteness issue. By contrast, under the proposed graph propagation framework, an exclusive regularization term can be naturally adopted to exploit the complementary information in our method. Extensive experiments demonstrate the effectiveness of the proposed method in comparison with state-of-the-art methods. The source code of our method is available at the https://github.com/CLiu272/TNNLS-PGP.

Great progress has been made in StyleGAN-based image editing. To associate with preset attributes, most existing approaches focus on supervised learning for semantically meaningful latent space traversal directions, and each manipulation step is typically determined for an individual attribute. To address this limitation, we propose a Text-guided Unsupervised StyleGAN Latent Transformation (TUSLT) model, which adaptively infers a single transformation step in the latent space of StyleGAN to simultaneously manipulate multiple attributes on a given input image. Specifically, we adopt a two-stage architecture for a latent mapping network to break down the transformation process into two manageable steps. Our network first learns a diverse set of semantic directions tailored to an input image, and later nonlinearly fuses the ones associated with the target attributes to infer a residual vector. The resulting tightly interlinked two-stage architecture delivers the flexibility to handle diverse attribute combinations. By leveraging the cross-modal text-image representation of CLIP, we can perform pseudo annotations based on the semantic similarity between preset attribute text descriptions and training images, and further jointly train an auxiliary attribute classifier with the latent mapping network to provide semantic guidance. We perform extensive experiments to demonstrate that the adopted strategies contribute to the superior performance of TUSLT.

Stereo-based 3D object detection, which aims at detecting 3D objects with stereo cameras, shows great potential in low-cost deployment compared to LiDAR-based methods and excellent performance compared to monocular-based algorithms. However, the impressive performance of stereo-based 3D object detection is at the huge cost of high-quality manual annotations, which are hardly attainable for any given scene. Semi-supervised learning, in which limited annotated data and numerous unannotated data are required to achieve a satisfactory model, is a promising method to address the problem of data deficiency. In this work, we propose to achieve semi-supervised learning for stereo-based 3D object detection through pseudo annotation generation from a temporal-aggregated teacher model, which temporally accumulates knowledge from a student model. To facilitate a more stable and accurate depth estimation, we introduce Temporal-Aggregation-Guided (TAG) disparity consistency, a cross-view disparity consistency constraint between the teacher model and the student model for robust and improved depth estimation. To mitigate noise in pseudo annotation generation, we propose a cross-view agreement strategy, in which pseudo annotations should attain high degree of agreements between 3D and 2D views, as well as between binocular views. We perform extensive experiments on the KITTI 3D dataset to demonstrate our proposed method's capability in leveraging a huge amount of unannotated stereo images to attain significantly improved detection results.

Face retouching aims to remove facial blemishes, while at the same time maintaining the textual details of a given input image. The main challenge lies in distinguishing blemishes from the facial characteristics, such as moles. Training an image-to-image translation network with pixel-wise supervision suffers from the problem of expensive paired training data, since professional retouching needs specialized experience and is time-consuming. In this paper, we propose a Blemish-aware and Progressive Face Retouching model, which is referred to as BPFRe. Our framework can be partitioned into two manageable stages to perform progressive blemish removal. Specifically, an encoder-decoder-based module learns to coarsely remove the blemishes at the first stage, and the resulting intermediate features are injected into a generator to enrich local detail at the second stage. We find that explicitly suppressing the blemishes can contribute to an effective collaboration among the components. Toward this end, we incorporate an attention module, which learns to infer a blemish-aware map and further determine the corresponding weights, which are then used to refine the intermediate features transferred from the encoder to the decoder, and from the decoder to the generator. Therefore, BPFRe is able to deliver significant performance gains on a wide range of face retouching tasks. It is worth noting that we reduce the dependence of BPFRe on paired training samples by imposing effective regularization on unpaired ones.

Semi-supervised class-conditional image synthesis is typically performed by inferring and injecting class labels into a conditional Generative Adversarial Network (GAN). The supervision in the form of class identity may be inadequate to model classes with diverse visual appearances. In this paper, we propose a Learnable Cluster Prompt-based GAN (LCP-GAN) to capture class-wise characteristics and intra-class variation factors with a broader source of supervision. To exploit partially labeled data, we perform soft partitioning on each class, and explore the possibility of associating intra-class clusters with learnable visual concepts in the feature space of a pre-trained language-vision model, e.g., CLIP. For class-conditional image generation, we design a cluster-conditional generator by injecting a combination of intra-class cluster label embeddings, and further incorporate a real-fake classification head on top of CLIP to distinguish real instances from the synthesized ones, conditioned on the learnable cluster prompts. This significantly strengthens the generator with more semantic language supervision. LCP-GAN not only possesses superior generation capability but also matches the performance of the fully supervised version of the base models: BigGAN and StyleGAN2-ADA, on multiple standard benchmarks.

Utilizing deep neural network for Domain Adaptation (DA) has made great progress on learning knowledge from source domain to solve tasks in other relevant target domains. However, conventional DA methods have several restrictions: first, the category set from the source and target domain should be identical; second, during the training process data from different domains are fed into the network simultaneously, which may not be practical in real-world applications. Consequently, we aim at tackling the Source Free Open Set Domain Adaptation scenario: source and target data cannot meet with each other and the target domain contains exclusive unknown classes. Specifically, we propose a method enhancing the unknown class identification ability by synthesizing unknown class data: a feature modifier with multiple Gated Recurrent Units (GRU) is designed to learn and modify key features from the input data in order that the modified data ’looks like’ the real unknown class class data. Both real and synthesis data are used to train the classifier such that it can identify each class correctly. We evaluate our method in multiple benchmarks and the proposed framework outperforms other methods in comparison.

Incomplete multi-view clustering (IMVC) is challenging, as it requires adequately exploring complementary and consistency information under the incompleteness of data. Most existing approaches attempt to overcome the incompleteness at instance-level. In this work, we develop a new approach to facilitate IMVC from a new perspective. Specifically, we transfer the issue of missing instances to a similarity graph completion problem for incomplete views, and propose a self-supervised multi-view graph completion algorithm to infer the associated missing entries. Further, by incorporating constrained feature learning, the inferred graph can be naturally leveraged in representation learning. We theoretically show that our feature learning process performs an Auto-Regressive filter function by encoding the learned similarity graph, which could yield discriminative representation for a clustering task. Extensive experiments demonstrate the effectiveness of the proposed method in comparison with state-of-the-art methods.

Generative Adversarial Network (GAN)-based models have greatly facilitated image synthesis. However, the model performance may be degraded when applied to finegrained data, due to limited training samples and subtle distinction among categories. Different from generic GAN-s, we address the issue from a new perspective of discovering and utilizing the underlying structure of real data to explicitly regularize the spatial organization of latent space. To reduce the dependence of generative models on labeled data, we propose a semi-supervised hyper-spherical GAN for class-conditional fine-grained image generation, and our model is referred to as SphericGAN. By projecting random vectors drawn from a prior distribution onto a hyper-sphere, we can model more complex distributions, while at the same time the similarity between the resulting latent vectors depends only on the angle, but not on their magnitudes. On the other hand, we also incorporate a mapping network to map real images onto the hyper-sphere, and match latent vectors with the underlying structure of real data via real-fake cluster alignment. As a result, we obtain a spatially organized latent space, which is useful for capturing class-independent variation factors. The experi-mental results suggest that our SphericGAN achieves state-of-the-art performance in synthesizing high-fidelity images with precise class semantics.

Although current cross-domain pedestrian detection frame-works have obtained certain positive results, the performance is still source data dependent, which is cumbersome and im-practical in practical applications. To address this issue, we propose a source-free unsupervised pedestrian detection with pseudo label mining and screening. First, a modified CSP de-tector with DropBlock and three detection heads is presented. Then, a multi-expert method is proposed to fuse pseudo la-bels from three detection heads. Finally, a clustering-based self-supervised learning is adopted to categorize pseudo la-bels into positive and negative classes, which forms a set of clusters via similarity of pseudo labels and give classification results based on two confidence scores of each label from the detector backbone and multi-expert fusion. Experimental re-sults on three benchmark datasets show that the proposed approach can achieve state-of-the-art performance and be even comparable with other latest works using source data.

Identifying cancer subtypes by integration of multi-omic data is beneficial to improve the understanding of disease progression, and provides more precise treatment for patients. Cancer subtypes identification is usually accomplished by clustering patients with unsupervised learning approaches. Thus, most existing integrative cancer subtyping methods are performed in an entirely unsupervised way. An integrative cancer subtyping approach can be improved to discover clinically more relevant cancer subtypes when considering the clinical survival response variables. In this study, we propose a Survival Supervised Graph Clustering (S2GC)for cancer subtyping by taking into consideration survival information. Specifically, we use a graph to represent similarity of patients, and develop a multi-omic survival analysis embedding with patient-to-patient similarity graph learning for cancer subtype identification. The multi-view (omic)survival analysis model and graph of patients are jointly learned in a unified way. The learned optimal graph can be unitized to cluster cancer subtypes directly. In the proposed model, the survival analysis model and adaptive graph learning could positively reinforce each other. Consequently, the survival time can be considered as supervised information to improve the quality of the similarity graph and explore clinically more relevant subgroups of patients. Experiments on several representative multi-omic cancer datasets demonstrate that the proposed method achieves better results than a number of state-of-the-art methods. The results also suggest that our method is able to identify biologically meaningful subgroups for different cancer types. (Our Matlab source code is available online at github: https://github.com/CLiu272/S2GC).

Ensemble learning has many successful applications because of its effectiveness in boosting the predictive performance of classification models. In this article, we propose a semisupervised multiple choice learning (SemiMCL) approach to jointly train a network ensemble on partially labeled data. Our model mainly focuses on improving a labeled data assignment among the constituent networks and exploiting unlabeled data to capture domain-specific information, such that semisupervised classification can be effectively facilitated. Different from conventional multiple choice learning models, the constituent networks learn multiple tasks in the training process. Specifically, an auxiliary reconstruction task is included to learn domain-specific representation. For the purpose of performing implicit labeling on reliable unlabeled samples, we adopt a negative$\ell _{1}$-norm regularization when minimizing the conditional entropy with respect to the posterior probability distribution. Extensive experiments on multiple real-world datasets are conducted to verify the effectiveness and superiority of the proposed SemiMCL model.

Broad learning system (BLS) is a novel and efficient model, which facilitates representation learning and classification by concatenating feature nodes and enhancement nodes. In spite of the efficient properties, BLS is still suboptimal when facing with imbalance problem. Besides, outliers and noises in imbalanced data remain a challenge for BLS. To address the above issues, in this paper we first propose a weighted BLS, which assigns a weight to each training sample, and adopt a general weighting scheme, which augments the weight of samples from the minority class. To further explore the prior distribution of original data, we design a density based weight generation mechanism to guide the specific weight matrix generation and propose the adaptive weighted broad learning system (AWBLS). This mechanism considers the inter-class and intra-class distance simultaneously in the density calculation. Finally, we propose the incremental weighted ensemble broad learning system (IWEB) by utilizing a progressive mechanism to further improve the stability and robustness of AWBLS. Extensive comparative experiments on 38 real-world data sets verfy that IWEB outperforms most of the imbalance ensemble classification methods.

In this paper, we propose two GAN-based enhanced deep subspace clustering approaches: deep subspace clustering via dual adversarial generative networks (DSC-DAG) and self-supervised deep subspace clustering with adversarial generative networks ($S^2 DSC-AG$). In DSC-DAG, the distributions of both the inputs and corresponding latent representations are learning via adversarial training simultaneously. Besides, there are two kinds of synthetical representations to facilitate the fine-tuning of the encoder module: the combinations of latent representations with certain random combination coefficients and the representations of real-like inputs derived from noise variables. In$S^DSC-AG$, a self-supervised information learning module substitutes for adversarial learning in the latent space, since both of them play the same role in learning discriminative latent representations. We analyze the connections between these methods and demonstrate their equivalences. We conduct extensive experiments on multiple real-world data sets against state-of-the-art subspace clustering methods in terms of accuracy, normalized mutual information and purity. Experimental results demonstrate the effectiveness and superiority of our proposed methods.

Learning effective generative models for natural image synthesis is a promising way to reduce the dependence of deep models on massive training data. This work focuses on Fine-Grained Image Synthesis (FGIS) in the semi-supervised setting where a small number of training instances are labeled. Different from generic image synthesis tasks, the available fine-grained data may be inadequate, and the differences among the object categories are typically subtle. To address these issues, we propose a Semantic Regularized class-conditional Generative Adversarial Network, which is referred to as SReGAN. We incorporate an additional discriminator and classifier into the generator-discriminator minimax game. Competing with two discriminators enforces the generator to model both marginal and class-conditional data distributions, which alleviates the problem of limited training data and labels. However, the discriminators may overlook the class separability. To induce the generator to discover the distinctions between classes, we construct semantically congruent and incongruent pairs in the generation process, and further regularize the generator by encouraging high similarities of congruent pairs, while penalizing that of incongruent ones in the classifier's feature space. We have conducted extensive experiments to verify the capability of SReGAN in generating high-fidelity images on a variety of FGIS benchmarks.

Generating realistic pedestrian instances in a semi-supervised setting is promising but challenging due to the limited labeled data. We propose an unreliable-to-reliable instance translation model (Un2Reliab) conditioned on unreliable instances which poorly align with pedestrians. Un2Reliab mainly consists of an encoder-decoder-like generative network and a discriminative network, which are jointly trained in a minimax game. We adopt regularization to ensure that the synthesized instances are semantically similar to the corresponding ground truth. Furthermore, to preserve the identities of persons, we propose another regularization to ensure that the synthesized instances associated with the same person should be consistent in appearance. As a result, Un2Reliab learns to restore the missing parts of the original instances. As a side benefit, the synthesized instances are brought into better alignment. Inclusion of the synthesized data improves both the diversity and quality of training data, which eventually leads to better generalization performance. Extensive experiments indicate that Un2Reliab is able to synthesize high-fidelity pedestrian instances and improve the previous state-of-the-art results on multiple semi-supervised pedestrian detection benchmarks.

Recently, multitask learning has been successfully applied to survival analysis problems. A critical challenge in real-world survival analysis tasks is that not all instances and tasks are equally learnable. A survival analysis model can be improved when considering the complexities of instances and tasks during the model training. To this end, we propose an asymmetric graph-guided multitask learning approach with self-paced learning for survival analysis applications. The proposed model is able to improve the learning performance by identifying the complex structure among tasks and considering the complexities of training instances and tasks during the model training. Especially, by incorporating the self-paced learning strategy and asymmetric graph-guided regularization, the proposed model is able to learn the model in a progressive way from "easy" to "hard" loss function items. In addition, together with the self-paced learning function, the asymmetric graph-guided regularization allows the related knowledge transfer from one task to another in an asymmetric way. Consequently, the knowledge acquired from those earlier learned tasks can help to solve complex tasks effectively. The experimental results on both synthetic and real-world TCGA data suggest that the proposed method is indeed useful for improving survival analysis and achieves higher prediction accuracies than the previous state-of-the-art methods.

Graph-based methods have achieved impressive performance on semisupervised classification (SSC). Traditional graph-based methods have two main drawbacks. First, the graph is predefined before training a classifier, which does not leverage the interactions between the classifier training and similarity matrix learning. Second, when handling high-dimensional data with noisy or redundant features, the graph constructed in the original input space is actually unsuitable and may lead to poor performance. In this article, we propose an SSC method with novel graph construction (SSC-NGC), in which the similarity matrix is optimized in both label space and an additional subspace to get a better and more robust result than in original data space. Furthermore, to obtain a high-quality subspace, we learn the projection matrix of the additional subspace by preserving the local and global structure of the data. Finally, we intergrade the classifier training, the graph construction, and the subspace learning into a unified framework. With this framework, the classifier parameters, similarity matrix, and projection matrix of subspace are adaptively learned in an iterative scheme to obtain an optimal joint result. We conduct extensive comparative experiments against state-of-the-art methods over multiple real-world data sets. Experimental results demonstrate the superiority of the proposed method over other state-of-the-art algorithms.

The class imbalance problem has posed a leading challenge in real-world applications. Traditional methods focus on either the data level or algorithm level to solve the binary classification problem on imbalanced data, and seldom consider searching an effective transformation for classification. Besides, the undersampling process adopted in them is always subjective and unilateral. To address the above issues, we first propose a hybrid classifier ensemble (HCE) framework to conduct binary imbalanced data classification, which mainly includes a metric-based data space transformation (MDST) and an adaptive two-stage undersampling process (ATUP). The MDST aims to find a more appropriate embedding space for original imbalance data sets, and the ATUP considers both informative and representative samples to generate balanced data sets. Furthermore, we design a progressive HCE (PHCE) framework to improve the performance of HCE by utilizing a progressive mechanism with local and global evaluation criteria to select ensemble members. Extensive comparative experiments conducted on 28 real-world data sets exhibit that our method PHCE outperforms the majority of imbalance ensemble classification approaches.

Generative Adversarial Networks (GANs) have shown impressive gains in image synthesis. GAN inversion was recently studied to understand and utilize the knowledge it learns, where a real image is inverted back to a latent code and can thus be reconstructed by the generator. Although increasing the number of latent codes can improve inversion quality to a certain extent, we find that important details may still be neglected when performing feature composition over all the intermediate feature channels. To address this issue, we propose a Prior multi-Subspace Feature Composition (PmSFC) approach for high-fidelity inversion. Considering that the intermediate features are highly correlated with each other, we incorporate a self-expressive layer in the generator to discover meaningful subspaces. In this case, the features at a channel can be expressed as a linear combination of those at other channels in the same subspace. We perform feature composition separately in the subspaces. The semantic differences between them benefit the inversion quality, since the inversion process is regularized based on different aspects of semantics. In the experiments, the superior performance of PmSFC demonstrates the effectiveness of prior subspaces in facilitating GAN inversion together with extended applications in visual manipulation.

Semi-supervised generative learning (SSGL) makes use of unlabeled data to achieve a trade-off between the data collection/annotation effort and generation performance, when adequate labeled data are not available. Learning precise class semantics is crucial for class-conditional image synthesis with limited supervision. Toward this end, we propose a semi-supervised Generative Adversarial Network with a Mask-Embedded Discriminator, which is referred to as MED-GAN. By incorporating a mask embedding module, the discriminator features are associated with spatial information, such that the focus of the discriminator can be limited in the specified regions when distinguishing between real and synthesized images. A generator is enforced to synthesize the instances holding more precise class semantics in order to deceive the enhanced discriminator. Also benefiting from mask embedding, region-based semantic regularization is imposed on the discriminator feature space, and the degree of separation between real and fake classes and among object categories can thus be increased. This eventually improves class-conditional distribution matching between real and synthesized data. In the experiments, the superior performance of MED-GAN demonstrates the effectiveness of mask embedding and associated regularizers in facilitating SSGL.

Previous state-of-the-art deep generative models improve fine-grained image generation quality by designing hierarchical model structures and synthesizing images across multiple stages. The learning process is typically performed without any supervision in object categories. To address this issue, while at the same time to alleviate the level of complexity of both model design and training, we propose a Single-Stage Controllable GAN (SSCGAN) for conditional fine-grained image synthesis in a semi-supervised setting. Considering the fact that fine-grained object categories may have subtle distinctions and shared attributes, we take into account three factors of variation for generative modeling: class-independent content, cross-class attributes and class semantics, and associate them with different variables. To ensure disentanglement among the variables, we maximize mutual information between the class-independent variable and synthesized images, map real data to the latent space of a generator to perform consistency regularization of cross-class attributes, and incorporate class semantic-based regularization into a discriminator’s feature space. We show that the proposed approach delivers a single-stage controllable generator and high-fidelity synthesized images of fine-grained categories. SSC-GAN establishes state-of-the-art semi-supervised image synthesis results across multiple fine-grained datasets.

One way of addressing the problem of unsupervised domain adaptation (UDA) is to perform adversarial training between two classifiers and their shared feature extractor. The two classifiers are enforced to detect the misaligned regions between the source and target domains, while the feature extractor aligns the features by confusing the classifiers. Although this method yields improvement, it ignores the relationship among target neighbors, which may consequently limit the model performance. In this work, we propose a new alignment strategy based on the "cluster assumption" to ensure the aligned target features preserve their clusters by avoiding overlap with decision boundaries. Furthermore, to make the aligned features more compact, we constrain them to be ro-bust against adversarial perturbation using the different views of the classifiers. Extensive experiments demonstrate the effectiveness of our solution on various datasets.

Data interpolation is typically used to explore and understand the latent representation learnt by a deep network. Naive linear interpolation may induce mismatch between the interpolated data and the underlying manifold of the original data. In this paper, we propose an Adversarial Adaptive Interpolation (AdvAI) approach for facilitating representation learning and image synthesis in autoencoders. To determine an interpolation path that stays on the manifold, we incorporate an interpolation correction module, which learns to offset the deviation from the manifold. Further, we perform matching with a prior distribution to control the characteristics of the representation. The data synthesized from random codes along with interpolation-based regularization are in turn used to constrain the representation learning process. In the experiments, the superior performance of the proposed approach demonstrates the effectiveness of AdvAI and associated regularizers in a variety of downstream tasks.

In this work, we propose an unsupervised ensemble learning method via network generation, referred to as UELNG. Specifically, we first generate weights of clustering ensemble models by adopting HyperGAN, and obtain diverse partitions for data. With these partitions, we can easily identify high-confident pseudo-labels as supervised information to predict low-entropy labels for unlabeled augmented data, thereby enhancing the quality of pseudo-labels and clustering accuracy. We conduct experiments on multiple data sets. Experimental results indicate that our method outperforms state-of-the-art methods by 0.3%, 1.8%, 5.7%, 3.2% and 2.4% on MNIST, STL-10, CIFAR-10, Reuters and 20News, respectively, which demonstrates the effectiveness of our proposed UELNG.

Recent approaches to unsupervised domain adaptation focus on transferring knowledge from source (labeled) data to target (unlabeled) data. Both data types share the same class space but originate from different domains. One way to achieve this transfer is tasking two classifiers to detect target features that diverge from source features. Meanwhile, a feature generator is adversarially refined to match the detected features with the source ones. However, the aligned features are still subject to ambiguity. Samples that are not smoothly distributed on the latent manifold are often missed in training. Moreover, target data may not be sufficient for adversarial learning. To overcome these problems, our proposed Adversarially Smoothed Feature Alignment (AdvSFA) model is designed to identify ambiguous target inputs by maximizing classifiers discrepancy in an extended class space. This enables the generator to receive valuable feedback from the classifiers and consequently learn more discriminative and smooth representation. Imposing smoothness on the latent manifold is a desirable property to improve model generalization and avoid having neighboring samples of different classes. To further promote such property, we not only task the generator to conduct feature alignment on target examples and but also in-between them. By adopting these constraints, our method shows a remarkable improvement across different adaptation tasks using two benchmark datasets.

Generative adversarial network (GAN)-based models possess superior capability of high-fidelity image synthesis. There are a wide range of semantically meaningful directions in the latent representation space of well-trained GANs, and the corresponding latent space walks are meaningful for semantic controllability in the synthesized images. To explore the underlying organization of a latent space, we propose an unsupervised Density-Preserving Latent Semantics Exploration model (DP-LaSE). The important latent directions are determined by maximizing the variations in intermediate features, while the correlation between the directions is minimized. Considering that latent codes are sampled from a prior distribution, we adopt a density-preserving regularization approach to ensure latent space walks are maintained in iso-density regions, since moving to a higher/lower density region tends to cause unexpected transformations. To further refine semantics-specific transformations, we perform subspace learning over intermediate feature channels, such that the transformations are limited to the most relevant subspaces. Extensive experiments on a variety of benchmark datasets demonstrate that DP-LaSE is able to discover interpretable latent space walks, and specific properties of synthesized images can thus be precisely controlled.

Healthcare question answering (HQA) system plays a vital role in encouraging patients to inquire for professional consultation. However, there are some challenging factors in learning and representing the question corpus of HQA datasets, such as high dimensionality, sparseness, noise, nonprofessional expression, etc. To address these issues, we propose an inception convolutional autoencoder model for Chinese healthcare question clustering (ICAHC). First, we select a set of kernels with different sizes using convolutional autoencoder networks to explore both the diversity and quality in the clustering ensemble. Thus, these kernels encourage to capture diverse representations. Second, we design four ensemble operators to merge representations based on whether they are independent, and input them into the encoder using different skip connections. Third, it maps features from the encoder into a lower-dimensional space, followed by clustering. We conduct comparative experiments against other clustering algorithms on a Chinese healthcare dataset. Experimental results show the effectiveness of ICAHC in discovering better clustering solutions. The results can be used in the prediction of patients' conditions and the development of an automatic HQA system.

Both target-specific and domain-invariant features can facilitate Open Set Domain Adaptation (OSDA). To exploit these features, we propose a Knowledge Exchange (KnowEx) model which jointly trains two complementary constituent networks: (1) a Domain-Adversarial Network (DAdvNet) learning the domain-invariant representation, through which the supervision in source domain can be exploited to infer the class information of unlabeled target data; (2) a Private Network (PrivNet) exclusive for target domain, which is beneficial for discriminating between instances from known and unknown classes. The two constituent networks exchange training experience in the learning process. Toward this end, we exploit an adversarial perturbation process against DAdvNet to regularize PrivNet. This enhances the complementarity between the two networks. At the same time, we incorporate an adaptation layer into DAdvNet to address the unreliability of the PrivNet's experience. Therefore, DAdvNet and PrivNet are able to mutually reinforce each other during training. We have conducted thorough experiments on multiple standard benchmarks to verify the effectiveness and superiority of KnowEx in OSDA.

Class-conditional generative models have gained popularity due to their characteristics of learning disentangled representations. However, these models typically require labeled examples in training. In this paper, we explore the feasibility of training these models on completely unlabeled data, under the assumption that we have access to other labeled data. The labeled data share the same label space, while their domain is shifted. Our model, which we refer to as KTransGAN, incorporates a classifier to transfer knowledge from the labeled data and performs collaborative learning with the conditional generator. By adopting these measures, KTransGAN is able to approximate the conditional distribution of the unlabeled data and simultaneously introduces a new solution to the unsupervised domain adaptation problem. To mitigate the training difficulty of our generative adversarial networks-based model, variational encoding and feature matching are also considered. From the empirical results, KTransGAN exhibits outstanding performance on a number of synthetic datasets and multiple real-world benchmarks. The quality of the synthesized instances is far superior to the pure variational autoencoding model. For example, on the CIFAR-10 dataset, our model scores 35.3 in FID, while the other model scores 128.45. In addition, the synthesis quality is close to the case when the model is trained in a fully supervised setting over the same number of training iterations. Regarding the classification performance, for instance, our model surpasses the highest state-of-the-art results (89.19%) by a large margin and achieves a test accuracy of 95.31% on the unlabeled data SVHN, while MNIST represents the labeled data. These results highlight the effectiveness of our proposed framework.

Semisupervised clustering methods improve performance by randomly selecting pairwise constraints, which may lead to redundancy and instability. In this context, active clustering is proposed to maximize the efficacy of annotations by effectively using pairwise constraints. However, existing methods lack an overall consideration of the querying criteria and repeatedly run semisupervised clustering to update labels. In this work, we first propose an active density peak (ADP) clustering algorithm that considers both representativeness and informativeness. Representative instances are selected to capture data patterns, while informative instances are queried to reduce the uncertainty of clustering results. Meanwhile, we design a fast-update-strategy to update labels efficiently. In addition, we propose an active clustering ensemble framework that combines local and global uncertainties to query the most ambiguous instances for better separation between the clusters. A weighted voting consensus method is introduced for better integration of clustering results. We conducted experiments by comparing our methods with state-of-the-art methods on real-world data sets. Experimental results demonstrate the effectiveness of our methods.

In this paper, we investigate a challenging unsupervised domain adaptation setting --- unsupervised model adaptation. We aim to explore how to rely only on unlabeled target data to improve performance of an existing source prediction model on the target domain, since labeled source data may not be available in some real-world scenarios due to data privacy issues. For this purpose, we propose a new framework, which is referred to as collaborative class conditional generative adversarial net to bypass the dependence on the source data. Specifically, the prediction model is to be improved through generated target-style data, which provides more accurate guidance for the generator. As a result, the generator and the prediction model can collaborate with each other without source data. Furthermore, due to the lack of supervision from source data, we propose a weight constraint that encourages similarity to the source model. A clustering-based regularization is also introduced to produce more discriminative features in the target domain. Compared to conventional domain adaptation methods, our model achieves superior performance on multiple adaptation tasks with only unlabeled target data, which verifies its effectiveness in this challenging setting.

Semi-supervised generative learning aims to learn the underlying class-conditional distribution of partially labeled data. Generative Adversarial Networks (GANs) have led to promising progress in this task. However, it still needs to further explore the issue of imbalance between real labeled data and fake data in the adversarial learning process. To address this issue, we propose a regularization technique based on Random Regional Replacement (R3-regularization) to facilitate the generative learning process. Specifically, we construct two types of between-class instances: cross-category ones and real-fake ones. These instances could be closer to the decision boundaries and are important for regularizing the classification and discriminative networks in our class-conditional GANs, which we refer to as R3-CGAN. Better guidance from these two networks makes the generative network produce instances with class-specific information and high fidelity. We experiment with multiple standard benchmarks, and demonstrate that the R3-regularization can lead to significant improvement in both classification and class-conditional image synthesis.

We address the problem of unsupervised domain adaptation (UDA) which aims at adapting models trained on a labeled domain to a completely unlabeled domain. One way to achieve this goal is to learn a domain-invariant representation. However, this approach is subject to two challenges: samples from two domains are insufficient to guarantee domain-invariance at most part of the latent space, and neighboring samples from the target domain may not belong to the same class on the low-dimensional manifold. To mitigate these shortcomings, we propose two strategies. First, we incorporate a domain mixup strategy in domain adversarial learning model by linearly interpolating between the source and target domain samples. This allows the latent space to be continuous and yields an improvement of the domain matching. Second, the domain discriminator is regularized via judging the relative difference between both domains for the input mixup features, which speeds up the domain matching. Experiment results show that our proposed model achieves a superior performance on different tasks under various domain shifts and data complexity.

Multitask feature selection (MTFS) methods have become more important for many real world applications, especially in a high-dimensional setting. The most widely used assumption is that all tasks share the same features, and the l2,1 regularization method is usually applied. However, this assumption may not hold when the correlations among tasks are not obvious. Learning with unrelated tasks together may result in negative transfer and degrade the performance. In this paper, we present a flexible MTFS by graph-clustered feature sharing approach. To avoid the above limitation, we adopt a graph to represent the relevance among tasks instead of adopting a hard task set partition. Furthermore, we propose a graph-guided regularization approach such that the sparsity of the solution can be achieved on both the task level and the feature level, and a variant of the smooth proximal gradient method is developed to solve the corresponding optimization problem. An evaluation of the proposed method on multitask regression and multitask binary classification problem has been performed. Extensive experiments on synthetic datasets and real-world datasets demonstrate the effectiveness of the proposed approach to capture task structure.

Clustering ensemble (CE) takes multiple clustering solutions into consideration in order to effectively improve the accuracy and robustness of the final result. To reduce redundancy as well as noise, a CE selection (CES) step is added to further enhance performance. Quality and diversity are two important metrics of CES. However, most of the CES strategies adopt heuristic selection methods or a threshold parameter setting to achieve tradeoff between quality and diversity. In this paper, we propose a transfer CES (TCES) algorithm which makes use of the relationship between quality and diversity in a source dataset, and transfers it into a target dataset based on three objective functions. Furthermore, a multiobjective self-evolutionary process is designed to optimize these three objective functions. Finally, we construct a transfer CE framework (TCE-TCES) based on TCES to obtain better clustering results. The experimental results on 12 transfer clustering tasks obtained from the 20newsgroups dataset show that TCE-TCES can find a better tradeoff between quality and diversity, as well as obtaining more desirable clustering results.

Deep learning demonstrates its impressive success across various machine learning problems. However, its performance often suffers in the case where the training and test data sets follow different distributions, due to the domain shift. Most current domain adaptation methods minimize the discrepancy between the source and target domains by enforcing the alignment of their marginal distributions without considering the class-level matching. Consequently, data from different classes may become close together after mapping. To address this issue, we propose an unsupervised domain adaptation method by generating image-label pairs in the target domain, in which the model is augmented with the generated target pairs and achieve class-level transfer. Specifically, we integrate generative adversarial networks (GAN) into the model predictor, where the generator fed with labels aims to produce corresponding target domain images with a well-designed semantic loss. Meanwhile, compared to previous methods which focus on discrepancy reduction across domains, i.e., image to image translation, our model focuses on semantic preservation during image generation. Our model is straightforward yet effective for unsupervised domain adaptation problems. Without any labels in the target domain in all the experiments, we demonstrate the validity of our approach by presenting the plausible generated target image-label pairs. In addition, our proposed method achieves the best or comparable performance on multiple unsupervised domain adaptation datasets which include image classification and semantic segmentation.

Using an ensemble of neural networks with consistency regularization is effective for improving performance and stability of deep learning, compared to the case of a single network. In this paper, we present a semi-supervised Deep Coupled Ensemble (DCE) model, which contributes to ensemble learning and classification landmark exploration for better locating the final decision boundaries in the learnt latent space. First, multiple complementary consistency regularizations are integrated into our DCE model to enable the ensemble members to learn from each other and themselves, such that training experience from different sources can be shared and utilized during training. Second, in view of the possibility of producing incorrect predictions on a number of difficult instances, we adopt class-wise mean feature matching to explore important unlabeled instances as classification landmarks, on which the model predictions are more reliable. Minimizing the weighted conditional entropy on unlabeled data is able to force the final decision boundaries to move away from important training data points, which facilitates semi-supervised learning. Ensemble members could eventually have similar performance due to consistency regularization, and thus only one of these members is needed during the test stage, such that the efficiency of our model is the same as the non-ensemble case. Extensive experimental results demonstrate the superiority of our proposed DCE model over existing state-of-the-art semi-supervised learning methods.

In this paper, we explore how to leverage readily available unlabeled data to improve semi-supervised human detection performance. For this purpose, we specifically modify the region proposal network (RPN) for learning on a partially labeled dataset. Based on commonly observed false positive types, a verification module is developed to assess foreground human objects in the candidate regions to provide an important cue for filtering the RPN's proposals. The remaining proposals with high confidence scores are then used as pseudo annotations for re-training our detection model. To reduce the risk of error propagation in the training process, we adopt a self-paced training strategy to progressively include more pseudo annotations generated by the previous model over multiple training rounds. The resulting detector re-trained on the augmented data can be expected to have better detection performance. The effectiveness of the main components of this framework is verified through extensive experiments, and the proposed approach achieves state-of-the-art detection results on multiple scene-specific human detection benchmarks in the semi-supervised setting.

The class imbalance problem has become a leading challenge. Although conventional imbalance learning methods are proposed to tackle this problem, they have some limitations: 1) undersampling methods suffer from losing important information and 2) cost-sensitive methods are sensitive to outliers and noise. To address these issues, we propose a hybrid optimal ensemble classifier framework that combines density-based undersampling and cost-effective methods through exploring state-of-the-art solutions using multi-objective optimization algorithm. Specifically, we first develop a density-based undersampling method to select informative samples from the original training data with probability-based data transformation, which enables to obtain multiple subsets following a balanced distribution across classes. Second, we exploit the cost-sensitive classification method to address the incompleteness of information problem via modifying weights of misclassified minority samples rather than the majority ones. Finally, we introduce a multi-objective optimization procedure and utilize connections between samples to self-modify the classification result using an ensemble classifier framework. Extensive comparative experiments conducted on real-world data sets demonstrate that our method outperforms the majority of imbalance and ensemble classification approaches.

For unsupervised domain adaptation, the process of learning domain-invariant representations could be dominated by the labeled source data, such that the specific characteristics of the target domain may be ignored. In order to improve the performance in inferring target labels, we propose a targetspecific network which is capable of learning collaboratively with a domain adaptation network, instead of directly minimizing domain discrepancy. A clustering regularization is also utilized to improve the generalization capability of the target-specific network by forcing target data points to be close to accumulated class centers. As this network learns and specializes to the target domain, its performance in inferring target labels improves, which in turn facilitates the learning process of the adaptation network. Therefore, there is a mutually beneficial relationship between these two networks. We perform extensive experiments on multiple digit and object datasets, and the effectiveness and superiority of the proposed approach is presented and verified on multiple visual adaptation benchmarks, e.g., we improve the state-ofthe-art on the task of MNIST→SVHN from 76.5% to 84.9% without specific augmentation.

Learning class-conditional data distributions is crucial for Generative Adversarial Networks (GAN) in semi-supervised learning. To improve both instance synthesis and classification in this setting, we propose an enhanced TripleGAN (EnhancedTGAN) model in this work. We follow the adversarial training scheme of the original TripleGAN, but completely re-design the training targets of the generator and classifier. Specifically, we adopt feature-semantics matching to enhance the generator in learning class-conditional distributions from both the aspects of statistics in the latent space and semantics consistency with respect to the generator and classifier. Since a limited amount of labeled data is not sufficient to determine satisfactory decision boundaries, we include two classifiers, and incorporate collaborative learning into our model to provide better guidance for generator training. The synthesized high-fidelity data can in turn be used for improving classifier training. In the experiments, the superior performance of our approach on multiple benchmark datasets demonstrates the effectiveness of the mutual reinforcement between the generator and classifiers in facilitating semi-supervised instance synthesis and classification.

Deep mutual learning jointly trains multiple essential networks having similar properties to improve semi-supervised classification. However, the commonly used consistency regularization between the outputs of the networks may not fully leverage the difference between them. In this paper, we explore how to capture the complementary information to enhance mutual learning. For this purpose, we propose a complementary correction network (CCN), built on top of the essential networks, to learn the mapping from the output of one essential network to the ground truth label, conditioned on the features learnt by another. To make the second essential network increasingly complementary to the first one, this network is supervised by the corrected predictions. As a result, minimizing the prediction divergence between the two complementary networks can lead to significant performance gains in semi-supervised learning. Our experimental results demonstrate that the proposed approach clearly improves mutual learning between essential networks, and achieves state-of-the-art results on multiple semi-supervised classification benchmarks. In particular, the test error rates are reduced from previous 21.23% and 14.65% to 12.05% and 10.37% on CIFAR-10 with 1000 and 2000 labels, respectively.

We propose a GAN-based scene-specific instance synthesis and classification model for semi-supervised pedestrian detection. Instead of collecting unreliable detections from unlabeled data, we adopt a class-conditional GAN for synthesizing pedestrian instances to alleviate the problem of insufficient labeled data. With the help of a base detector, we integrate pedestrian instance synthesis and detection by including a post-refinement classifier (PRC) into a minimax game. A generator and the PRC can mutually reinforce each other by synthesizing high-fidelity pedestrian instances and providing more accurate categorical information. Both of them compete with a class-conditional discriminator and a class-specific discriminator, such that the four fundamental networks in our model can be jointly trained. In our experiments, we validate that the proposed model significantly improves the performance of the base detector and achieves state-of-the-art results on multiple benchmarks. As shown in Figure 1, the result indicates the possibility of using inexpensively synthesized instances for improving semi-supervised detection models.

Autoencoders enjoy a remarkable ability to learn data representations. Research on autoencoders shows that the effectiveness of data interpolation can reflect the performance of representation learning. However, existing interpolation methods in autoencoders do not have enough capability of traversing a possible region between two datapoints on a data manifold, and the distribution of interpolated latent representations is not considered.To address these issues, we aim to fully exert the potential of data interpolation and further improve representation learning in autoencoders. Specifically, we propose the multidimensional interpolation to increase the capability of data interpolation by randomly setting interpolation coefficients for each dimension of latent representations. In addition, we regularize autoencoders in both the latent and the data spaces by imposing a prior on latent representations in the Maximum Mean Discrepancy (MMD) framework and encouraging generated datapoints to be realistic in the Generative Adversarial Network (GAN) framework. Compared to representative models, our proposed model has empirically shown that representation learning exhibits better performance on downstream tasks on multiple benchmarks.

In gene expression data analysis, the problems of cancer classification and gene selection are closely related. Successfully selecting informative genes will significantly improve the classification performance. To identify informative genes from a large number of candidate genes, various methods have been proposed. However, the gene expression data may include some important correlation structures, and some of the genes can be divided into different groups based on their biological pathways. Many existing methods do not take into consideration the exact correlation structure within the data. Therefore, from both the knowledge discovery and biological perspectives, an ideal gene selection method should take this structural information into account. Moreover, the better generalization performance can be obtained by discovering correlation structure within data. In order to discover structure information among data and improve learning performance, we propose a structured penalized logistic regression model which simultaneously performs feature selection and model learning for gene expression data analysis. An efficient coordinate descent algorithm has been developed to optimize the model. The numerical simulation studies demonstrate that our method is able to select the highly correlated features. In addition, the results from real gene expression datasets show that the proposed method performs competitively with respect to previous approaches.

Classification of high-dimensional data with very limited labels is a challenging task in the field of data mining and machine learning. In this paper, we propose the multiobjective semisupervised classifier ensemble (MOSSCE) approach to address this challenge. Specifically, a multiobjective subspace selection process (MOSSP) in MOSSCE is first designed to generate the optimal combination of feature subspaces. Three objective functions are then proposed for MOSSP, which include the relevance of features, the redundancy between features, and the data reconstruction error. Then, MOSSCE generates an auxiliary training set based on the sample confidence to improve the performance of the classifier ensemble. Finally, the training set, combined with the auxiliary training set, is used to select the optimal combination of basic classifiers in the ensemble, train the classifier ensemble, and generate the final result. In addition, diversity analysis of the ensemble learning process is applied, and a set of nonparametric statistical tests is adopted for the comparison of semisupervised classification approaches on multiple datasets. The experiments on 12 gene expression datasets and two large image datasets show that MOSSCE has a better performance than other state-of-the-art semisupervised classifiers on high-dimensional data.

In the era of data science, a huge amount of data has emerged in the form of tensors. In many applications, the collected tensor data are incomplete with missing entries, which affects the analysis process. In this paper, we investigate a new method for tensor completion, in which a low-rank tensor approximation is used to exploit the global structure of data, and sparse coding is used for elucidating the local patterns of data. Regarding the characterization of low-rank structures, a weighted nuclear norm for the tensor is introduced. Meanwhile, an orthogonal dictionary learning process is incorporated into sparse coding for more effective discovery of the local details of data. By simultaneously using the global patterns and local cues, the proposed method can effectively and efficiently recover the lost information of incomplete tensor data. The capability of the proposed method is demonstrated with several experiments on recovering MRI data and visual data, and the experimental results have shown the excellent performance of the proposed method in comparison with recent related methods.

Traditional ensemble learning approaches explore the feature space and the sample space, respectively, which will prevent them to construct more powerful learning models for noisy real-world dataset classification. The random subspace method only search for the selection of features. Meanwhile, the bagging approach only search for the selection of samples. To overcome these limitations, we propose the hybrid incremental ensemble learning (HIEL) approach which takes into consideration the feature space and the sample space simultaneously to handle noisy dataset. Specifically, HIEL first adopts the bagging technique and linear discriminant analysis to remove noisy attributes, and generates a set of bootstraps and the corresponding ensemble members in the subspaces. Then, the classifiers are selected incrementally based on a classifier-specific criterion function and an ensemble criterion function. The corresponding weights for the classifiers are assigned during the same process. Finally, the final label is summarized by a weighted voting scheme, which serves as the final result of the classification. We also explore various classifier-specific criterion functions based on different newly proposed similarity measures, which will alleviate the effect of noisy samples on the distance functions. In addition, the computational cost of HIEL is analyzed theoretically. A set of nonparametric tests are adopted to compare HIEL and other algorithms over several datasets. The experiment results show that HIEL performs well on the noisy datasets. HIEL outperforms most of the compared classifier ensemble methods on 14 out of 24 noisy real-world UCI and KEEL datasets.

High dimensional data classification with very limited labeled training data is a challenging task in the area of data mining. In order to tackle this task, we first propose a feature selection-based semi-supervised classifier ensemble framework (FSCE) to perform high dimensional data classification. Then, we design an adaptive semi-supervised classifier ensemble framework (ASCE) to improve the performance of FSCE. When compared with FSCE, ASCE is characterized by an adaptive feature selection process, an adaptive weighting process (AWP), and an auxiliary training set generation process (ATSGP). The adaptive feature selection process generates a set of compact subspaces based on the selected attributes obtained by the feature selection algorithms, while the AWP associates each basic semi-supervised classifier in the ensemble with a weight value. The ATSGP enlarges the training set with unlabeled samples. In addition, a set of nonparametric tests are adopted to compare multiple semi-supervised classifier ensemble (SSCE)approaches over different datasets. The experiments on 20 high dimensional real-world datasets show that: 1) the two adaptive processes in ASCE are useful for improving the performance of the SSCE approach and 2) ASCE works well on high dimensional datasets with very limited labeled training data, and outperforms most state-of-the-art SSCE approaches.

Multitask clustering methods are proposed to improve performances of related tasks concurrently, because they explore the relationship among tasks via exploiting the coefficient matrix or the shared feature matrix. However, divergent effects of features in learning this relationship are seldom considered. To further improve performances, we propose a new multitask clustering approach through exploring correlations among tasks, clusters, and features based on effects of features on clusters. First, a Feature-Cluster (FeaCluster) matrix is introduced to capture the similarity and the distinct task-feature information simultaneously for each task. With the FeaCluster matrix, two affinities are calculated to constitute the interdependencies among tasks: the former is the graphical affinity based on feature-task and task-cluster correlations, while the latter is the reconstructive affinity. Here, the feature-task correlation considers effects of features on tasks, and the task-cluster correlation considers the overall effects of features on clusters. The reconstructive affinity is obtained by minimizing the reconstruction error when representing the FeaCluster matrix for a given task with a linear combination of others. The interdependencies among tasks allow transferring asymmetric shared information, exploring significant features and preserving key information when mapping data into the subspace. The experimental results on multiple data sets reveal that the proposed approach outperforms the state-of-the-art clustering methods in terms of accuracy and normal mutual information.

Existing domain adaptation approaches generalize models trained on the labeled source domain data to the unlabeled target domain data by forcing feature distributions of two domains closer. However, these approaches are likely to ignore the semantic information during the feature alignment between source and target domain. In this paper, we propose a new unsupervised domain adaptation framework to learn the cross-domain features and disentangle the semantic information concurrently. Specifically, we firstly combine the task-specific classification and domain adversarial learning to obtain the cross-domain features by mapping the data of both domains with the shared feature extractor. Secondly, we integrate the domain adversarial learning and the within-domain reconstruction to disentangle the semantic information from the domain information. Thirdly, we include a cross-domain transformation to further refine the feature extractor, which in turn improves the performances of the task classifier. We compare our proposed model to previous state-of-the-art methods on domain adaptation digit classification tasks. Experimental results show that our model achieves better performances than the other counterparts, which demonstrates the superiority and effectiveness of our model.

Generic human detectors perform poorly in application scenes in which conditions are significantly different from those of the benchmark data sets. Based on the assumption that only a limited number of labeled examples are available, we propose a variant semi-supervised boosting approach for improving scene adaptiveness by utilizing unlabeled data. Specifically, we train a max-margin-based model as an initial detector for new example collection, instead of using generic detectors, and then a better model is trained via boosting in which the newly obtained examples influence the training process through their similarities to the labeled examples. Since the widely used human descriptors are usually high dimensional and redundant, we employ a graph-based method to determine the weight representing the importance of each feature, such that the weighted similarity measurement leads to a performance gain. In the experiments, the effectiveness of the proposed approach “Variant SemiBoost” is demonstrated and state-of-the-art performance on challenging data sets is achieved.

The objective of cluster analysis is to partition a set of data points into several groups based on a suitable distance measure. We first propose a model called local gravitation among data points. In this model, each data point is viewed as an object with mass, and associated with a local resultant force (LRF) generated by its neighbors. The motivation of this paper is that there exist distinct differences between the LRFs (including magnitudes and directions) of the data points close to the cluster centers and at the boundary of the clusters. To capture these differences efficiently, two new local measures named centrality and coordination are further investigated. Based on empirical observations, two new clustering methods called local gravitation clustering and communication with local agents are designed, and several test cases are conducted to verify their effectiveness. The experiments on synthetic data sets and real-world data sets indicate that both clustering approaches achieve good performance on most of the data sets.

Semisupervised learning methods are often adopted to handle datasets with very small number of labeled samples. However, conventional semisupervised ensemble learning approaches have two limitations: 1) most of them cannot obtain satisfactory results on high dimensional datasets with limited labels and 2) they usually do not consider how to use an optimization process to enlarge the training set. In this paper, we propose the progressive semisupervised ensemble learning approach (PSEMISEL) to address the above limitations and handle datasets with very small number of labeled samples. When compared with traditional semisupervised ensemble learning approaches, PSEMISEL is characterized by two properties: 1) it adopts the random subspace technique to investigate the structure of the dataset in the subspaces and 2) a progressive training set generation process and a self evolutionary sample selection process are proposed to enlarge the training set. We also use a set of nonparametric tests to compare different semisupervised ensemble learning methods over multiple datasets. The experimental results on 18 real-world datasets from the University of California, Irvine machine learning repository show that PSEMISEL works well on most of the real-world datasets, and outperforms other state-of-the-art approaches on 10 out of 18 datasets.

The difference between sample distributions of public data sets and specific scenes can be very significant. As a result, the deployment of generic human detectors in real-world scenes most often leads to sub-optimal detection performance. To avoid the labor-intensive task of manual annotations, we propose a semi-supervised approach for training deep convolutional networks on partially labeled data. To exploit a large amount of unlabeled target data, the knowledge learnt from public data sets is transferred to new model training by adapting an auxiliary detector to the target scene. We hypothesize that the components of the auxiliary detector capture essential human characteristics useful for constructing a scene-adapted detector. A selective ensemble algorithm is proposed to select a subset of the components relevant to the target scene for recombination. The resulting model is applied for collecting high-confidence samples from unlabeled target data. Furthermore, a deep convolutional network is trained by progressively labeling and selecting new training samples in a self-paced way. The detailed experimental evaluation verifies the effectiveness and superiority of the proposed approach in scene-specific human detection.

Traditional cluster ensemble approaches have several limitations. (1) Few make use of prior knowledge provided by experts. (2) It is difficult to achieve good performance in high-dimensional datasets. (3) All of the weight values of the ensemble members are equal, which ignores different contributions from different ensemble members. (4) Not all pairwise constraints contribute to the final result. In the face of this situation, we propose double weighting semi-supervised ensemble clustering based on selected constraint projection(DCECP) which applies constraint weighting and ensemble member weighting to address these limitations. Specifically, DCECP first adopts the random subspace technique in combination with the constraint projection procedure to handle high-dimensional datasets. Second, it treats prior knowledge of experts as pairwise constraints, and assigns different subsets of pairwise constraints to different ensemble members. An adaptive ensemble member weighting process is designed to associate different weight values with different ensemble members. Third, the weighted normalized cut algorithm is adopted to summarize clustering solutions and generate the final result. Finally, nonparametric statistical tests are used to compare multiple algorithms on real-world datasets. Our experiments on 15 high-dimensional datasets show that DCECP performs better than most clustering algorithms.

In a semi-supervised setting, direct training of a deep discriminative model on partially labeled images often suffers from overfitting and poor performance, because only a small number of labeled images are available, and errors in label propagation are, in many cases, inevitable. In this paper, we introduce an auxiliary clustering task to explore the structure of the image data, and judiciously weigh unlabeled data to alleviate the influence of ambiguous data on model training. For this purpose, we propose a cross-task network composed of two streams to jointly learn two tasks: classification and clustering. Based on the model predictions, a large number of pairwise constraints can be generated from unlabeled images, and are fed to the clustering stream. Since pairwise constraints encode weak supervision information, the clustering is tolerant of errors in labeling. Unlabeled images are weighted according to the distances to the clusters discovered, and a better discriminative model is trained on the classification stream associated with a weighted softmax loss. Furthermore, a self-paced learning paradigm is adopted to gradually train our deep model from easy examples to difficult ones. Experimental results on widely used image classification datasets confirm the effectiveness and superiority of the proposed approach.

Nonparametric statistical analysis, such as the Friedman test (FT), is gaining more and more attention due to its useful applications in a lot of experimental studies. However, traditional FT for the comparison of multiple learning algorithms on different datasets adopts the naive ranking approach. The ranking is based on the average accuracy values obtained by the set of learning algorithms on the datasets, which neither considers the differences of the results obtained by the learning algorithms on each dataset nor takes into account the performance of the learning algorithms in each run. In this paper, we will first propose three kinds of ranking approaches, which are the weighted ranking approach, the global ranking approach (GRA), and the weighted GRA. Then, a theoretical analysis is performed to explore the properties of the proposed ranking approaches. Next, a set of the modified FTs based on the proposed ranking approaches are designed for the comparison of the learning algorithms. Finally, the modified FTs are evaluated through six classifier ensemble approaches on 34 real-world datasets. The experiments show the effectiveness of the modified FTs.

The objective of cluster structure ensemble is to find a unified cluster structure from multiple cluster structures obtained from different datasets. Unfortunately, not all the cluster structures contribute to the unified cluster structure. This paper investigates the problem of how to select the suitable cluster structures in the ensemble which will be summarized to a more representative cluster structure. Specifically, the cluster structure is first represented by a mixture of Gaussian distributions, the parameters of which are estimated using the expectation-maximization algorithm. Then, several distribution-based distance functions are designed to evaluate the similarity between two cluster structures. Based on the similarity comparison results, we propose a new approach, which is referred to as the distribution-based cluster structure ensemble (DCSE) framework, to find the most representative unified cluster structure. We then design a new technique, the distribution-based cluster structure selection strategy (DCSSS), to select a subset of cluster structures. Finally, we propose using a distribution-based normalized hypergraph cut algorithm to generate the final result. In our experiments, a nonparametric test is adopted to evaluate the difference between DCSE and its competitors. We adopt 20 real-world datasets obtained from the University of California, Irvine and knowledge extraction based on evolutionary learning repositories, and a number of cancer gene expression profiles to evaluate the performance of the proposed methods. The experimental results show that: 1) DCSE works well on the real-world datasets and 2) DCSE based on DCSSS can further improve the performance of the algorithm.

Conventional semi-supervised clustering approaches have several shortcomings, such as (1) not fully utilizing all useful must-link and cannot-link constraints, (2) not considering how to deal with high dimensional data with noise, and (3) not fully addressing the need to use an adaptive process to further improve the performance of the algorithm. In this paper, we first propose the transitive closure based constraint propagation approach, which makes use of the transitive closure operator and the affinity propagation to address the first limitation. Then, the random subspace based semi-supervised clustering ensemble framework with a set of proposed confidence factors is designed to address the second limitation and provide more stable, robust, and accurate results. Next, the adaptive semi-supervised clustering ensemble framework is proposed to address the third limitation, which adopts a newly designed adaptive process to search for the optimal subspace set. Finally, we adopt a set of nonparametric tests to compare different semi-supervised clustering ensemble approaches over multiple datasets. The experimental results on 20 real high dimensional cancer datasets with noisy genes and 10 datasets from UCI datasets and KEEL datasets show that (1) The proposed approaches work well on most of the real-world datasets. (2) It outperforms other state-of-the-art approaches on 12 out of 20 cancer datasets, and 8 out of 10 UCI machine learning datasets.

Recently, cluster ensemble approaches have gained more and more attention [1]–[2], due to useful applications in the areas of pattern recognition, data mining, bioinformatics, and so on. When compared with traditional single clustering algorithms, cluster ensemble approaches are able to integrate multiple clustering solutions obtained from different data sources into a unified solution, and provide a more robust, stable and accurate final result.

Transcription factor binding sites (TFBSs) are relatively short (5-15 bp) and degenerate. Identifying them is a computationally challenging task. In particular, protein binding microarray (PBM) is a high-throughput platform that can measure the DNA binding preference of a protein in a comprehensive and unbiased manner; for instance, a typical PBM experiment can measure binding signal intensities of a protein to all possible DNA k-mers (k = 8∼10). Since proteins can often bind to DNA with different binding intensities, one of the major challenges is to build TFBS (also known as DNA motif) models which can fully capture the quantitative binding affinity data. To learn DNA motif models from the non-convex objective function landscape, several optimization methods are compared and applied to the PBM motif model building problem. In particular, representative methods from different optimization paradigms have been chosen for modeling performance comparison on hundreds of PBM datasets. The results suggest that the multimodal optimization methods are very effective for capturing the binding preference information from PBM data. In particular, we observe a general performance improvement if choosing di-nucleotide modeling over mono-nucleotide modeling. In addition, the models learned by the best-performing method are applied to two independent applications: PBM probe rotation testing and ChIP-Seq peak sequence prediction, demonstrating its biological applicability.

Traditional cluster ensemble approaches have three limitations: (1) They do not make use of prior knowledge of the datasets given by experts. (2) Most of the conventional cluster ensemble methods cannot obtain satisfactory results when handling high dimensional data. (3) All the ensemble members are considered, even the ones without positive contributions. In order to address the limitations of conventional cluster ensemble approaches, we first propose an incremental semi-supervised clustering ensemble framework (ISSCE) which makes use of the advantage of the random subspace technique, the constraint propagation approach, the proposed incremental ensemble member selection process, and the normalized cut algorithm to perform high dimensional data clustering. The random subspace technique is effective for handling high dimensional data, while the constraint propagation approach is useful for incorporating prior knowledge. The incremental ensemble member selection process is newly designed to judiciously remove redundant ensemble members based on a newly proposed local cost function and a global cost function, and the normalized cut algorithm is adopted to serve as the consensus function for providing more stable, robust, and accurate results. Then, a measure is proposed to quantify the similarity between two sets of attributes, and is used for computing the local cost function in ISSCE. Next, we analyze the time complexity of ISSCE theoretically. Finally, a set of nonparametric tests are adopted to compare multiple semisupervised clustering ensemble approaches over different datasets. The experiments on 18 real-world datasets, which include six UCI datasets and 12 cancer gene expression profiles, confirm that ISSCE works well on datasets with very high dimensionality, and outperforms the state-of-the-art semi-supervised clustering ensemble approaches.

With the development of social media applications, short text mining is becoming more and more important. Due to the sparseness of short text data, both the feature correlation information (word co-occurrence) and data contiguity information (context information) are less reliable, thus most existing text mining methods which are designed to address regular text data are less efficient in short text mining tasks. According to our observation from analysis of discriminative term distribution in short text data, we found that discriminative terms distribute in a non-uniform way among different domains, while background words have a tendency to distribute uniformly. This observation can be measured by a suitably defined functional of a term's probability distribution over different domains. In this paper, we adopt this distribution as the weight of terms to address the sparseness problem of short text data. We evaluate our method on two datasets, and experimental results show that our method outperforms previous approaches which require information infusion, and a number of state-of-the-art clustering algorithms. Furthermore, our method can obtain a more coherent clustering result.

Performing clustering analysis is one of the important research topics in cancer discovery using gene expression profiles, which is crucial in facilitating the successful diagnosis and treatment of cancer. While there are quite a number of research works which perform tumor clustering, few of them considers how to incorporate fuzzy theory together with an optimization process into a consensus clustering framework to improve the performance of clustering analysis. In this paper, we first propose a random double clustering based cluster ensemble framework (RDCCE) to perform tumor clustering based on gene expression data. Specifically, RDCCE generates a set of representative features using a randomly selected clustering algorithm in the ensemble, and then assigns samples to their corresponding clusters based on the grouping results. In addition, we also introduce the random double clustering based fuzzy cluster ensemble framework (RDCFCE), which is designed to improve the performance of RDCCE by integrating the newly proposed fuzzy extension model into the ensemble framework. RDCFCE adopts the normalized cut algorithm as the consensus function to summarize the fuzzy matrices generated by the fuzzy extension models, partition the consensus matrix, and obtain the final result. Finally, adaptive RDCFCE (A-RDCFCE) is proposed to optimize RDCFCE and improve the performance of RDCFCE further by adopting a self-evolutionary process (SEPP) for the parameter set. Experiments on real cancer gene expression profiles indicate that RDCFCE and A-RDCFCE works well on these data sets, and outperform most of the state-of-the-art tumor clustering algorithms.

Multi-view learning, one of the important sub-fields in the area of machine learning, has gained more and more attention in class prediction of gene expression datasets. In this paper, we propose a new classifier ensemble framework, named as multi-view based Ad-a boost classifier ensemble framework (MV-ACE), which not only utilizes a random view generation technique to regulate different views and applies adaboost to adjust the training set, but also designs an adaptive process which explores the feasible combination of multiple views through an optimization process. Traditional multi-view learning focuses on exploring diverse views and the best integration of multiple views in a straight-forward manner, such as the linear combination of different views. Our proposed model, however, additionally applies a progressive training approach to improve the accuracies of the base classifiers. Moreover, we investigate the assembly of views at the model level, and employ an adaptive process to optimize the multi-view learning model to improve its performance. Our experiments on 12 cancer gene data sets for the classification task show that(i) MV-ACE works well on a diverse class of cancer gene expression profiles. (ii) It outperforms most of the state-of-the-art classifier ensemble approaches on these datasets.

Outlier ranking methods can provide a quantitative measure to evaluate the outlierness of data instances in data clustering and attract great interest in pattern recognition and data mining communities. However, it has been pointed out that the diverse scaling ranges of these scores bring difficulty to result interpretation. Moreover, popular outlier ranking scores based on simple distance measures might not accurately reflect the complex affinity among data points. In this paper, we propose a new outlier ranking method based on consensus affinity of a cluster ensemble. Two new outlier ranking scores generalized from well-known clustering evaluation measures, Rvv from the RAND measure and ARIvv from Adjusted Rand Index (ARI), are adopted for outlierness evaluation. Compared to other outlierness ranking measures, the two new measures have the desired bounds without additional transformations. Consistent with the improvement of Adjusted Rand Index (ARI) over RAND, we find that ARIvv also significantly outperforms Rvv. Benefiting from the consensus affinity of a cluster ensemble, our proposed method with the ARIvv score provides significant improvement beyond a number of competing algorithms on public UCI benchmark data sets. Studies with both theoretical analysis and experimental validation show the effectiveness of our proposed methods.

In recent years, semi-supervised clustering receives considerable attention in the pattern recognition and data mining communities. This type of clustering algorithms takes advantage of partial prior knowledge, and significant improved performance beyond traditional unsupervised clustering algorithms is observed. In general, the partial prior knowledge is mainly in the form of pairwise constraints, which specify whether point pairs should be in the same cluster or in different clusters. Moreover, some other forms of constraints also attract research interests, for example, the balance constraint or the size constraint. However, it is also important to consider different types of constraints simultaneously, since different types of prior knowledge might have their own bias when considered separately. In this paper, we propose an improved algorithm to incorporate the pairwise and size constraints into a unified framework. Experiments on several benchmark data sets demonstrate that the proposed unified algorithm outperforms previous approaches under a variety of different conditions, which demonstrates that judicious integration of different types of constraints can result in improved performance than in those cases where only a single kind of constraint is used.

Tumor clustering is one of the important techniques for tumor discovery from cancer gene expression profiles, which is useful for the diagnosis and treatment of cancer. While different algorithms have been proposed for tumor clustering, few make use of the expert's knowledge to better the performance of tumor discovery. In this paper, we first view the expert's knowledge as constraints in the process of clustering, and propose a feature selection based semi-supervised cluster ensemble framework (FS-SSCE) for tumor clustering from bio-molecular data. Compared with traditional tumor clustering approaches, the proposed framework FS-SSCE is featured by two properties: (1) The adoption of feature selection techniques to dispel the effect of noisy genes. (2) The employment of the binate constraint based K-means algorithm to take into account the effect of experts' knowledge. Then, a double selection based semi-supervised cluster ensemble framework (DS-SSCE) which not only applies the feature selection technique to perform gene selection on the gene dimension, but also selects an optimal subset of representative clustering solutions in the ensemble and improve the performance of tumor clustering using the normalized cut algorithm. DS-SSCE also introduces a confidence factor into the process of constructing the consensus matrix by considering the prior knowledge of the data set. Finally, we design a modified double selection based semi-supervised cluster ensemble framework (MDS-SSCE) which adopts multiple clustering solution selection strategies and an aggregated solution selection function to choose an optimal subset of clustering solutions. The results in the experiments on cancer gene expression profiles show that (i) FS-SSCE, DS-SSCE and MDS-SSCE are suitable for performing tumor clustering from bio-molecular data. (ii) MDS-SSCE outperforms a number of state-of-the-art tumor clustering approaches on most of the data sets.

People naturally escape from a place when unexpected events happen. Based on this observation, efficient detection of crowd escape behavior in surveillance videos is a promising way to perform timely detection of anomalous situations. In this paper, we propose a Bayesian framework for escape detection by directly modeling crowd motion in both the presence and absence of escape events. Specifically, we introduce the concepts of potential destinations and divergent centers to characterize crowd motion in the above two cases respectively, and construct the corresponding class-conditional probability density functions of optical flow. Escape detection is finally performed based on the proposed Bayesian framework. Although only data associated with nonescape behavior are included in the training set, the density functions associated with the case of escape can also be adaptively updated using observed data. In addition, the identified divergent centers indicate possible locations at which the unexpected events occur. The performance of our proposed method is validated in a number of experiments on crowd escape detection in various scenarios.

The binding of foreign peptides to MHC II molecules play a vital role in stimulating CD4+ helper T lymphocytes immune response. We present a pan-specific method, OWA-PSSM, that uses 35 pocket profiles generated by the TEPITOPE method to predict MHC II/peptide binding. Additionally, we develop a novel weighting approach, incorporated with OWA (Ordered Weighted Averaging) weights, for the construction of position specific scoring matrices (PSSMs) for 879 DRB alleles. This method is evaluated on four independent benchmark datasets and is demonstrated to outperform the TEPITOPEpan method. For the DRB alleles covered by TEPITOPE, the prediction performance of OWA-PSSM is comparable with TEPITOPE whereas OWA-PSSM can make prediction for up to 879 DRB alleles, while TEPITOPE can only perform prediction for 50 DRB alleles.

In order to perform successful diagnosis and treatment of cancer, discovering, and classifying cancer types correctly is essential. One of the challenging properties of class discovery from cancer data sets is that cancer gene expression profiles not only include a large number of genes, but also contains a lot of noisy genes. In order to reduce the effect of noisy genes in cancer gene expression profiles, we propose two new consensus clustering frameworks, named as triple spectral clustering-based consensus clustering (SC3) and double spectral clustering-based consensus clustering (SC2Ncut) in this paper, for cancer discovery from gene expression profiles. SC3 integrates the spectral clustering (SC) algorithm multiple times into the ensemble framework to process gene expression profiles. Specifically, spectral clustering is applied to perform clustering on the gene dimension and the cancer sample dimension, and also used as the consensus function to partition the consensus matrix constructed from multiple clustering solutions.Compared with SC3, SC2Ncut adopts the normalized cut algorithm, instead of spectral clustering, as the consensus function.Experiments on both synthetic data sets and real cancer gene expression profiles illustrate that the proposed approaches not only achieve good performance on gene expression profiles, but also outperforms most of the existing approaches in the process of class discovery from these profiles.

Feature selection is widely established as one of the fundamental computational techniques in mining microarray data. Due to the lack of categorized information in practice, unsupervised feature selection is more practically important but correspondingly more difficult. Motivated by the cluster ensemble techniques, which combine multiple clustering solutions into a consensus solution of higher accuracy and stability, recent efforts in unsupervised feature selection proposed to use these consensus solutions as oracles. However,these methods are dependent on both the particular cluster ensemble algorithm used and the knowledge of the true cluster number. These methods will be unsuitable when the true cluster number is not available, which is common in practice. In view of the above problems, a new unsupervised feature ranking method is proposed to evaluate the importance of the features based on consensus affinity. Different from previous works, our method compares the corresponding affinity of each feature between a pair of instances based on the consensus matrix of clustering solutions. As a result, our method alleviates the need to know the true number of clusters and the dependence on particular cluster ensemble approaches as in previous works. Experiments on real gene expression data sets demonstrate significant improvement of the feature ranking results when compared to several state-of-the-art techniques.

In this paper, we propose a crowd motion partitioning approach based on local-translational motion approximation in a scattered motion field. To represent crowd motion in an accurate and parsimonious way, we compute optical flow at the salient locations instead of at all the pixel locations. We then transform the problem of crowd motion partitioning into a problem of scattered motion field segmentation. Based on our assumption that local crowd motion can be approximated by a translational motion field, we develop a local-translation domain segmentation (LTDS) model in which the evolution of domain boundaries is derived from the Gâteaux derivative of an objective functional and further extend LTDS to the case of scattered motion field. The experiment results on a set of synthetic vector fields and a set of videos depicting real-world crowd scenes indicate that the proposed approach is effective in identifying the homogeneous crowd motion components under different scenarios.

MOTIVATION: RNA 3D motifs are recurrent substructures in an RNA subunit and are building blocks of the RNA architecture. They play an important role in binding proteins and consolidating RNA tertiary structures. RNA 3D motif searching consists of two steps: candidate generation and candidate filtering. We proposed a novel method, known as Feature-based RNA Motif Filtering (FRMF), for identifying motifs based on a set of moment invariants and the Earth Mover's Distance in the second step. RESULTS: A positive set of RNA motifs belonging to six characteristic types, with eight subtypes occurring in HM 50S, is compiled by us. The proposed method is validated on this representative set. FRMF successfully finds most of the positive fragments. Besides the proposed new method and the compiled positive set, we also recognize some new motifs, in particular a π-turn and some non-standard A-minor motifs are found. These newly discovered motifs provide more information about RNA structure conformation. AVAILABILITY: Matlab code can be downloaded from www.cs.cityu.edu.hk/~yingshen/FRMF.html CONTACT: [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Although there are a variety of approaches to solve multiobjective optimization problems, few of them makes systematic use of the neighborhood relationship between the candidate solutions observed during the search process to improve the final results. In this paper, a new evolutionary algorithm, referred to as the neighborhood knowledge-based evolutionary algorithm (NKEA), is proposed to solve the multiobjective optimization problem. NKEA not only takes into account the advantages of NSGA-II, and JGGA, such as the fast nondominated sorting algorithm and horizontal transmission of information in a candidate solution, but also exploits systematically the neighborhood knowledge acquired during the search process. Specifically, NKEA consists of three major stages: the direction learning stage, the mutual adaptation stage, and the self adaptation stage. NKEA not only uses the fast nondominated sorting algorithm to find the Pareto optimal solutions, but also adopts the elitist strategy to maintain the best individuals for the next generation based on this strategy. Two adaptive control functions in the mutual adaptation stage and the self adaptation stage are designed to adjust the respective contributions of coarse local search and fine local search, which allows NKEA to perform a more thorough local search. Finally, we introduce a new notion, known as the measure space, which integrates multiple measures, such as the convergence metric and the diversity metric, to evaluate the performance of the algorithm. The results of our experiments show that NKEA not only achieves good performance in a large number of multiobjective optimization problems, but also outperforms most of the state-of-the-art approaches in these problems.

Semantic similarity defined on Gene Ontology (GO) aims to provide the functional relationship between different biological processes, molecular functions, or cellular components. In this paper, a novel method, namely the Shortest Path (SP) algorithm, for measuring the semantic similarity on GO is proposed based on both the GO structure information and the term's property. The proposed algorithm searches for the shortest path that connects two terms and uses the sum of weights on the shortest path to compute the semantic similarity for GO terms. A method for evaluating the nonlinear correlation between two variables is also introduced for validation. Extensive experiments conducted on two public gene expression datasets demonstrate the overall superiority of SP method over the other state-of-the-art methods evaluated.

Adjusted Rand Index (ARI) is one of the most popular measure to evaluate the consistency between two partitions of data sets in the areas of pattern recognition. In this paper, ARI is generalized to a new measure, Adjusted Rand Index between a similarity matrix and a cluster partition (ARImp), to evaluate the consistency between a set of clustering solutions (or cluster partitions) and their associated consensus matrix in a cluster ensemble. The generalization property of ARImp from ARI is proved and its preservation of desirable properties of ARI is illustrated with simulated experiments. Also, we show with application experiments on several real data sets that ARImp can serve as a filter to identify the less effective cluster ensemble methods.

In this paper, we propose a novel 3-D model retrieval framework, which is referred to as hybrid 3-D model associative retrieval. Unlike the conventional 3-D model similarity retrieval approach, the query model and the models obtained by 3-D model hybrid associative retrieval have the following properties: They belong to different model classes and have different shape characteristics in general but are semantically related and preassembled in a certain associative group. For instance, given a furniture associative group { desk, chair, bed}, we may probably like to use a desk as a query model to search for a list of matching models, which belong to the chair or bed class. We consider the following possibilities: 1) there can be more than two classes in an association group and 2) different association groups might have different numbers of classes. The hybrid associative retrieval is performed in two stages: 1) to establish the relationship between different 3-D model categories with semantic associations, we propose three approaches based on neural network learning and 2) to address the aforementioned two conditions, we use a cyclic-shift scheme to partition different associative groups into two-class pairwise associative groups and then adopt two different strategies to combine the final retrieval results. Experiments by using different data sets demonstrate the effectiveness and efficiency of our proposed framework on the new hybrid associative retrieval task.

Within the framework of credibility theory, several fuzzy portfolio selection models have been researched such as mean-variance model, chance constrained programming model, entropy optimization model and so on. However, all of them are proposed in the forms of single-objective programming, and there is no investigation on the transaction costs between the new portfolio and the existing one. In this paper, a fuzzy multi-objective mean-variance-skewness model with transaction costs is presented. In order to solve this model, a hybrid intelligent algorithm is designed by integrating simulated annealing algorithm, relevance vector machine and fuzzy simulation techniques, where the relevance vector machine is used to approximate the expected value, variance and skewness of portfolio returns and the fuzzy simulation is used to generate the training data for relevance vector machine.

Constrained clustering has recently become an active research topic. This type of clustering methods takes advantage of partial knowledge in the form of pairwise constraints, and acquires significant improvement beyond the traditional un-supervised clustering. However, most of the existing constrained clustering methods use constraints which are selected at random. Recently active constrained clustering algorithms utilizing active constraints have proved themselves to be more effective and efficient. In this paper, we propose an improved algorithm which introduces multiple representatives into constrained clustering to make further use of the active constraints. Experiments on several benchmark data sets and public image data sets demonstrate the advantages of our algorithm over the referenced competitors.

A number of different approaches based on high-throughput data have been developed for cancer classification. However, these methods often ignore the underlying correlation between the expression levels of different biomarkers which are related to cancer. From a biological viewpoint, the modeling of these abnormal associations between biomarkers will play an important role in cancer classification. In this paper, we propose an approach based on the concept of Biomarker Association Networks (BAN) for cancer classification. The BAN is modeled as a neural network, which can capture the associations between the biomarkers by minimizing an energy function. Based on the BAN, a new cancer classification approach is developed. We validate the proposed approach on four publicly available biomarker expression datasets. The derived Biomarker Association Networks are observed to be significantly different for different cancer classes, which help reveal the underlying deviant biomarker association patterns responsible for different cancer types. Extensive comparisons show the superior performance of the BAN-based classification approach over several conventional classification methods.

In this paper, we propose a new evolutionary algorithm called nearest neighbor evolutionary algorithm (NNE) to solve the unconstrained optimization problem. Specifically, NNE consists of two major steps: coarse nearest neighbor evolutionary and fine nearest neighbor evolutionary. The coarse nearest neighbor evolutionary step pays more attention to searching the optimal solutions in the global way, while the fine nearest neighbor evolutionary step focuses on searching the best solutions in the local way. NNE repeats two major steps until the terminate condition is reached. NNE not only adopts the elitist strategy and maintains the best individuals for the next generation, but also considers the knowledge obtained in the searching process. The experiments demonstrate that (1) NNE achieves good performance in most of numerical optimization problems; (2) NNE outperforms most of state-of-art evolutionary algorithms, such as traditional genetic algorithm (GA), the jumping gene genetic algorithm (JGGA).

Although there exist a lot of approaches to solve constrained optimization problem, few of them makes use of the knowledge obtained in the searching process. In the paper, a new algorithm called nearest neighbor evolutionary algorithm (NNE) is proposed to solve the constrained optimization problem. NNE not only performs global search and local search in the searching process, but also considers the knowledge obtained in the searching process. NNE also avail itself of the elitist strategy and keeps the best individuals for the next generation. The results in the experiments show that NNE not only achieves good performance in a lot of constrained optimization problems, but also outperforms most of state-of-art approaches in most of constrained optimization problems, such as ASCHEA and SEMS.

In this paper, we propose a novel 3D human motion sequence retrieval method based on the similarity of the motion data distribution. First, for each motion sequence in the database, the Self-Organizing Maps (SOM) clustering algorithm is adopted to partition the frames into different classes to get the associated class reference vectors. Then given a query motion, Probabilistic Principal Component Analysis (PPCA) is applied to estimate the distribution of its data. We adopt two different approaches to model the query data. In the first one, we directly estimate the distribution of the original data. For the other one, we estimate the class reference vector’s distribution after training by SOM, instead of that of the original data. Both of these approaches model the data using a Gaussian distribution. Finally the similarity between the query example and the motion sequence in a database is measured using the Mahalanobis distance. Experimental results on the CMU database demonstrate that the proposed method achieves good performance.

A novel approach for 3D motion capture data retrieval based on the Hierarchical Self Organizing Map (HSOM) is proposed. Given a query motion sequence, our goal is to search for all the similar motions from a database. Specifically, a feature vector based on the distribution of the human motion data is first extracted from each motion sequence in the database. Then, Singular Value Decomposition (SVD) is applied to reduce the dimensionality of the feature vector. To improve the retrieval efficiency, a two-level indexing scheme based on the HSOM is constructed, in which the motion sequences are first partitioned with the reduced feature vectors at the top level and then at the lower level, the original feature vectors are adopted to classify the cluster associated with a parent node at the first level into sub-clusters. Finally, fuzzy search is implemented to traverse the index structure to search for similar motions. Experimental results show that our method can achieve good performance in terms of retrieval accuracy and efficiency.

In order to enable more effective image retrieval via keywords, automatic image annotation and categorization becomes an important problem in computer vision and content based image retrieval. Unfortunately, there exists a semantic gap between the low-level feature vectors and the high-level semantics or concepts. In this paper, we design a basic concept repertory to bridge this semantic gap. Specifically, a basic concept repertory, in the form of a dictionary, is first designed to store a set of classifiers, with each of these representing a concept and a set of rules which is used to distinguish concepts with similar characteristics. We propose a new classifier based on our proposed bagging-adaboosting ensemble (BAE) approach. The experiments demonstrate the good performance of our approaches.

Protein phosphorylation is an important step in many biological processes, such as cell cycles, membrane transport, apoptosis, and so on. We design a new classifier ensemble approach called Bagging-Adaboost Ensemble (BAE) for the prediction of eukaryotic protein phosphorylation sites, which incorporates the bagging technique and the adaboost technique into the classifier framework to improve the accuracy, stability and robustness of the final result. To our knowledge, this is the first time in which the ensemble approach is applied to predict phosphorylation sites. Our prediction system based on BAE focuses on five kinase families: CDK, CK2, MAPK, PKA, and PKC. BAE achieves good performance in six families, and the accuracies of the prediction system for these families are 84:7%, 87:4%, 85:5%, 85:2%, and 82:3% respectively.

Recently, researchers are paying more attention to 3D model classification due to its useful applications in multimedia, computer graphics, and so on. Although there exist a number of approaches to classify 3D models, few of them consider the prior knowledge during the process of 3D model classification. In this paper, we propose a new framework called knowledge based cluster ensemble which incorporates the prior knowledge of the dataset into the cluster ensemble framework to classify 3D models. The experiments show that knowledge based cluster ensemble framework works well on 3D human head model database.

In this paper we propose a new partial closure-based constrained clustering algorithm. We introduce closures into the partial constrained clustering and we propose a new measurement to order the importance of the constrained closures. Experiments on public datasets demonstrate the advantages of our algorithm over the standard Kmeans and two state-of-the-art constrained clustering algorithms.

In this paper, we propose a new algorithm called fuzzy cluster ensemble algorithm (FCEA) which integrates the fuzzy logic theory and traditional cluster ensembles for 3D head model classification. Specifically, FCEA consists of two parts: (i) data processing on the distributed locations and (ii) data fusion on the centralized location. In the distributed locations, data processing includes (i) extracting feature vectors from 3D head models, (ii) performing basic fuzzy clustering algorithm to obtain fuzzy membership matrix, while data fusion on the centralized location contains (i) creating a fuzzy cluster ensemble constructor by integrating different fuzzy membership matrices from the distributed locations, and (ii) obtaining the final results of 3D head model classification based on the fuzzy logic theory and the fuzzy cluster ensemble constructor. The experiments show that FCEA works well on 3D head model database.

Although there exist a lot of cluster ensemble approaches, few of them consider the prior knowledge of the datasets. In this paper, we propose a new cluster ensemble approach called knowledge based cluster ensemble (KCE) which incorporates the prior knowledge of the dataset into the cluster ensemble framework. Specifically, the prior knowledge of the dataset is first represented by the side information which is encoded as pairwise constraints. Then, KCE generates a set of cluster solutions by the basic clustering algorithm. Next, KCE transforms the pairwise constraints to the confidence factor of the cluster solutions. In the following, the new data matrix is constructed by considering all the cluster solutions and their corresponding confidence factor. Finally, the results are obtained by partitioning the consensus matrix. The experiments illustrate that (1) KCE works well on the real datasets; (2) KCE outperforms most of the state-of-art cluster ensemble approaches.

Due to the rapid development of motion capture technology, more and more human motion databases appear. In order to effectively and efficiently manage human motion database, human motion classification is necessary. In this paper, we propose an Ensemble based Human Motion Classification Approach (EHMCA). Specifically, EHMCA first extracts the descriptors from human motion sequences. Then, singular value decomposition (SVD) is adopted to reduce the dimensionality of all the feature vectors. In the following step, a cluster ensemble approach is designed to construct the consensus matrix from the feature vectors. Finally, the normalized cut algorithm is applied to partition the consensus matrix and assign the feature vectors into the corresponding clusters. Experiments on the CMU database illustrate that the proposed approach achieves good performance.

Pattern mining gains more and more attention due to its useful applications in many areas, such as machine learning, database, multimedia, biology, and so on. Though there exist a lot of approaches for pattern mining, few of them consider the local distribution of the data. In the paper, we not only design six challenge datasets related to the local patterns, but also propose a new pattern mining algorithm based on local distribution. Unlike traditional pattern mining algorithms, our new algorithm first creates a local distribution for each data point by a random approach. Then, the distribution curve of each data point is simulated by the sum of low frequency curves obtained by the wavelet approach. In the third step, the coefficients of these low frequency curves for each data point are clustered by the normalized cut approach. Finally, the patterns of the datasets are obtained by the new pattern mining algorithm. The experiments show that our new algorithm outperforms traditional unsupervised learning approaches, such as K-means, EM, spectral clustering algorithm (SCA), and so on, on these six new datasets.

Support vector machine (SVM) is a hot topic in many areas, such as machine learning, computer vision, data mining, and so on, due to its powerful ability to perform classification. Though there exist a lot of approaches to improve the accuracy and the efficiency of the models of SVM, few of them address how to eliminate the redundant data from the input training vectors. As it is known, most of support vectors distributes in the boundary of the class, which means the vectors in the center of the class are useless. In the paper, we propose a new approach based on Gaussian model to preserve the training vectors in the boundary of the class and eliminate the training vectors in the center of the class. The experiments show that our approach can reduce most of the input training vectors and preserve the support vectors at the same time, which leads to a significant reduction in the computational cost and maintains the accuracy.

3D point based head model classification gains more and more attention due to its useful application in mechanical engineering, computer vision and biology. In this paper, we propose a new approach to perform 3D point based head model classification. Each head model is represented by a Gaussian distribution with the constrained boundary. The constrained boundary is the minimal enclosing ellipsoid of 3D head model. In order to estimate the parameter of the Gaussian distribution, we propose the generalized minimax algorithm. The generalized minimax algorithm for parameter estimation saves a lot of computational cost when comparing with traditional minimax algorithm. The Mahalanobis distance is applied to measure the similarity between two Gaussian distributions with respect to two head models. The experiments show that the new approach works well during the process of 3D point based head model classification.

Theoretic study in this paper shows that we can obtain exact long-range contacts by adopting one classifier if the centers of sequence profiles of residue pairs for long-range contacts and non-long-range contacts are known. The adopted classifier, referred to as multiple conditional probability mass function classifier (MCPMFC), can find an optimized transformation of the variables for each of the classes and therefore resulting in K separate classifiers. As a result, about 44.48% long-range contacts are around at the sequence profile (SP) centre for long-range contacts and about 20.9% long-range contacts are correctly predicted when considering the top L/5 (L is the protein sequence length) predicted contacts and the residue pair with 24 apart. The highest cluster result gives us a clue that SP center should be a sound pathway to investigate contact map in protein structures.

MOTIVATION: Consensus clustering, also known as cluster ensemble, is one of the important techniques for microarray data analysis, and is particularly useful for class discovery from microarray data. Compared with traditional clustering algorithms, consensus clustering approaches have the ability to integrate multiple partitions from different cluster solutions to improve the robustness, stability, scalability and parallelization of the clustering algorithms. By consensus clustering, one can discover the underlying classes of the samples in gene expression data. RESULTS: In addition to exploring a graph-based consensus clustering (GCC) algorithm to estimate the underlying classes of the samples in microarray data, we also design a new validation index to determine the number of classes in microarray data. To our knowledge, this is the first time in which GCC is applied to class discovery for microarray data. Given a pre specified maximum number of classes (denoted as K(max) in this article), our algorithm can discover the true number of classes for the samples in microarray data according to a new cluster validation index called the Modified Rand Index. Experiments on gene expression data indicate that our new algorithm can (i) outperform most of the existing algorithms, (ii) identify the number of classes correctly in real cancer datasets, and (iii) discover the classes of samples with biological meaning. AVAILABILITY: Matlab source code for the GCC algorithm is available upon request from Zhiwen Yu.

In this paper, a novel 3D head model retrieval approach is proposed, in which only a single 2D face view query is required. The proposed approach will be important for multimedia application areas such as virtual world construction and game design, in which 3D virtual characters with a given set of facial features can be rapidly constructed based on 2D view queries, instead of having to generate each model anew. To achieve this objective, we construct an adaptive mapping through which each 2D view feature vector is associated with its corresponding 3D model feature vector. Given this estimated 3D model feature vector, similarity matching can then be performed in the 3D model feature space. To avoid the explicit specification of the complex relationship between the 2D and 3D feature spaces, a neural network approach is adopted in which the required mapping is implicitly specified through a set of training examples. In addition, for efficient feature representation, principal component analysis (PCA) is adopted to achieve dimensionality reduction for facilitating both the mapping construction and the similarity matching process. Since the linear nature of the original PCA formulation may not be adequate to capture the complex characteristics of 3D models, we also consider the adoption of its nonlinear counterpart, i.e., the so-called kernel PCA approach, in this work. Experimental results show that the proposed approach is capable of successfully retrieving the set of 3D models which are similar in appearance to a given 2D face view.

Recently, the detection of visual attention regions (VAR) is becoming more important due to its useful application in the area of multimedia. Although there exist a lot of approaches to detect visual attention regions, few of them consider the semantic gap between the visual attention regions and high-level semantics. In this paper, we propose a rule based technique for the extraction of visual attention regions at the object level based on real-time clustering, such that VAR detection can be performed in a very efficient way. The proposed technique consists of four stages: 1) a fast segmentation technique which is called the real time clustering algorithm (RTCA); 2) a refined specification of VAR which is known as the hierarchical visual attention regions (HVAR); 3) a new algorithm known as the rule based detection algorithm (RADA) to obtain the set of HVARs in real time, and 4) a new adaptive image display module and the corresponding adaptation operations using HVAR. We also define a new background measure which combines both feature contrast and the geometric property of the region to identify the background region, and a confidence factor which is used to extract the set of hierarchical visual attention regions. Compared with existing techniques, our approach has two advantages: 1) the approach detects the visual attention region at the object level, which bridges the gap between traditional visual attention regions and high-level semantics; 2) our approach is efficient and easy to implement

In this paper we present a novel method of using genetic algorithm (GA) to learn a graphical model which is used for human motion characterization. The modeling of human movements will involve a high dimensional joint probability density function. With this graphical model, the joint probability distribution can be decomposed into a number of low dimensional distributions which are represented as tree models and triangulated models. To automatically search for such a model from a database of cases is a NP-hard problem. We use GA to solve this problem, which can optimize both the ordering structure and the conditional independence relationship of the graphical model. The searched graphical models are used to classify different types of human motions. The experimental results demonstrate that, compared with a previous greedy search algorithm, the GA is more effective for optimization of the graphical model

EM is one of the popular algorithms which can be applied to skin segmentation. Due to the high computational cost of EM, traditional EM is difficult to apply to a large skin database. Inspired by the idea of subsampling, we integrate EM with incremental clustering and hierarchical clustering to estimate the parameters of mixture models. The algorithm first selects the samples by the incremental clustering approach and hierarchical clustering approach. Then, EM is applied to the sample set. The experiments show that the new EM algorithm works well in the skin database.

Given a time horizon parameter h and an object set O, predictive collision detection finds all the object pairsi, oj, ti> which will collide in the future time interval [t, t+h] (where 1lesi, jlesn, tiisin[t, t+h]). Although there are a number of state-of-the-an approaches to solve collision detection problems, predictive collision detection is addressed for the first time. In this paper, we propose a grid-based predictive collision detection algorithm (GPCD), which is a general technique for the efficient detection of the collision of object pairs in a future time interval. GPCD first determines a candidate list which stores the object pairs having a non-zero probability to collide in a future time. Then, GPCD achieves low running time based on two pruning strategies: (i) space intersection test and (ii) time intersection test. These two pruning strategies eliminate most of the false collision cases in an initial filtering phase. In the refinement phase, a bounding-volume tree is applied to refine the detection results. Our experiments show that GPCD works well for the purpose of predictive collision detection

EM algorithm is an important unsupervised clustering algorithm, but the algorithm has several limitations. In this paper, we propose a fast EM algorithm (FEMA) to address the limitations of EM and enhance its efficiency. FEMA achieves low running time by combining principal component analysis (PCA), a grid cell expansion algorithm (GCEA) and a hierarchical cluster tree. PCA and multi-dimensional grid are applied to find a set of "good" initial parameters for the EM algorithm, while the hierarchical cluster tree deals with the case where the cluster is concave by making use of a merging algorithm. The experiments indicate that FEMA outperforms EM by reducing 45% of the CPU time

Given a point-based 3D computer graphics model which is defined by a point set P(P={piisinR3}) and a desired reduced number of output samples Ns, the simplification approach finds a point set Pswhich (i) satisfies |Ps|=Ns(|Ps| is the cardinality of Ps) and (ii) minimizes the difference of the corresponding surface Ss(defined by Ps) and the original surface S(defined by P). Although a number of previous approaches have been proposed for simplification, most of them (i) do not focus on point-based 3D models, (ii) do not consider efficiency, quality and generality together. In this paper, we introduce an adaptive simplification method (ASM) which is an efficient technique for simplifying point-based complex 3D model. ASM achieves low running time by clustering the points locally based on the preservation of geometric characteristics. Finally, we analyze the performance of ASM and show that it outperforms most of the current state-of-the-art methods in terms of efficiency, quality and generality

This paper analyzes the long-range interactions, which plays a fundamental and important role in many biologic fields, between residues in protein using principal component analysis (PCA). Firstly, one angular coordinate system of long-range interaction regions is constructed conveniently. Afterwards, a matrix of the angular values of residues can be analyzed by principal component analysis technique. Projecting the angular matrix onto its eigenvectors, it can be found that the projection is to satisfy Boltzmann distribution. By analyzing the thermodynamic environment of the interaction region and scaling the interaction regions, it can be concluded that the distribution of long-range interactions may also be obtained and as a result applied in prediction of contact map.

Identifying protein-protein interaction sites is crucial for understanding of the principles of biological systems and processes, as well as mutant design. This paper describes a novel method that can predict protein interaction sites in heterocomplexes using information of evolutionary conservation and spatial sequence profile. A predictor was generated to distinguish the interface residues from protein surface region by radial basis neural networks, which is trained by expectation maximization algorithm. Based on a non-redundant data set of heterodimers consisting of 75 protein chains, the efficiency and the effectiveness of our proposed approach can be validated by a better performance such as the accuracy of 0.60, the sensitivity of 58.3% and the specificity of 59.9%.

In this paper, we propose a new approach for similarity matching in image retrieval based on approximate query processing in a hierarchical SOM. We first map the high dimensional input vectors to a low dimensional grid by a local membership function which preserves the relationships between the input vectors and their neighboring weight vectors. Then, we use a hierarchical tree to reduce the computation cost for finding the best match unit. Finally, we retrieve the k nearest neighbors of the query vector by an approximate query processing approach. The experiments show that the proposed approach works well on both synthetic datasets and image databases.

In this paper, we introduce a novel shape signature, called Volumetric Extended Gaussian Image (VEGI). It captures the volumetric distribution of a 3D mesh model along the latitude-longitude direction without conventional pose normalization and is translation and scaling invariant. Rotation invariance is accomplished by further calculating the spherical harmonic transform of this directional distribution. Due to the completeness and orthonormality properties of spherical harmonics, the VEGI also provides multi-resolution description of a model so that a multi-level indexing scheme based on Hierarchical Self Organizing Map (HSOM) can be established to improve retrieval efficiency. Experimental results show that our retrieval architecture has high discriminative power and outperforms many existing methods.

Estimating the selectivity of range query when applied to image database is an important stage for optimizing a image database query. In this paper, we propose a new approach to approximate the selectivity of a range query based on a probabilistic framework. Specifically, each image is represented as a sequence of states in a multi-dimensional space, where each dimension corresponds to a particular time stamp. We then associate a histogram with each dimension, and the histogram bins in each dimension correspond to specific states. A range query in the proposed probabilistic framework is the sum of the probabilities of observing a sequence of states within a hyper-sphere. A minimum bounding box and a maximum enclosed box are used to determine the upper bound and the lower bound of the selectivity of the range query. The experiments show that this probabilistic framework works well for an image database.

Personalized education (PE) emphasizes the importance of individual differences in learning. To deliver personalized e-learning services and content, PE encompasses the abilities of identifying and understanding individual learner's needs and competence so as to deploy appropriate learning pedagogy and content to enhance learning. In this paper, we introduce a hidden Markov model based classification approach to enable a multimedia e-learning system to characterize different types of users through their navigation or content access patterns. Our experiments show that the proposed approach is capable of assigning student users to their corresponding categories with high accuracies. The results of such classifications would find applications in adaptive user interface design, user profiling and as supportive tools in personalized e-learning

In this paper, a novel 3D head model retrieval framework is proposed. First, kernel PCA is adopted both to reduce the data dimension and to extract features for model characterization. Second, based on the derived features, a hierarchical indexing structure for 3D model database is constructed using the hierarchical self organizing map (HSOM). Third, an efficient search approach is presented based on the established indexing structure that requires only feature matching between the query model and a small number of SOM nodes. The main advantages of our approach include high retrieval precision due to the discrimination capacity of kernel PCA, and low computation cost due to the hierarchical indexing structure and data dimension reduction. In addition, the topology-preserving property of HSOM also facilitates the exploration of the model database with the possibility of further knowledge discovery.

In this paper, we propose a novel 3D head model retrieval approach in which the queries are 2D face views instead of less readily available 3D head models. The basic idea is to characterize the corresponding relations between 2D view feature and 3D model feature based on a machine learning approach. Thus the subsequent feature matching can be carried out in 3D feature space. As an effective solution to regression problems, relevance vector machine is used in this paper to establish an association between 2D and 3D features. Experimental results show that our proposed 2D query based method is comparable with the direct 3D query based one.

Recently, a number of approaches have been proposed which use compressed domain features for image retrieval and classification. While the main motivation of these approaches is to improve processing efficiency and reduce computational requirements, we propose a method which also aims at enhancing the content characterization capabilities of the compressed domain features in addition to efficiency improvement. We model the compressed domain feature values as random variables and approximate their associated probability mass functions as histograms. We then transform these histograms in such a way that the resulting classification rate based on these transformed histograms is improved. With a large number of possible transformations, we adopt an evolutionary strategy (ES) to search for the. optimal one. Experiments show that our proposed approach is able to obtain a better classification rate while the efficiency advantage of using compressed domain features is retained.

This paper proposed a partitioning neural root finder (PNRF) to find the minimum modulus (real or complex) roots of an arbitrary polynomial by imposing a minimum m order root moment (RM) into the constrained learning algorithm (CLA), where the constraint "the minimum m order RM" will ensure the minimum modulus root to be obtained. If the PNRF is recursively updated, the ordered roots from minimum modulus to maximum one can be achieved. Simulations show that this partitioning neural root-finding method is indeed able to find the minimum modulus root and the ordered roots of arbitrary polynomials readily and efficiently.

The reliable and efficient transmission of high-quality variable bit rate (VBR) video through the Internet generally requires network resources be allocated in a dynamic fashion. This includes the determination of when to renegotiate for network resources, as well as how much to request at a given time. The accuracy of any resource request method depends critically on its prediction of future traffic patterns. Such a prediction can be performed using the content and traffic information of short video segments. This paper presents a systematic approach to select the best features for prediction, indicating that while content is important in predicting the bandwidth of a video hit stream, the use of both content and available short-term bandwidth statistics can yield significant improvements. A new framework for traffic prediction is proposed in this paper; experimental results show a smaller mean-square resource prediction error and higher overall link utilization.

We address the problem of adaptive regularization in image restoration by adopting a neural-network learning approach. Instead of explicitly specifying the local regularization parameter values, they are regarded as network weights which are then modified through the supply of appropriate training examples. The desired response of the network is in the form of a gray level value estimate of the current pixel using weighted order statistic (WOS) filter. However, instead of replacing the previous value with this estimate, this is used to modify the network weights, or equivalently, the regularization parameters such that the restored gray level value produced by the network is closer to this desired response. In this way, the single WOS estimation scheme can allow appropriate parameter values to emerge under different noise conditions, rather than requiring their explicit selection in each occasion. In addition, we also consider the separate regularization of edges and textures due to their different noise masking capabilities. This in turn requires discriminating between these two feature types. Due to the inability of conventional local variance measures to distinguish these two high variance features, we propose the new edge-texture characterization (ETC) measure which performs this discrimination based on a scalar value only. This is then incorporated into a fuzzified form of the previous neural network which determines the degree of membership of each high variance pixel in two fuzzy sets, the EDGE and TEXTURE fuzzy sets, from the local ETC value, and then evaluates the appropriate regularization parameter by appropriately combining these two membership function values.

We propose a machine learning approach for characterizing the perceptual importance of particular regions in an image. A modular neural network architecture is adopted for encoding our usual notion of a perceptually important region in such a way that generalization of this knowledge to previously unseen images is possible. Specifically, users are allowed to specify examples of perceptually significant regions in images, which are then incorporated as training data for the network. An important characteristic of this approach is its provision for grouping distinct regions into a single perceptually significant area through the previous user guidance, unlike conventional segmentation approaches which partition the image into homogeneous regions without further specifying the relationship between these regions.

Image restoration is a difficult problem due to the ill-conditioned nature of the associated inverse filtering operation, which requires regularization techniques. The choice of the corresponding regularization parameter is thus an important issue since an incorrect choice would either lead to noisy appearances in the smooth regions or excessive blurring of the textured regions. In addition, this choice has to be made adaptively across, different image regions to ensure the best subjective quality for the restored image. We employ evolutionary programming (EP) to solve this adaptive regularization problem by generating a population of potential regularization strategies, and allowing them to compete under a new error measure which characterizes a large class of images in terms of their local correlational properties. The nonavailability of explicit gradient information for this measure motivates the adoption of EP techniques for its optimization, which allows efficient search at multiple error surface points. The adoption of EP also allows the broadening of the range of possible cost functions for image processing so that we can choose the most relevant function rather than the most tractable one for a particular image processing application.

In this paper, we investigate the feasibility of characterizing significant image edges using a model-based neural network with modular architecture. Instead of employing traditional mathematical models for characterization, we ask human users to select what they regard as significant features on an image, and then incorporate these selected edges directly as training examples for the network. Unlike conventional edge detection schemes where decision thresholds have to be specified, the current NN-based edge characterization scheme implicitly represents these decision parameters in the form of network weights which are updated during the training process. Experiments have confirmed that the resulting network is capable of generalizing this previously acquired knowledge to identify important edges in images not included in the training set. Most importantly, the current approach is very robust against noise contaminations, such that no re-training of the network is required when it is applied to noisy images.

We address the problem of adaptive regularization in image restoration by adopting a neural network learning approach. The local regularization parameter values are regarded as network weights which are then modified through the supply of appropriate training examples. We also consider the separate regularization of edges and textures due to their different noise masking capabilities, which in turn requires discrimination between these two feature types. A new edge-texture characterization (ETC) measure is derived and incorporated into a fuzzified form of the previous NN for the above purpose.

We present a computer vision system based on an integrated neural network architecture. In the low level vision subsystem, a network of networks-a biologically inspired network is used to recursively perform filtering, segmentation and edge detection; in the intermediate level and the high level, hierarchically structured arrays of self-organizing tree maps-extension of the popular self-organizing map are utilized to carry out image/feature analysis. The system has been applied to solve a number of real world problems. Some interesting and encouraging results are reported.

The authors introduce a near-lossless scheme for the compression of digital mammogram databases. In the scheme a self-organizing neural network is first used to separate the breast area from the background. Then an optimized JPEG coding algorithm is introduced to code the segmented breast area only. The combined segmentation/compression procedure is motivated by the massive storage requirement of mammograms. The proposed scheme exploits the fact that a large proportion of the mammogram consists of uninteresting background, and the breast region occupies only a small area. The experimental results have confirmed that the scheme is capable of extending beyond the compression limits of conventional transform coding methods and achieving a far lower bit rate. As a result, the current approach provides an efficient means for the storage and transmission of digital mammogram databases.