Domain adaptive retrieval aims to transfer knowledge from a labeled source domain to an unlabeled target domain, enabling effective retrieval while mitigating domain discrepancies. However, existing methods encounter several fundamental limitations: 1) neglecting class-level semantic alignment and excessively pursuing pair-wise sample alignment; 2) lacking either pseudo-label reliability consideration or geometric guidance for assessing label correctness; 3) directly quantizing original features affected by domain shift, undermining the quality of learned hash codes. In view of these limitations, we propose Prototype-based Semantic Consistency Alignment (PSCA), a two-stage framework for effective domain adaptive retrieval. In the first stage, a set of orthogonal prototypes directly establishes class-level semantic connections, maximizing inter-class separability while gathering intra-class samples. During the prototype learning, geometric proximity provides a reliability indicator for semantic consistency alignment through adaptive weighting of pseudo-label confidences. The resulting membership matrix and prototypes facilitate feature reconstruction, ensuring quantization on reconstructed rather than original features, thereby improving subsequent hash coding quality and seamlessly connecting both stages. In the second stage, domain-specific quantization functions process the reconstructed features under mutual approximation constraints, generating unified binary hash codes across domains. Extensive experiments validate PSCA's superior performance across multiple datasets.

The increasing prominence of short video platforms has positioned them as a primary channel for public awareness of current events, while also facilitating the widespread dissemination of fake news, thus highlighting the critical need for automated detection technologies. In contrast to fake news confined to text and images, short video news encompasses multiple modalities and extensive information, presenting heightened challenges. Most existing research emphasizes the analysis of news content or user comments alone, while overlooking the crucial role of publishers, leading to poor model performance when handling fake news lacking obvious false signals. Therefore, we propose a Publisher Profiling Module to identify new false signals. To enable a more comprehensive detection of misinformation, we design a Multi-View Aggregation (MVA) model, simultaneously evaluating news from three distinct perspectives: sentiment analysis, content understanding, and publisher profiling. Late fusion is applied at the decision level to leverage the complementary strengths of these perspectives, addressing the limitations of single-view methods. Our experiments conducted on the FakeSV and FVC datasets demonstrate the superior performance of the proposed method.

Multi-view multi-label classification aims to utilize the rich information contained in multiple views for accurate classification. However, in real-world applications, its performance is often severely constrained by the concurrent missingness of both views and labels. To address this problem, this paper first targets the drawback of representation degradation in traditional feature disentanglement methods caused by strong consistency constraints and proposes a soft consistency constraint. This constraint not only effectively aligns the shared information and maximally avoids the compression of information beneficial to the classification task, but it also enhances the aggregation effect of high-quality representations on other representations. Furthermore, to address the coarse-grained problem of traditional fusion strategies, we designed a quality assessment network that achieves instance-level dynamic weighted fusion in a data-driven manner. Extensive experiments on multiple benchmark datasets demonstrate that our method achieves state-of-the-art performance in both incomplete and complete data scenarios, showcasing its robustness and generality.

Multi-view diabetic retinopathy (DR) grading has achieved remarkable performance by capturing more comprehensive pathological features than single-view methods. However, complete multi-view fundus images are often difficult to obtain in clinical practice, and the performance degrades significantly when fewer views are available. To overcome this limitation, we propose the first incomplete multi-view DR grading framework, aiming to provide accurate diagnosis regardless of the number of available views. It introduces two novel modules. First, cross-view spatial correlation attention (CSCA) captures region correlations across views, automatically identifying and fusing diagnostically relevant spatial features to improve feature representation. Second, self-supervised mask consistency learning (SMCL) formulates a novel pretext task of missing-view information reconstruction by strategically masking inter- and intra-view regions, enabling the model to infer complete features from incomplete views. Benefiting from CSCA and SMCL, our method enhances structural feature consistency across views and effectively compensates for missing information during DR grading. Extensive experiments demonstrate that our method achieves state-of-the-art grading performance, particularly under realistic conditions where some views are unavailable.

Semi-supervised learning can leverage both labeled and unlabeled samples simultaneously to improve performance. However, existing methods often present the following issues: (1) The emphasis of learning is put on either the similarity structures or the regression losses of data, neglecting the interaction between them. (2) The similarity structures among boundary samples might be unreliable, which misleads label propagation and impairs the performance of models on out-of-sample data. (3) They often involve the inverses of high-order matrices, making them inefficient in computation. To overcome these issues, we propose a scalable semi-supervised learning framework with Discriminative Label Propagation and Correction (DLPC), which collaboratively exploits the regression losses and similarity structures of data. Particularly, each sample is projected onto the independent class labels associated with nonnegative adjustment vectors rather than the propagated labels, such that the distances between samples from different classes are naturally enlarged, making regression losses more effective for boundary samples. Benefiting from this, the regression losses can guide the propagation of labels in boundary areas. Thus, the label information is first propagated through dynamically optimized graph structures and then corrected by the regression losses, effectively improving the quality of labels and facilitating feature projection learning. Furthermore, an accelerated solution has been developed to reduce the computational costs of DLPC on sample scales, thereby making it scalable to relatively large-scale problems. Moreover, the proposed DLPC can not only be applied to single-view scenarios but also extended to multi-view tasks. Additionally, an optimization strategy with fast convergence has been presented for DLPC, and extensive experiments demonstrate the effectiveness and superiority of DLPC over state-of-the-art competitors.

Multi-view data encompasses various data types, including multi-feature, multi-sequence, and multi-modal data. Multi-view multi-label classification aims to leverage the rich semantic information contained in multiple views to achieve enhanced multi-label classification performance. In practical applications, the absence of views and labels poses a significant challenge to multi-view multi-label classification tasks. Premised on the assumption that shared semantic information across multiple views is sufficient to support the downstream task, we propose CTRL, a novel incomplete multi-view multi-label classification framework to address the multi-view learning challenge on the data with partially missing views and missing labels in this paper. The core mechanism of CTRL lies in learning a high-purity, low-redundancy condensed representation that adequately captures the essential information of the original data. Specifically, we design a new objective loss to enhance the semantic information of shared cross-view within the joint representation learning process while simultaneously suppressing intra-view redundant information that is irrelevant to the downstream task. This enables CTRL to extract task-relevant representations even when views are incomplete. Furthermore, we employ the Beta Evidential Neural Network to model the label distribution. This network is then integrated with Dempster-Shafer theory, enabling our model to perform label-level classification uncertainty estimation. This also allows us to use the estimated uncertainty and belief mass to create high-reliability pseudo-labels, resulting in further gains in model performance. Experimental results on multiple benchmark datasets demonstrate the superior performance of our proposed model in terms of accuracy, robustness, and reliability.

Fine-tuning pre-trained vision-language models (VLMs) has shown substantial benefits in a wide range of downstream tasks, often achieving impressive performance with minimal labeled data. Parameter-efficient fine-tuning techniques, in particular, have demonstrated their effectiveness in enhancing downstream task performance. However, these methods frequently struggle to generalize to out-of-distribution (OOD) data due to their reliance on non-causal representations, which can introduce biases and spurious correlations that negatively impact decision-making. Such spurious factors hinder the model's generalization ability beyond the training distribution. To address these challenges, in this paper, we propose a novel causal intervention-based prompt tuning method to adapt VLMs to few-shot OOD generalization. Specifically, we leverage the front-door adjustment technique from causal inference to mitigate the effects of spurious correlations and enhance the model's focus on causal relationships. Built upon VLMs, our approach begins by decoupling causal and non-causal representations in the vision-language alignment process. The causal representation that captures only essential semantically relevant information can serve as a mediator variable between the input image and output label, mitigating the biases from the latent confounder. To further enrich this causal representation, we propose a novel text-based diversity augmentation technique that uses textual features to provide additional semantic context. This augmentation technique can enhance the diversity of the causal representation, making it more robust and generalizable to various OOD scenarios. Experimental results across multiple OOD datasets demonstrate that our method significantly outperforms existing approaches, achieving state-of-the-art generalization performance.

Currently, an increasing number of researchers are focusing on partial multiview incomplete multilabel learning. However, many methods generally integrate features from multiple views via an average weighting strategy, which overlooks the potential mismatch between the contribution of each view and their assigned fusion weights and thus generates unreliable fused features. To address this issue, we propose a novel uncertainty-driven reliable dynamic fusion framework for partial multiview incomplete multilabel learning. Unlike existing methods, the proposed uncertainty-driven reliable sample-level dynamic fusion module operates on the principle that samples exhibiting greater uncertainty possess fewer reliable features. This module evaluates the uncertainty of each sample and, in turn, estimates the reliability of features with the uncertainty of sample judgement, thereby obtaining reliable weights to guide the information fusion of multiple views. Furthermore, many existing approaches for handling incomplete multilabel scenarios typically concentrate on the information from annotated labels, neglecting the potential information of unknown tags. To bridge this gap, we incorporate an innovative pseudolabelling strategy that effectively identifies trustworthy pseudolabels that correspond to those unannotated uncertain labels, thereby adding additional supervisory information to assist model training. Moreover, we also devise a feature masking strategy to further augment the encoder's representation learning capabilities. The experimental results across five datasets demonstrate that our method outperforms current state-of-the-art methods.

As a prominent research topic, multi-view multi-label classification (MvMlC) aims to assign multiple labels to samples by integrating information from various perspectives. However, in real-world scenarios, MvMlC frequently faces the learning challenge of data with missing views and labels, typically resulting from sensor malfunctions, or the costly and time-consuming process of manual annotation. In addition, learning robust representations that are both consistent across views and specific to individual views remains a challenge. To address these issues, we propose a novel double incomplete multi-view multi-label classification framework based on Disentangling Consistent and Specific Information (DCSI). Specifically, we employ a dual-channel encoder with identical architecture but distinct objectives to extract cross-view consistent information and view-specific unique information from all views, respectively. Meanwhile, a view discriminator is constructed to decouple these two types of information, facilitating the extraction of pure consistent and specific information. Moreover, we meticulously design fusion strategies tailored to each representation type. Regarding consistent representations, we propose a dynamic-confidence-aware fusion mechanism that assesses the reliability of each view's representations in relation to the classification task, enabling the model to prioritize information from trustworthy representations. For specific representations, in light of their complementary rather than redundant property, we suggest treating such representations from each view equally to ensure fairness. Through experimental validation on five datasets, the results demonstrate that our method outperforms existing state-of-the-art methods.

Unmanned aerial vehicle (UAV) object detection is essential for applications such as surveillance, agriculture, and disaster response. However, UAV imagery often contains small, dense, and occluded objects, posing challenges for existing methods. To address these challenges, we propose DeCenter, a novel Density-Center Guided Perception Enhancement framework for UAV object detection. DeCenter is composed of two key modules that jointly enhance the perception of small and crowded objects. First, the Density-Guided Object Center Heatmap Generator (DOCHG) adaptively generates Gaussian kernel-based heatmaps according to local density information, guiding the model to emphasize central neighborhoods of objects in crowded regions. This mechanism reduces overlaps between adjacent instances and alleviates missed detections under occlusion. Second, the Density-Center Feature Enhancement module (DCFE) integrates complementary cues from density features and object centers, adaptively balancing region-level object distribution with fine-grained localization. By fusing these signals, DCFE enhances the quality of feature representations, making them more discriminative for dense small objects while suppressing background noise. Experimental results on VisDrone and UAVDT datasets show that DeCenter achieves competitive overall accuracy with clear improvements in detecting dense small objects, offering an effective solution for UAV object detection. The code will be available at https://github.com/bluuzzz/decenter.

Diffusion models have been used extensively for hyperspectral and multispectral image fusion; however, their intrinsic hallucination phenomenon frequently results in a loss of high-frequency details in the fused images. To address this limitation, this paper proposes a fusion method that integrates a differential attention mechanism with a two-stage conditional diffusion model. The proposed method leverages differential attention to compute the difference between two independent attention maps, thereby producing sparser and more focused attention representations. In addition, the two-stage conditional injection strategy is implemented to realize precise control over the image generation process. In the first stage, feature-level linear modulation via affine transformation is applied within the encoder to maintain global structural consistency. Then, in the second stage, wavelet features extracted from the conditioning images are injected into the decoder to facilitate the restoration of fine-grained details. Extensive validation experiments on the CAVE, Harvard, Pavia Center and Chikusei datasets verify the effectiveness of the proposed method. Compared with numerous state-of-the-art approaches, our method consistently achieves superior performance across key evaluation metrics. On the CAVE dataset, the ERGAS and RMSE metrics improved by 2.16% and 5.01%, and these metrics increased by 1.78% and 1.38% on the Pavia Center dataset, respectively. The code will be available at https://github.com/Ruijie2580/DifferentialDiff.

Deep multi-view clustering aims to exploit the rich semantic information contained in heterogeneous multi-view data to uncover the underlying relationships among samples. However, existing deep multi-view clustering models often overlook inter-cluster separability and the effective integration of semantic information across views, resulting in insufficient feature discriminability and consequently limited clustering performance. To address the above issues, this paper proposes a novel deep multi-view clustering method via cluster-semantic guidance. We separate clusters to enhance inter-cluster discriminability, while incorporating a knowledge distillation mechanism to ensure cluster stability and facilitate the learning of clustering-friendly representations. Furthermore, by aggregating sample-level semantic information, the model is guided to follow a cluster-oriented learning strategy that promotes the extraction of discriminative features, thereby strengthening the sample representation capability. Our method effectively learns discriminative and clustering-friendly representations, guiding the model to acquire distinctive feature embeddings from a cluster-oriented perspective. Our comprehensive experiments across datasets of varying scales confirm the model's effectiveness, showing superior clustering performance over existing state-of-the-art methods.

Human motion prediction is a key task in computer vision and human-robot interaction, which has received much attention in recent years. However, existing approaches suffer from two issues: 1) They typically rely only on complete data and overlook real-world challenges such as missing observations. 2) Recent works fail to capture the diverse relations among body parts in different action categories, which limits their prediction performance. To address the above problems, we propose a novel Incomplete human Motion Prediction method through motion Re covery and Structure-Semantic fusion (IMPRESS). Specifically, for motion recovery, we introduce a wavelet-based self-attention module. It captures motion details from high-frequency features and extracts global trends from low-frequency components. To enhance the relations among different body parts, we design a structure-semantic fusion graph convolutional network. Moreover, we employ a dual-channel sliding window attention mechanism to capture motion periodicity, enabling smoother predictions. Extensive experiments on two benchmark datasets (Human3.6M, CMU-MoCap) demonstrate that IMPRESS achieves state-of-the-art average prediction performance under both complete and incomplete observations.

Accurate medical image segmentation plays a vital role in clinical diagnostics by facilitating the precise delineation of anatomical structures and pathological regions. However, the performance of existing segmentation methods is often constrained by the scarcity of high-quality annotated datasets, as manual labeling is both labor-intensive and reliant on domain-specific expertise. To address this limitation without requiring additional annotations, we propose a novel multimodal segmentation framework that leverages medical text annotations as an auxiliary modality to complement visual information. In particular, our approach introduces a learnable encoding strategy for joint distribution modeling of image and text, which enables discriminative fusion and effectively suppresses cross-modal redundancy. Moreover, we innovatively design a frequency-domain prompt encoder based on the discrete wavelet transform (DWT) to capture multi-frequency features, thereby significantly enhancing the model's ability to delineate fine-grained boundaries. Overall, our framework integrates cross-attention for effective cross-modal interaction, employs joint distribution modeling to enable discriminative and redundancy-reduced multimodal fusion, and incorporates auxiliary supervision to strengthen the learning of task-relevant features. Extensive experiments on nine public datasets across three clinical tasks-including cell, lung infection, and polyp segmentation-demonstrate that our method achieves competitive segmentation performance while maintaining favorable computational efficiency. Comprehensive ablation studies and feature distribution visualizations further validate the effectiveness and robustness of our proposed components. The code will be made publicly available at https://github.com/chenpeng052/MDFP.

Partially Relevant Video Retrieval (PRVR) aims to retrieve videos that match a given textual query only partially. This task is inherently challenging due to the modality gap between text and video, which is further exacerbated by the partial semantic correspondence between linguistic descriptions and visual content. To address these challenges, we propose a bidirectional cross-modal alignment mechanism that collaboratively optimizes both visual and textual modalities. In the visual modality, a major difficulty lies in the absence of visual cues that directly correspond to textual semantics, limiting the model's ability to align visual representations with textual meanings under unsupervised conditions. To overcome this issue, we construct a semantic-visual association library, which stores paired visual and textual features with semantic annotations. During training, the model dynamically retrieves the most semantically similar visual samples from this library based on the current visual feature vector. These retrieved samples, preliminarily associated with semantics via cross-modal matching, are used to form dynamic anchors that guide visual representation learning. By leveraging these enriched visual features, the model progressively refines the visual representations to achieve better alignment with the corresponding textual inputs, thereby enhancing cross-modal consistency. In the textual modality, we enhance textual representations by integrating semantically aligned visual features selected from the same association library, further narrowing the modality gap. Extensive experiments on benchmark datasets under partial semantic correspondence scenarios demonstrate that our method achieves state-of-the-art performance. The source code of the paper is available at https://github.com/cyanlll/BOA.

Zero-shot anomaly detection (ZSAD) is a challenging task that aims to detect anomalies in images without any prior knowledge of the anomaly classes. This task is especially difficult because anomalies are rare, diverse, and often manifest differently across domains, making it hard for models to generalize when training data is scarce or unavailable. Recently, vision-language models (VLMs), such as CLIP, have shown great potential in ZSAD, but they often struggle to adapt to unseen domains due to the lack of domain-aware knowledge. To address these challenges, we propose the Domain Adaptation CLIP (DACLIP), a novel approach that adapts domain-aware knowledge to the VLM. Specifically, DACLIP leverages a Domain-Aware Knowledge Adaptation (DAKA) strategy to enhance CLIP for ZSAD across different domains. The DAKA strategy comprises multiple experts that specialize in target domains, enabling the model to dynamically select and combine specialized experts tailored to anomaly characteristics, thus improving its ability to generalize and detect a wide range of anomalies. Furthermore, we introduce learnable domain-aware prompts that are jointly learned by and injected into both the CLIP encoders (visual and text) and the DAKA modules. This dual-pathway learning enables the model to capture domain-specific features at multiple levels of the architecture, allowing for more effective adaptation to new domains and anomaly types. We evaluate our approach on several benchmark datasets spanning industrial and medical domains. Extensive experiments demonstrate that DACLIP consistently outperforms state-of-the-art methods in ZSAD, achieving significant improvements in both image-level and pixel-level anomaly detection tasks.

Reconstruction-based methods have achieved excellent performance in anomaly detection. Diffusion models are considered highly suitable for anomaly detection tasks due to their strong ability in reconstruction. Nevertheless, diffusion-based models require the reconstruction of noise features, which may lack the capacity for fine-grained feature reconstruction and fail to provide adequate semantic information for reconstruction guidance. To solve the aforementioned problems, this paper proposes a Fine-Grained Enhancement Convolutional Diffusion Transformer Anomaly Detection (FECDTAD) framework for multi-class anomaly detection. The core model of the proposed framework is the Fine-Grained Enhancement Convolutional Denoising Transformer (FECDT), which employs the diffusion transformer paradigm. To enhance fine-grained reconstruction in the diffusion process, the FECDTAD adopts a series of feature information fusion strategies. Specifically, to enhance both fine-grained perception and global understanding, the FECDT model employs a simple feature fusion module to integrate shallow-level and deep-level features extracted from a pre-trained vision transformer. To enhance the capacity for fine-grained feature reconstruction, the FECDT integrates local and global information via a CNN-Transformer architecture. Moreover, to provide guidance for the reconstruction of anomalous areas, semantic information is propagated into the FECDT through a Cross-Attention module. Experimental results demonstrate that the proposed method is effective and can surpass the state-of-the-art methods.

Multimodal perception and fusion play a vital role in uncrewed aerial vehicle (UAV) object detection. Existing methods typically adopt global fusion strategies across modalities. However, due to illumination variation, the effectiveness of RGB and infrared modalities may differ across local regions within the same image, particularly in UAV perspectives where occlusions and dense small objects are prevalent, leading to suboptimal performance of global fusion methods. To address this issue, we propose an adaptive fine-grained fusion network for multimodal UAV object detection. First, we design a local feature consistency-based modality fusion module, which adaptively assigns local fusion weights according to the structural consistency of high-response regions across modalities, thereby enabling more effective aggregation of object-relevant features. Second, we introduce a mutual information-guided feature contrastive loss to encourage the preservation of modality-specific information during the early training phase. Experimental results demonstrate that the proposed method effectively addresses the issue of object occlusion in UAV perspectives, achieving state-of-the-art performance on multimodal UAV object detection benchmarks. Code will be available at https://github.com/lingf5877/AFFNet.

Recent advances in multi-view fundus imaging show great promise for automated diabetic retinopathy (DR) grading. However, mainstream end-to-end CNN/Transformer pipelines rely on striding or tokenization that compresses spatial detail, causing small, low-contrast lesions (e.g., microaneurysms) to be under-represented and creating performance ceilings. Prior efforts have mitigated this by incorporating external lesion- or vessel-level annotations into models. However, such labels are costly to acquire, break the end-to-end training, and make performance over-reliant on the annotation quality. To reduce dependence on expensive annotations, we propose an end-to-end framework that generates lesion proposals on the fly during training and inference, providing self-derived cues for grading. First, we introduce a Grade-Activated Lesion Proposal (GALP) module that derives grade-conditioned evidence maps (GEMs) from stage-wise auxiliary classifiers and selects the top-K high-evidence regions per view as lesion proposals. Second, we propose a Cross-View Lesion Expert Guided Regional Fusion (LGRF) module, which selectively activates experts for a view's lesion proposals based on contextual guidance from other views, ensuring that only the most relevant feature extractors contribute to fusion. Experimental results on two multi-view DR datasets show that our method matches or surpasses strong baselines without external annotations, confirming that self-generated proposals can substantially reduce annotation needs.

Unrestrained palmprint recognition refers to a comprehensive identity authentication technology, that performs personal authentication based on the palmprint images captured in uncontrolled environments, i.e., smartphone cameras, surveillance footage, or near-infrared scenarios. However, unrestrained palmprint recognition faces significant challenges due to the variability in image quality, lighting conditions, and hand poses present in such settings. We observed that many existing methods utilize the subspace structure as a prior, where the block diagonal property of the data has been proved. In this paper, we consider a unified learning model to guarantee the consensus block diagonal property for all views, named high-confident block diagonal analysis for multi-view palmprint recognition (HCBDA_MPR). Particularly, this paper proposed a multi-view block diagonal regularizer to guide that all views learn a consensus block diagonal structure. In such a manner, the main discriminant features from each view can be preserved while the learning of the strict block diagonal structure across all views. Experimental results on a number of real-world unrestrained palmprint databases proved the superiority of the proposed method, where the highest recognition accuracies were obtained in comparison with the other state-of-the-art related methods.

As a basic machine learning task, Multi-View Classification (MVC) has garnered considerable attention and achieved great success. However, the existing MVC methods, especially late fusion style ones still suffer from some problems: 1) hidden valuable information is not well exploited; 2) a lack of interaction before decision making. To address these problems, we propose a novel framework named ”TrashtoTreasure” that leverages mutual information to effectively exploit hidden valuable information. Specifically, the framework explicitly disentangles multi-view information into ”useful” components and ”trash” (noisy) components, and further extracts potentially valuable ”treasure” information from the ”trash”components of all views. Additionally, we design a tailored objective function that facilitates the effective separation of ”useful” and ”trash” components, as well as the synergistic extraction of ”treasure” information. This function guides model optimization through triple mutual information constraints. Experimental results on synthetic data and several real-world data sets verified the effectiveness and superiority of the proposed method. The fresh perspective offered by this article may inspire more interesting exploration in this direction. The codes are available athttps://github.com/jiezhang054/TrashToTreasure.

Spatial Transcriptomics offers unprecedented opportunities to explore tissue architecture by capturing gene expression with spatial context. However, effectively learning discriminative and spatially smooth representations for accurate spatial domain identification remains a significant challenge. To address this, we propose CSMVL, a multi-view representation learning framework to learn high-quality spot representations by synergistically enhancing both discriminability and spatial continuity. CSMVL introduces a cluster structure learning strategy that guides cell representations within the same domain toward their cluster center while simultaneously separating distinct cluster centers, thereby improving intra-domain compactness and inter-domain separability. Furthermore, graph smoothness regularization is introduced to ensure that representations of spatially adjacent cells within the same domain transition smoothly, reflecting the inherent spatial continuity of biological tissues. Extensive experiments on public ST datasets demonstrate CSMVL's superiority, achieving an average ARI of 71.64% and NMI of 73.43%, outperforming existing state-of-the-art methods

Multi-modal learning is extensively applied to diagnose brain diseases such as epilepsy and Alzheimer's disease. However, incomplete multi-modal data, where some modalities are unavailable or difficult to collect, limits the effectiveness of conventional methods. Additionally, existing approaches often overlook semantic relationships between neighbors with the same-label and latent information in missing modalities. To address these challenges, we propose an adjacent-aware distillation recovery framework designed for incomplete multi-modal learning, with a focus on diagnosing representative brain diseases, i.e. epilepsy and Alzheimer's disease. The key novelty of our framework lies in its joint design of adjacent-aware modality recovery and multi-modal representation learning in a single end-to-end pipeline. Specifically, we introduce a label-guided adjacent-aware recovery module that uses a self-attention mechanism to exploit neighbor semantics and generate distribution-consistent features for high-quality modality reconstruction. The recovered features are then refined through a knowledge distillation pathway into a modality generator, enhancing generalization under severe data incompleteness. For multi-modal representation learning, the recovered modality information is fused with the original incomplete information to enhance feature extraction and representation. Extensive experiments demonstrate the effectiveness of our method in diagnosing epilepsy and Alzheimer's disease.

Partial Multi-Label Learning (PML) deals with a practical scenario where each instance is associated with a set of candidate labels, among which only a subset corresponds to the ground-truth labels while the others are unrelated. Existing PML methods typically employ label association recovery as a structural disambiguation strategy to identify credible labels. However, these methods attempt to recover associations directly from candidate labels, which leads to unreliable disambiguation due to the distorted label structures. To this end, this paper proposes a novel PML method via dual label association recovery (PML-DLAR).The essential strategy is to first eliminate the spurious correlations in label space before recovering dual label associations. Specifically, a biorthogonal transformation is employed to decouple the instance-level and label-level association structures affected by noisy labels. Subsequently, reliable instance-level associations are reconstructed through global geometric structure alignment between feature and pseudo-label spaces. Finally, class specific feature representations are constructed through class prototypes to guide label-level semantic association recovery in label space. Comprehensive experiments validate the superior performance of PML-DLAR over state-of-the-art methods.

Semi-supervised learning (SSL) offers a promising solution to the challenge of learning from limited labeled data by leveraging the potential of unlabeled data, thus circumventing the need for costly labeling efforts. However, common SSL methods often encounter domain shifts in many real-world scenarios, where class distribution is imbalanced. In order to make machine learning more robust to imbalanced datasets, it is imperative to ensure that consistent representations are learned for each class, regardless of the amount of data available. Therefore, we propose a straightforward yet effective kernel function mapping strategy to align the representations of each class in an infinite-dimensional space. Specifically, we employ a Gaussian kernel function to map the representations of unlabeled data to the centroids of labeled data, enabling similarity comparisons in the infinite-dimensional space. In this way, we are able to refine the predicted pseudo-labels at the representation level. To better handle class imbalance, we note that it is common to obtain a high recall but low precision for the majority classes and a high precision but low recall for the minority classes. A selective strategy is adopted for predictions corrected for the majority classes while maintaining confidence in the pseudo-labels assigned to the minority classes. Extensive evaluations on various benchmarks and training settings validate the superior performance of the proposed method compared to the existing relevant state-of-the-art approaches.

The acquisition of expert-annotated data remains a critical bottleneck for medical image segmentation, thereby constraining the clinical applicability of highly accurate models. Crucially, despite this scarcity of labeled data, the intrinsic homogeneity in human anatomy across the cohort provides a fundamental basis (or: a promising leverage point) for enhancing model generalization and training efficiency by exploiting inter-instance anatomical complementarity. In this study, we propose a novel semi-supervised approach for medical image segmentation that fully exploits this inter-instance complementarity. The proposed model operates at two levels, integrating a sophisticated copy-paste augmentation module (CPAM) and a trainable region calibration mechanism (TRCM) within the simple mean teacher (MT) framework. Specifically, CPAM is a carefully designed copy-paste strategy that facilitates the exchange of informative regions between samples, thereby enhancing the diversity and robustness of the training data. TRCM leverages the predictions from labeled regions to guide and calibrate the trainable regions in unlabeled data. The calibrated regions typically yield high-quality pseudo-labels, which effectively improve model training. CPAM and TRCM work synergistically, complementing each other to enhance model performance. Experiments on diverse medical image datasets-including LA, ACDC, BraTS2019, and Pancreas-NIH-covering both MRI and CT modalities demonstrate the robust efficacy of our proposed model. In settings with limited annotated data, the model consistently outperforms current state-of-the-art methods across multiple evaluation metrics. The code is available at https://github.com/shuaiaihang/shuaiAIMedcalLab.

Medical image segmentation provides useful information about the shape and size of organs, which is beneficial for improving diagnosis, analysis, and treatment. Despite traditional deep learning-based models can extract domain-specific knowledge, they face a generalization bottleneck due to the limited embedded knowledge scope. Vision foundation models have been demonstrated to be effective in extracting generalizable knowledge, but they cannot extract domain-specific knowledge without fine-tuning. In this work, we propose a novel multi-view evidential learning-based framework, which can extract both domain-specific and generalizable knowledge from multi-view features by combining the advantages of traditional and vision foundation models. Specifically, a novel multi-view state space model (MV-SSM) is designed to extract task-related knowledge while removing redundant information within multi-view features. The proposed MV-SSM utilizes Mamba, a state space model, to model cross-view contextual dependencies between domain-specific and generalizable features. Additionally, evidential learning is adopted to quantify the segmentation uncertainty of the model for boundary. In special, variational Dirichlet is introduced to characterize the distribution of the result probabilities, parameterized with collected evidence to quantify uncertainty. As a result, the model can reduce the segmentation uncertainties of boundaries by optimizing the parameters of the Dirichlet distribution. Experimental results on three datasets show that our method obtains superior segmentation performance.

The objective of referring expression comprehension (REC) is to accurately identify the object in an image described by a given expression. Existing REC methods, including transformer-based and graph-based approaches among others, have shown robust performance in REC tasks. In this study, we present a groundbreaking framework named DiffusionREC for REC task. This framework reimagines the REC task as a text guided bounding box denoising diffusion process, through which noisy bounding boxes are refined and distilled to pinpoint the target box. Throughout the training process, the bounding box of the target object diffuses from its ground-truth position towards a random distribution. Simultaneously, a filtering-based object decoder is introduced to reverse this diffusion of noise, conditional on the provided expression, the result from previous denoised step and the interaction between the expression and the image. At the inference stage, we begin by randomly generating a collection of boxes. Subsequently, the filtering-based object decoder is iteratively employed to refine and prune these bounding boxes, taking into account the conditions on the given expression, the results from the previous denoised step, and the interaction between the expression and the image. Extensive experiments conducted on six datasets demonstrate that DiffusionREC outperforms previous REC methods, yielding superior performances.

Deep online cross-modal hashing has gained much attention from researchers recently, as its promising applications with low storage requirement, fast retrieval efficiency and cross modality adaptive, etc. However, there still exists some technical hurdles that hinder its applications, e.g., 1) how to extract the coexistent semantic relevance of cross-modal data, 2) how to achieve competitive performance when handling the real time data streams, 3) how to transfer the knowledge learned from offline to online training in a lightweight manner. To address these problems, this paper proposes a lightweight contrastive distilled hashing (LCDH) for cross-modal retrieval, by innovatively bridging the offline and online cross-modal hashing by similarity matrix approximation in a knowledge distillation framework. Specifically, in the teacher network, LCDH first extracts the cross-modal features by CLIP, which are further fed into an attention module for representation enhancement after feature fusion. Then, the output of the attention module is fed into a FC layer to obtain hash codes for aligning the sizes of similarity matrices for online and offline training. In the student network, LCDH extracts the visual and textual features by lightweight models, and then the features are fed into a FC layer to generate binary codes. Finally, by approximating the similarity matrices, the performance of online hashing in the lightweight student network can be enhanced by the supervision of coexistent semantic relevance that is distilled from the teacher network. Experimental results on three widely used datasets demonstrate that LCDH outperforms some state-of-the-art methods.

With the advancement of computer vision, numerous models have been proposed for screening of fundus diseases. However, the recognition of multiple fundus diseases is often hampered by the simultaneous presence of multiple disease types and the confluence of lesion types in fundus images. This paper addresses these challenges by conceptualizing them as multi-level feature fusion and self-supervised disease-indicative feature learning problems. We decode fundus images at various levels of granularity to delineate scenarios wherein multiple diseases and lesions co-occur. To effectively integrate these features, we introduce a hierarchical vision transformer (HVT) that adeptly captures both inter-level and intra-level dependencies. A novel forward-attention module is proposed to enhance the integration of lower-level semantic information into higher semantic layers, thereby enriching the representation of complex features. Additionally, we introduce a novel self-supervised mask-consistent feature learner (MCFL). Unlike traditional mask-autoencoders that reconstruct original images using encoder-decoder structures, MCFL utilizes a teacher-student framework to reconstruct mask-consistent feature maps. In this setup, exponential moving averaging is employed to derive classification-guided features, serving as labels for reconstruction rather than merely reconstructing the original images. This innovative approach facilitates the extraction of disease-indicative features. Extensive experiments demonstrate that our method significantly outperforms existing state-of-the-art models.

Video anomaly detection (VAD) aims at locating the abnormal events in videos. Recently, the Weakly Supervised VAD has made great progress, which only requires video-level annotations when training. In practical applications, different institutions may have different types of abnormal videos. However, the abnormal videos cannot be circulated on the internet due to privacy protection. To train a more generalized anomaly detector that can identify various anomalies, it is reasonable to introduce federated learning into WSVAD. In this paper, we propose Global and Local Context-driven Federated Learning, a new paradigm for privacy protected weakly supervised video anomaly detection. Specifically, we utilize the vision-language association of CLIP to detect whether the video frame is abnormal. Instead of leveraging handcrafted text prompts for CLIP, we propose a text prompt generator. The generated prompt is simultaneously influenced by text and visual. On the one hand, the text provides global context related to anomaly, which improves the model's ability of generalization. On the other hand, the visual provides personalized local context because different clients may have videos with different types of anomalies or scenes. The generated prompt ensures global generalization while processing personalized data from different clients. Extensive experiments show that the proposed method achieves remarkable performance.

Edit-based approaches for Grammatical Error Correction (GEC) have attracted volume attention due to their outstanding explanations of the correction process and rapid inference. Through exploring the characteristics of the generalized and specific knowledge learning for GEC, we discover that efficiently training GEC systems with satisfactory generalization capacity prefers more generalized knowledge rather than specific knowledge. Current gradient-based methods for training GEC systems, however, usually prioritize minimizing training loss over generalization loss. This paper proposes the strategy of Adjusting Learning Rate Based on Mermory Rate to optimize the edit-based GEC scorer (ALRMR-GEC). Specifically, we introduce the memory rate, a novel metric, to provide an explicit indicator for the model’s state of learning generalized and specific knowledge, which can effectively guide the GEC system to adjust the learning rate timely. Extensive experiments, conducted by optimizing the published edit scorer on the BEA2019 dataset, have shown our ALRMR-GEC significantly enhances the model generalization ability with stable and satisfactory performance nearly irrespective of the initial learning rate selection. Also, our method can accelerate the training over tenfold faster in certain cases. Finally, the experiments indicate the memory rate introduced in our ALRMR-GEC guides the GEC editscorer to learn more generalized knowledge.

In incomplete multi-view multi-label learning scenarios, it is crucial to use the incomplete multi-view data to extract consistent and specific representations from different data sources and to fully exploit the missing label information. However, most previous approaches ignore the separation problem between view-shared and specific information. To address this problem, in this paper, we propose a method that can separate view-consistent features from view-specific features under the Variational Autoen-coder (VAE) framework. Specifically, we first introduce cross-view reconstruction to capture view-consistent features and extract shared information from different views through unsupervised pre-training. Subsequently, we develop a disentangling module to learn specific features by minimizing the variational upper bound of mutual information between consistent and specific features. Finally, we utilize prior label relevance information derived from training data to guide the learning of the distribution of label semantic embeddings, aggregating relevant semantic embeddings and maintaining the label relevance topology in the semantic space. In extensive experiments, our model outperforms existing state-of-the-art algorithms on several real-world datasets, which fully validates its strong adaptability to missing views and labels.

To reduce the reliance of visible-infrared person re-identification (ReID) models on labeled cross-modal samples, this paper explores a weakly supervised cross-modal person ReID method that uses only single-modal sample identity labels, addressing scenarios where cross-modal identity labels are unavailable. To mitigate the impact of missing cross-modal labels on model performance, we propose a heterogeneous expert collaborative consistency learning framework, designed to establish robust cross-modal identity correspondences in a weakly supervised manner. This framework leverages labeled data from each modality to independently train dedicated classification experts. To associate cross-modal samples, these classification experts act as heterogeneous predictors, predicting the identities of samples from the other modality. To improve prediction accuracy, we design a cross-modal relationship fusion mechanism that effectively integrates predictions from different experts. Under the implicit supervision provided by cross-modal identity correspondences, collaborative and consistent learning among the experts is encouraged, significantly enhancing the model's ability to extract modality-invariant features and improve cross-modal identity recognition. Experimental results on two challenging datasets validate the effectiveness of the proposed method.

Multi-view data involves various data forms, such as multi-feature, multi-sequence and multimodal data, providing rich semantic information for downstream tasks. The inherent challenge of incomplete multi-view missing multi-label learning lies in how to effectively utilize limited supervision and insufficient data to learn discriminative representation. Starting from the sufficiency of multi-view shared information for downstream tasks, we argue that the existing contrastive learning paradigms on missing multi-view data show limited consistency representation learning ability, leading to the bottleneck in extracting multi-view shared information. In response, we propose to minimize task-independent redundant information by pursuing the maximization of cross-view mutual information. Additionally, to alleviate the hindrance caused by missing labels, we develop a dual-branch soft pseudo-label cross-imputation strategy to improve classification performance. Extensive experiments on multiple benchmarks validate our advantages and demonstrate strong compatibility with both missing and complete data.

With advancements in visual language models (VLMs) and large language models (LLMs), video anomaly detection (VAD) has progressed beyond binary classification to fine-grained categorization and multidimensional analysis. However, existing methods focus mainly on coarse-grained detection, lacking anomaly explanations. To address these challenges, we propose Ex-VAD, an Explainable Fine-grained Video Anomaly Detection approach that combines fine-grained classification with detailed explanations of anomalies. First, we use a VLM to extract frame-level captions, and an LLM converts them to video-level explanations, enhancing the model's explainability. Second, integrating textual explanations of anomalies with visual information greatly enhances the model's anomaly detection capability. Finally, we apply label-enhanced alignment to optimize feature fusion, enabling precise fine-grained detection. Extensive experimental results on the UCF-Crime and XD-Violence datasets demonstrate that Ex-VAD significantly outperforms existing State-of-The-Art methods.

Image steganography hides multiple images for multiple recipients into a single cover image. All secret images are usually revealed without authentication, which reduces security among multiple recipients. It is elegant to design an authentication mechanism for isolated reception. We explore such mechanism through sufficient experiments, and uncover that additional authentication information will affect the distribution of hidden information and occupy more hiding space of the cover image. This severely decreases effectiveness and efficiency in large-capacity hiding. To overcome such a challenge, we first prove the authentication feasibility within image steganography. Then, this paper proposes an image steganography network collaborating with separate authentication and efficient scheme. Specifically, multiple pairs of lock-key are generated during hiding and revealing. Unlike traditional methods, our method has two stages to make appropriate distribution adaptation between locks and secret images, simultaneously extracting more reasonable primary information from secret images, which can release hiding space of the cover image to some extent. Furthermore, due to separate authentication, fused information can be hidden in parallel with a single network rather than traditional serial hiding with multiple networks, which can largely decrease the model size. Extensive experiments demonstrate that the proposed method achieves more secure, effective, and efficient image steganography. Code is available at https://github.com/Revive624/Authentication-Image-Steganography.

Multimodal protein features play a crucial role in protein function prediction. However, these features encompass a wide range of information, ranging from structural data and sequence features to protein attributes and interaction networks, making it challenging to decipher their complex interconnections. In this work, we propose a multimodal protein function prediction method (DSRPGO) by utilizing dynamic selection and reconstructive pre-training mechanisms. To acquire complex protein information, we introduce reconstructive pre-training to mine more fine-grained information with low semantic levels. Moreover, we put forward the Bidirectional Interaction Module (BInM) to facilitate interactive learning among multimodal features. Additionally, to address the difficulty of hierarchical multi-label classification in this task, a Dynamic Selection Module (DSM) is designed to select the feature representation that is most conducive to current protein function prediction. Our proposed DSRPGO model improves significantly in BPO, MFO, and CCO on human datasets, thereby outperforming other benchmark models.

Pixel-level anomaly detection is indispensable in industrial defect detection and medical diagnosis. Recently, Segment Anything Model (SAM) has achieved promising results in many vision tasks. However, direct application of the SAM to pixel-level anomaly detection tasks results in unsatisfactory performance, meanwhile SAM needs the manual prompt. Although some automatically prompt-based SAM has been proposed, these automated prompting approaches merely utilize partial image features as prompts and fail to incorporate crucial features such as multi-scale image features to generate more suitable prompts. In this paper, we propose a novel Omni Dimensional State Space Model-driven SAM (ODS-SAM) for pixel-level anomaly detection. Specifically, the proposed method adopts the SAM architecture, ensuring easy implementation and avoiding the need for fine-tuning. A State-Space Model-based residual Omni Dimensional module is designed to automatically generate suitable prompts. This module can effectively leverage multi-scale and global information, facilitating an iterative search for optimal prompts in the prompt space. The identified optimal prompts are then fed into SAM as high-dimensional tensors. Experimental results demonstrate that the proposed ODS-SAM outperforms state-of-the-art models on both industrial and medical image datasets.

Zero-Shot Industrial Anomaly Detection (ZSIAD) aims to identify and localize anomalies in industrial images from unseen categories. Owing to the powerful generalization capabilities, Vision-Language Models (VLMs) have achieved growing interest in ZSIAD. To guide the model toward understanding and localizing the semantically complex industrial anomalies, existing VLM-based methods have attempted to provide additional prompts to the model through learnable text prompt templates. However, these zero-shot methods lack detailed descriptions of specific anomalies, making it difficult to classify and segment the diverse range of industrial anomalies accurately. To address the aforementioned issue, we firstly propose the multi-stage prompt generation agent for ZSIAD. Specifically, we leverage the Multi-modal Language Large Model (MLLM) to articulate the detailed differential information between normal and test samples, which can provide detailed text prompts to the model through further refinement and anti-false alarm constraint. Moreover, we introduce the Visual Fundamental Model (VFM) to generate anomaly-related attention prompts for more accurate localization of anomalies with varying sizes and shapes. Extensive experiments on seven real-world industrial anomaly detection datasets have shown that the proposed method not only outperforms recent SOTA methods, but also its explainable prompts provide the model with a more intuitive basis for anomaly identification.

Existing deep neural network (DNN)-based blind image quality assessment (BIQA) methods primarily rely on human-rated datasets for training. However, collecting human labels is extremely time-consuming and labor-intensive, posing a significant bottleneck for practical applications. To address this challenge, we propose a Deep opinion-Unaware BIQA model by learning and adapting from Multiple Annotators, termed DUBMA, thereby eliminating the need for human annotations. Specifically, we first generate a large-scale set of distorted image pairs and then assign relative quality rankings using existing full-reference IQA models. The resulting dataset is subsequently employed for training our DUBMA. Due to the inherent discrepancies between synthetic and real-world distortions, a domain shift may occur. To address this, we propose an outlier-robust unsupervised domain adaptation approach leveraging optimal transport. This strategy effectively reduces the gap between synthetic and real-world distortion domains, thereby boosting the model’s adaptability and overall performance. Extensive experiments show that DUBMA outperforms existing opinion-unaware BIQA methods in terms of prediction accuracy across multiple datasets.

Contactless palmprint recognition has recently emerged as a promising biometric technology. However, traditional methods that require sharing user data introduce substantial security risks. While federated learning offers privacy-preserving solutions, it often compromises recognition accuracy due to feature distribution drift caused by external factors such as lighting and devices. To address this issue, we propose an adaptive personalized federated learning framework (AdaptPFL). The central innovation lies in decomposing palmprint features into identity-related and contextual-related components using a feature decoupling mechanism. This design isolates the influence of external environmental factors on identity recognition through de-entanglement. Furthermore, two adaptive aggregation strategies are introduced to correct client drift: (1) Intra-Local Adaptive Aggregation (ILAA), which addresses intra-client drift by adaptively combining the two decoupled feature types; (2) Global-Local Adaptive Aggregation (GLAA), which corrects inter-client drift by adaptively aggregating model parameters. Experimental results demonstrate that AdaptPFL achieves superior performance compared to existing state-of-the-art methods.

Current existing clustering methods for handling incomplete multi-view data primarily concentrate on learning a common representation or graph from the available views, while overlooking the latent information contained in the missing views and the imbalance of information among different views. Furthermore, instances with weak discriminative features usually degrading the precision of consistent representation or graph across all views. To address these problems, in this paper, we propose a simple but efficient method, called high-confident local structure guided consensus graph learning for incomplete multi-view clustering (HLSCG_IMC). Specifically, this method can adaptively learn a strict block diagonal structure from the available samples using a block diagonal representation regularizer. Different from the existing methods using a simple pairwise affinity graph for structure construction, we consider the influence of instances located at the edge of two clusters on the construction of graph for each view. By harnessing the proposed high-confident strict block diagonal structures, the approach seeks to directly guide the learning of the robust consensus graph. A number of experiments have been conducted to verify the efficacy of our approach.

Supervised cross-modal hashing has achieved remarkable progress in retrieving related items across different modalities. However, in practical applications, a significant portion of data remains unlabeled, such as online data on websites, which must be included for effective retrieval. To address this challenge, while maintaining the high accuracy and efficiency of supervised methods, few works have attempted to adapt existing supervised techniques to handle unsupervised tasks through a general modular approach. To this end, we introduce a novel cross-modal hashing method, termed Label Prediction Inherited Hashing (LPIH). Initially, LPIH leverages labeled data to learn high-quality general label functions using supervised methods. Subsequently, it inherits the existing hash codes from existing supervised methods to further refine the pseudo-label information. Finally, LPIH integrates the refined pseudo-label information with the existing hash functions to learn new hash functions specifically tailored for unsupervised tasks. Extensive experimental results on three public datasets demonstrate the superior performance of LPIH compared to state-of-the-art (SOTA) cross-modal hashing methods. Specifically, LPIH achieves an average precision improvement of 5% over SOTA methods, highlighting its effectiveness in bridging the gap between supervised and unsupervised learning in the context of cross-modal retrieval.

Recent advancements in reasoning capability of Multimodal Large Language Models (MLLMs) demonstrate its effectiveness in tackling complex visual tasks. However, existing MLLM-based Video Anomaly Detection (VAD) methods remain limited to shallow anomaly descriptions without deep reasoning. In this paper, we propose a new task named Video Anomaly Reasoning (VAR), which aims to enable deep analysis and understanding of anomalies in the video by requiring MLLMs to think explicitly before answering. To this end, we propose Vad-R1, an end-to-end MLLM-based framework for VAR. Specifically, we design a Perception-to-Cognition Chain-of-Thought (P2C-CoT) that simulates the human process of recognizing anomalies, guiding the MLLMs to reason about anomalies step-by-step. Based on the structured P2C-CoT, we construct Vad-Reasoning, a dedicated dataset for VAR. Furthermore, we propose an improved reinforcement learning algorithm AVA-GRPO, which explicitly incentivizes the anomaly reasoning capability of MLLMs through a self-verification mechanism with limited annotations. Experimental results demonstrate that Vad-R1 achieves superior performance, outperforming both open-source and proprietary models on VAD and VAR tasks.

Dynamic functional brain networks (DFBNs) are powerful tools in neuroscience research. Recent studies reveal that DFBNs contain heterogeneous neural nodes with more extensive connections and more drastic temporal changes, which play pivotal roles in coordinating the reorganization of the brain. Moreover, the spatio-temporal patterns of these nodes are modulated by the brain's historical states. However, existing methods not only ignore the spatio-temporal heterogeneity of neural nodes, but also fail to effectively encode the temporal propagation mechanism of heterogeneous activities. These limitations hinder the deep exploration of spatio-temporal relationships within DFBNs, preventing the capture of abnormal neural heterogeneity caused by brain diseases. To address these challenges, this paper propose a neuro-heterogeneity guided temporal graph learning strategy (NeuroH-TGL). Specifically, we first develop a spatio-temporal pattern decoupling module to disentangle DFBNs into topological consistency networks and temporal trend networks that align with the brain's operational mechanisms. Then, we introduce a heterogeneity mining module to identify pivotal heterogeneity nodes that drive brain reorganization from the two decoupled networks. Finally, we design temporal propagation graph convolution to simulate the influence of the historical states of heterogeneity nodes on the current topology, thereby flexibly extracting heterogeneous spatio-temporal information from the brain. Experiments show that our method surpasses several state-of-the-art methods, and can identify abnormal heterogeneous nodes caused by brain diseases.

Alzheimer's Disease (AD) poses a significant health threat to the aging population, underscoring the critical need for early diagnosis to delay disease progression and improve patient quality of life. Recent advances in heterogeneous multimodal artificial intelligence (AI) have facilitated comprehensive joint diagnosis, yet practical clinical scenarios frequently encounter incomplete modalities due to factors like high acquisition costs or radiation risks. Moreover, traditional convolution-based architecture face inherent limitations in capturing long-range dependencies and handling heterogeneous medical data efficiently. To address these challenges, in our proposed heterogeneous multimodal diagnostic framework (HAD), we develop a multi-view Hilbert curve-based Mamba block and a hierarchical spatial feature extraction module to simultaneously capture local spatial features and global dependencies, effectively alleviating spatial discontinuities introduced by voxel serialization. Furthermore, to balance semantic consistency and modal specificity, we build a unified mutual information learning objective in the heterogeneous multimodal embedding space, which maintains effective learning of modality-specific information to avoid modality collapse caused by model preference. Extensive experiments demonstrate that our HAD significantly outperforms state-of-the-art methods in various modality-missing scenarios, providing an efficient and reliable solution for early-stage AD diagnosis.

Label prototype learning has emerged as an effective paradigm in Partial Multi-Label Learning (PML), providing a distinctive framework for modeling structured representations of label semantics while naturally filtering noise through prototype-based label confidence estimation. However, existing prototype-based methods face a critical limitation: class prototypes are the biased estimates due to noisy candidate labels, particularly when positive samples are scarce. To this end, we first propose a mutually class prototype alignment strategy bypassing noise interference by introducing two different transformation matrices, which makes the class prototypes learned by the fuzzy clustering and candidate label set mutually alignment for correcting themselves. Such alignment is also passed on to the fuzzy memberships label in turn. In addition, to eliminate noise interference in the candidate label set during the classifier learning, we use the learned permutation matrix to transform the fuzzy memberships label for learning a label reliability indicator matrix accompanied by the candidate label set. This makes the label reliability indicator matrix absolutely prevent the occurrence of numerical values located in non-label and simultaneously eliminate the introduction of incorrect label as much as possible. The resulting indicator matrix guides a robust multi-label classifier training process, jointly optimizing label confidence and classifier parameters. Extensive experiments demonstrate that our proposed model exhibits significant performance advantages over state-of-the-art PML approaches.

Multi-view clustering aims to enhance clustering performance by leveraging information from diverse sources. However, its practical application is often hindered by a barrier: the lack of correspondences across views. This paper focuses on the understudied problem of fully incomplete multi-view clustering (FIMC), a scenario where existing methods fail due to their reliance on partial alignment. To address this problem, we introduce the Contrastive Prototype Matching Network (CPMN), a novel framework that establishes a new paradigm for cross-view alignment based on matching high-level categorical structures. Instead of aligning individual instances, CPMN performs a more robust cluster prototype alignment. CPMN first employs a correspondence-free graph contrastive learning approach, leveraging mutual $k$-nearest neighbors (MNN) to uncover intrinsic data structures and establish initial prototypes from entirely unpaired views. Building on the prototypes, we introduce a cross-view prototype graph matching stage to resolve category misalignment and forge a unified clustering structure. Finally, guided by this alignment, we devise a prototype-aware contrastive learning mechanism to promote semantic consistency, replacing the reliance on the initial MNN-based structural similarity. Extensive experiments on benchmark datasets demonstrate that our method significantly outperforms various baselines and ablation variants, validating its effectiveness.

Poison-only Clean-label Backdoor Attacks (PCBAs) aim to covertly inject attacker-desired behavior into DNNs by merely poisoning the dataset without changing the labels. To effectively implant a backdoor, multiple triggers are proposed for various attack requirements of Attack Success Rate (ASR) and stealthiness. Additionally, sample selection enhances clean-label backdoor attacks' ASR by meticulously selecting "hard'' samples instead of random samples to poison. Current methods, however, 1) usually handle the sample selection and triggers in isolation, leading to severely limited improvements on both ASR and stealthiness. Consequently, attacks exhibit unsatisfactory performance on evaluation metrics when converted to PCBAs via a mere stacking of methods. Therefore, we seek to explore the bi-directional collaborative relations between the sample selection and triggers to address the above dilemma. 2) Since the strong specificity within triggers, the simple combination of sample selection and triggers fails to substantially enhance both evaluation metrics, with generalization preserved among various attacks. Therefore, we seek to propose a set of components to significantly improve both stealthiness and ASR based on the commonalities of attacks. Specifically, Component A ascertains two critical selection factors, and then makes them an appropriate combination based on the trigger scale to select more reasonable "hard'' samples for improving ASR. Component B is proposed to select samples with similarities to relevant trigger implanted samples to promote stealthiness. Component C reassigns trigger poisoning intensity on RGB colors through distinct sensitivity of the human visual system to RGB for higher ASR, with stealthiness ensured by sample selection including Component B. Furthermore, all components can be strategically integrated into diverse PCBAs, enabling tailored solutions that balance ASR and stealthiness enhancement for specific attack requirements. Extensive experiments demonstrate the superiority of our components in stealthiness, ASR, and generalization. Our code will be released as soon as possible.

As a cross-topic of multi-view learning and multi-label classification, multi-view multi-label classification has gradually gained traction in recent years. The application of multi-view contrastive learning has further facilitated this process; however, the existing multi-view contrastive learning methods crudely separate the so-called negative pair, which largely results in the separation of samples belonging to the same category or similar ones. Besides, plenty of multi-view multi-label learning methods ignore the possible absence of views and labels. To address these issues, in this paper, we propose an incomplete multi-view missing multi-label classification network named RANK. In this network, a label-driven multi-view contrastive learning strategy is proposed to leverage supervised information to preserve the intra-view structure and perform the cross-view consistency alignment. Furthermore, we break through the view-level weights inherent in existing methods and propose a quality-aware subnetwork to dynamically assign quality scores to each view of each sample. The label correlation information is fully utilized in the final multi-label cross-entropy classification loss, effectively improving the discriminative power. Last but not least, our model is not only able to handle complete multi-view multi-label data, but also works on datasets with missing instances and labels. Extensive experiments confirm that our RANK outperforms existing state-of-the-art methods.

Referring expression comprehension (REC) aims at locating the target object described by an expression. We observe that most of the graph-based REC methods only focus on establishing relations between all objects in an image and the given expression during the graph construction while ignoring the relationships between objects in the same category. As a result, these methods are sub-optimal in locating the target object described by the expression, particularly when the target object is surrounded by objects of similar categories. Meanwhile, during reasoning, numerous irrelevant objects are considered for expression, which will introduce significant harmful noise. To address these issues, this paper proposes a new graph-based group division network (GBGDN). Different from the existing works, our work partitions the constructed graphs into several sub-graphs based on the categories of objects and expressions. In each sub-graph, the common visual features of objects will be strengthened through a feature enhancement strategy. Subsequently, the enhanced sub-graphs and expressions undergo joint processing via a filtering-based reasoning module designed to reduce the influence of unrelated nodes in each sub-graph, facilitating more accurate reasoning and matching. Experimental results across various datasets, including RefCOCO /+/g, Flickr30K Entities, RefClef, and Ref-reasoning, showcase the superiority of our proposed method over existing approaches. Most importantly, our method does not need pre-training.

Recently, the topic of multi-view multi-label classification has aroused significant attention from scholars. Plenty of methods adopt an average weighting scheme to merge the features obtained from multiple views, which commonly ignore the quality difference of information provided by multiple views and thus limit the credibility of the fusion feature for the overall task. Besides, most of these methods assume the views and labels are complete while neglecting both views and labels may be incomplete. To solve these problems, we propose a quality-aware representation fusion network for partial multi-view incomplete multi-label classification, named QARF-net. Since assigning equal fusion weights for each view may be not in line with the actual contributions of individual views, a view quality-aware module is proposed to learn suitable weights for different views dynamically based on the quality of each view’s information, which provides a reliable guide for fusing the information of multiple views. In addition, considering the consistency characteristics of multi-view data, we impose a sample-level dual constraint to preserve the consistency property of the feature in multi-view space and constrain the sample structure in the fused feature space, respectively. Last but not least, QARF-net can not only deal with complete multi-view multi-label classification tasks but also tackle partial multi-view incomplete multi-label classification tasks. Experimental results on five real-world datasets indicate that our proposed method outperforms state-of-the-art methods.

Three significant challenges have been limiting the stable palmprint recognition via mobile devices: 1) rotations and unconsensus scales of the unconstrait hand; 2) noises generated in the open imaging environments; and 3) low quality images captured in the low-illumination conditions. Current palmprint representation methods rely on rich prior knowledge and lack any adaptability to its environment. In this paper, we propose a multi-view hierarchical graph learning based palmprint recognition (MVHG_PR) method, which comprehensively presents the discriminant palmprint features from multiple views. Fully exploiting different types of characteristics, it aims to adaptively perform multi-view feature description and feature selection. To this end, a novel regularized heterogeneous graph learning strategy is proposed for construction of the intra- and inter-class relationships, learning high-order structures for different views between four tuples, rather than just pair-wise intrinsic structures. In the proposed model, the learned hierarchical graph is given an elastic power from the label information to precisely reflect the intra-class and the inter-class relationships in each view, such that the projected structures can be aligned locally and globally. Besides this, we constructed a mobile palmprint dataset to simulate as many open application circumstance as possible to verify the effectiveness of contactless palmprint recognition methods. Experimental results have proven the superiority of the proposed MVHG_PR by achieving the best recognition performances on a number of real-world palmprint databases. The proposed mobile palmprint database and the code of the proposed MVHG_PR are available athttps://github.com/ShupingZhao/MVHG_PR-for-contactless-palmprint-recognition.

Accurate 3D scene understanding in outdoor environments heavily relies on high-quality point clouds. However, LiDAR-scanned data often suffer from extreme sparsity, severely hindering downstream 3D perception tasks. Existing point cloud upsampling methods primarily focus on individual objects, thus demonstrating limited generalization capability for complex outdoor scenes. To address this issue, we propose PVNet, a diffusion model-based point-voxel interaction framework to perform LiDAR point cloud upsampling without dense supervision. Specifically, we adopt the classifier-free guidance-based DDPMs to guide the generation, in which we employ a sparse point cloud as the guiding condition and the synthesized point clouds derived from its nearby frames as the input. Moreover, we design a voxel completion module to refine and complete the coarse voxel features for enriching the feature representation. In addition, we propose a point-voxel interaction module to integrate features from both points and voxels, which efficiently improves the environmental perception capability of each upsampled point. To the best of our knowledge, our approach is the first scene-level point cloud upsampling method supporting arbitrary upsampling rates. Extensive experiments on various benchmarks demonstrate that our method achieves state-of-the-art performance. The source code will be available at https://github.com/chengxianjing/PVNet.

Multi-view clustering typically leverages the consistency and complementarity among views to partition different samples. However, existing deep learning-based methods often face the dilemma between selecting complementary information and capturing essential details: 1) Capturing complementary semantics among views may introduce label-irrelevant redundant information. 2) Only extracting consistent semantic information will cause information loss, hindering the clarity in downstream tasks. To address these issues, we propose a novel method from the perspective of meta-learning to learn clustering-friendly representations with minimal redundancy. Specifically, we train an information compressor to guide the model in describing the original samples as compact as possible with minimal information, thus learning the key semantics with minimized redundancy. Meta-learning bi-level optimization promotes the nested optimization of feature embedding and information compressor. Meanwhile, a semantic puzzle mechanism complements the semantic fragments by exploiting the relationships between low-level features, resulting in a consensus representation with strong discriminative power. We conducted extensive experiments on datasets with various sizes to validate the effectiveness of our model, demonstrating significant performance improvements over several state-of-the-art methods.

Multi-view clustering (MVC) aims to exploit the latent relationships between heterogeneous samples in an unsupervised manner, which has served as a fundamental task in the unsupervised learning community and has drawn widespread attention. In this work, we propose a new deep multi-view contrastive clustering method via graph structure awareness (DMvCGSA) by conducting both instance-level and cluster-level contrastive learning to exploit the collaborative representations of multi-view samples. Unlike most existing deep multi-view clustering methods, which usually extract only the attribute features for multi-view representation, we first exploit the view-specific features while preserving the latent structural information between multi-view data via a GCN-embedded autoencoder, and further develop a similarity-guided instance-level contrastive learning scheme to make the view-specific features discriminative. Moreover, unlike existing methods that separately explore common information, which may not contribute to the clustering task, we employ cluster-level contrastive learning to explore the clustering-beneficial consistency information directly, resulting in improved and reliable performance for the final multi-view clustering task. Extensive experimental results on twelve benchmark datasets clearly demonstrate the encouraging effectiveness of the proposed method compared with the state-of-the-art models.

Referring expression comprehension (REC) is a crucial task in understanding how a given text description identifies a target object within an image. Existing two-stage REC methods have demonstrated strong performance due to their rational framework design. However, during the encoding of object candidates in an image, most two-stage methods rely exclusively on features extracted from pre-trained detectors, often neglecting the contextual relationships between an object and its neighboring elements. This limitation hinders the full capture of contextual and relational information, reducing the discriminative power of object representations and negatively impacting subsequent processing. In this paper, we propose two novel plug-and-adapt modules: expression-guided label representation module (ELR) and cross-modal calibrated semantic module (CCS), designed to enhance two-stage REC methods. Specifically, the ELR module connects the noun phases of expression to the categorical labels of object candidates in the image, ensuring effective alignment between them. Guided by these connections, a CCS module is introduced to represent each object candidate by integrating its features with those of neighboring candidates from multiple perspectives. This preserves the intrinsic information of each candidate while incorporating relational cues from other objects, enabling more precise embeddings and effective downstream processing in two-stage REC methods. Extensive experiments on six datasets demonstrate the importance of incorporating prior statistical knowledge, and detailed analysis shows that the proposed modules strengthen the alignment between image and text. As a result, our method achieves competitive performance and is compatible with most two-stage methods in the REC task. The code is available on Github: https://github.com/freedom6927/ELR_CCS.git.

In recent years, multi-view data often suffer from incompleteness owing to environmental factors, equipment failures. Thus Incomplete Multi-View Clustering (IMVC) has become an important research focus, which aims to alleviate the adverse impacts of missing views and leverage inter-view complementary information to enhance clustering performance. However, existing IMVC methodologies suffer from three critical limitations: 1) Inadequate integration of cross-view learning and cross-instance learning; 2) Lack of explicit modeling for dynamic interactions between view-specific information and cross-view shared semantics; 3) Inability to dynamically capture high-order topological correlations under view-missing conditions, leading to semantic misalignment among samples. To address these challenges, we propose an IMVC framework CrossNet-VGA based on variational collaboration and graph attention fusion. Specifically, We formulate a novel multi-view evidence lower bound to explicitly separate view-specific latent variables and cross-view shared latent variables, and achieve inter-view semantic fusion by integrating variational distributions shared across views. Contrastive learning is employed to maximize mutual information and promote feature distribution uniformity, thereby achieving consistent representation learning. We employ dynamic $k$ -nearest neighbor graph construction and multi-head graph attention mechanisms to capture the inter-sample deep topological correlations, achieving robust structural alignment. Comprehensive experiments conducted on 6 public datasets demonstrate that CrossNet-VGA significantly outperforms the competing methods both on accuracy and robustness. The anonymous code of this work is available on GitHub at https://github.com/ggg2111/2025-TIP-CrossNet-VGA.

Cross-modal retrieval is a promising technique nowadays to find semantically similar instances in other modalities while a query instance is given from one modality. However, there still exists many challenges for reducing heterogeneous modality gap by embedding label information to discrete hash codes effectively, solving the binary optimization when generating unified hash codes and reducing the discrepancy of data distribution efficiently during common space learning. In order to overcome the above-mentioned challenges, we propose a Collaboratively Semantic alignment and Metric learning for cross-modal Hashing (CSMH) in this paper. Specifically, by a kernelization operation, CSMH first extracts the non-linear data features for each modality, which are projected into a latent subspace to align both marginal and conditional distributions simultaneously. Then, a maximum mean discrepancy-based metric strategy is customized to mitigate the distribution discrepancies among features from different modalities. Finally, semantic information obtained from the label similarity matrix, is further incorporated to embed the latent semantic structure into the discriminant subspace. Experimental results of CSMH and baseline methods on four widely-used datasets show that CSMH outperforms some state-of-the-art hashing baseline methods for cross-modal retrieval on efficiency and precision.

Most current multi-view clustering methods necessitate that a sample's features be view-aligned or at least partially aligned across different views. Regrettably, real-world applications often fail to meet this requirement due to spatial, temporal, or spatiotemporal mismatches, resulting in the view-unaligned issue. To tackle this issue, we conceptualize the view-unaligned problem and demonstrate that it can be transformed into a view-aligned problem through reordering. Building on this concept, we introduce an innovative reorder matrix that realigns view-unaligned features. Utilizing these realigned features, we develop a sophisticated and efficient approach called Reordered$k$-means (RKM), which merges NMF with$k$-means. Unlike traditional$k$-means, our method converts the binary challenge into an$\ell _{0}$problem, confirming the merit of this advancement. Furthermore, RKM's efficacy is affirmed on benchmarks, indicating substantial enhancements in handling the view-unaligned issue and maintaining competitive results with view-aligned problems.

Efforts in weakly-supervised video anomaly detection center on detecting abnormal events within videos by coarse-grained labels, which has been successfully applied to many real-world applications. However, a significant limitation of most existing methods is that they are only effective for specific objects in specific scenarios, which makes them prone to misclassification or omission when confronted with previously unseen anomalies. Relative to conventional anomaly detection tasks, Open-world Weakly-supervised Video Anomaly Detection (OWVAD) poses greater challenges due to the absence of labels and fine-grained annotations for unknown anomalies. To address the above problem, we propose a multi-scale evidential vision-language model to achieve open-world video anomaly detection. Specifically, we leverage generalized visual-language associations derived from CLIP to harness the full potential of large pre-trained models in addressing the OWVAD task. Subsequently, we integrate a multi-scale temporal modeling module with a multimodal evidence collector to achieve precise frame-level detection of both seen and unseen anomalies. Extensive experiments on two widely-utilized benchmarks have conclusively validated the effectiveness of our method. The code will be made publicly available.

In the field of multi-view multi-label learning, the challenges of incomplete views and missing labels are prevalent due to the complexity of manual labeling and data acquisition errors. These challenges significantly reduce the quality of latent representations and hinder prediction by multi-label classification. To address this issue, we propose a novel Category-driven Semi-supervised Contrastive Recovery (CSCR) framework in this study. Our framework aims to fully integrate existing label information into incomplete representation learning and classification. Specifically, to address the limitations posed by incomplete views and labels, we construct a label coincidence matrix based on existing labels, which serves as a similarity matrix in subsequent semi-supervised contrastive learning and multi-view classification. By leveraging this matrix, we design a semi-supervised multi-view contrastive learning module, which constructs sample pairs on the basis of inter-view correspondences and label similarity. It learns discriminative latent representations without the need for data augmentation. A weighted multi-label classification module is subsequently employed to integrate the predictions from each view to obtain the final classification result. Experimental evaluations on five challenging datasets demonstrate the superiority of our model over existing state-of-the-art methods.

Cross-modal hash learning has drawn widespread attention for large-scale multimodal retrieval because of its stability and efficiency in approximate similarity searches. However, most existing cross-modal hashing approaches employ discrete label-guided information to coarsely reflect intra- and intermodality correlations, making them less effective to measuring the semantic similarity of data with multiple modalities. In this paper, we propose a new heterogeneous pairwise-semantic enhancement hashing (HPsEH) for large-scale cross-modal retrieval by distilling higher-level pairwise-semantic similarity from supervision information. First, we adopt a supervised self-expression to learn a data-specific quantified semantic matrix, which uses real values to measure both the similarity and dissimilarity ranks of paired instances, such that the intrinsic semantics of the data can be well captured. Then, we fuse the label-based information and quantified semantic similarity to collaboratively learn the hash codes of multimodal data, such that both the intermodality consistency and modality-specific features can be simultaneously obtained during hash code learning. Moreover, we employ effective iterative optimization to address the discrete binary solution and massive pairwise matrix calculation, making the HPsEH scalable to large-scale datasets. Extensive experimental results on three widely used datasets demonstrate the superiority of our proposed HPsEH method over most state-of-the art approaches.

Incomplete multiview clustering (IMVC) has received extensive attention in recent years. However, existing works still have several shortcomings: 1) some works ignore the correlation of sample pairs in the global structural distribution; 2) many methods are computational expensive, thus cannot be applicable to the large-scale incomplete data clustering tasks; and 3) some methods ignore the refinement of the bipartite graph structure. To address the above issues, we propose a novel anchor graph network for IMVC, which includes a generative model and a similarity metric network. Concretely, the method uses a generative model to construct bipartite graphs, which can mine latent global structure distributions of sample pairs. Later, we use graph convolution network (GCN) with the constructed bipartite graphs to learn the structural embeddings. Notably, the introduction of bipartite graphs can greatly reduce the computational complexity and thus enable our model to handle large-scale data. Unlike previous works based on bipartite graph, our method employs bipartite graphs to guide the learning process in GCNs. In addition, an innovative adaptive learning strategy that can construct robust bipartite graphs is incorporated into our method. Extensive experiments demonstrate that our method achieves the comparable or superior performance compared with the state-of-the-art methods.

Due to the inefficiency of pixel-level annotations, weakly supervised salient object detection with image-category labels (WSSOD) has been receiving increasing attention. Previous works usually endeavor to generate high-quality pseudolabels to train the detectors in a fully supervised manner. However, we find that the detection performance is often limited by two types of noise contained in pseudolabels: 1) holes inside the object or at the edge and outliers in the background and 2) missing object portions and redundant surrounding regions. To mitigate the adverse effects caused by them, we propose local pixel correction (LPC) and key pixel attention (KPA), respectively, based on two key properties of desirable pseudolabels: 1) spatial continuity, meaning an object region consists of a cluster of adjacent points; and 2) nonequal importance, meaning pixels have different importance for training. Specifically, LPC fills holes and filters out outliers based on summary statistics of the neighborhood as well as its size. KPA directs the focus of training toward ambiguous pixels in multiple pseudolabels to discover more accurate saliency cues. To evaluate the effectiveness of our method, we design a simple yet strong baseline we call weakly supervised saliency detector with Transformer (WSSDT) and unify the proposed modules into WSSDT. Extensive experiments on five datasets demonstrate that our method significantly improves the baseline and outperforms all existing congeneric methods. Moreover, we establish the first benchmark to evaluate WSSOD robustness. The results show that our method can improve detection robustness as well. The code and robustness benchmark are available at https://github.com/Horatio9702/SCNI.

Exploring the structure information is crucial for data clustering task, particularly for the sceneries of incomplete multiview clustering (IMVC) when some views are missing. However, almost all of the existing graph-based IMVC methods either introduce the Laplacian constraint with fixed graphs or simply fuse the graphs of all views, which are vulnerable to the quality of the constructed graphs. To address this issue, we propose a new graph-based method, called confident local structure-aware incomplete multiview spectral clustering. Different from existing works, our method seeks to adaptively uncover the inherent similarity structure among the available instances in each view and learn the optimal consensus graph within a unified learning framework. Moreover, to mitigate the adverse effects of imbalance information across incomplete views and improve the quality of consensus graph, we further impose some adaptive weights on the consensus graph learning model w.r.t. each view and introduce some confident structure graphs to explore the most confident similarity information in the model. In contrast to existing works, our approach simultaneously takes into account the pairwise similarity information and neighbor group-based confident structure information. This dual consideration makes our method more effective in achieving the optimal consensus graph and delivering superior IMVC performance. Experimental results on several datasets demonstrate that our method effectively learns a high-quality and clustering-friendly graph from incomplete multiview data, and it outperforms many state-of-the-art IMVC methods in terms of clustering performance.

As a combination of emerging multi-view learning methods and traditional multi-label classification tasks, multi-view multi-label classification has shown broad application prospects. The diverse semantic information contained in heterogeneous data effectively enables the further development of multi-label classification. However, the widespread incompleteness problem on multi-view features and labels greatly hinders the practical application of multi-view multi-label classification. Therefore, in this paper, we propose an attention-induced missing instances imputation technique to enhance the generalization ability of the model. Different from existing incomplete multi-view completion methods, we attempt to approximate the latent features of missing instances in embedding space according to cross-view joint attention, instead of recovering missing views in kernel space or original feature space. Accordingly, multi-view completed features are dynamically weighted by the confidence derived from joint attention in the late fusion phase. In addition, we propose a multi-view multi-label classification framework based on label-semantic feature learning, utilizing the statistical weak label correlation matrix and graph attention network to guide the learning process of label-specific features. Finally, our model is compatible with missing multi-view and partial multi-label data simultaneously and extensive experiments on five datasets confirm the advancement and effectiveness of our embedding imputation method and multi-view multi-label classification model.

Recently, multi-view multi-label classification (MvMLC) has received a significant amount of research interest and many methods have been proposed based on the assumptions of view completion and label completion. However, in real-world scenarios, multi-view multi-label data tends to be incomplete due to various uncertainties involved in data collection and manual annotation. As a result, the conventional MvMLC methods fail. In this paper, we propose a new two-stage MvMLC network to solve this incomplete MvMLC issue with partial missing views and missing labels. Different from the existing works, our method attempts to leverage the diverse information from the partially missing data based on the information theory. Specifically, our method aims to minimize task-irrelevant information while maximizing task-relevant information through the principles of information bottleneck theory and mutual information extraction. The first stage of our network involves training view-specific classifiers to concentrate the task-relevant information. Subsequently, in the second stage, the hidden states of these classifiers serve as input for an alignment model, an autoencoder-based mutual information extraction framework, and a weighted fusion classifier to make the final prediction. Extensive experiments performed on five datasets validate that our method outperforms other state-of-the-art methods. Code is available at https://github.com/KevinTan10/TSIEN.

Diabetic Retinopathy (DR), the leading cause of blindness in diabetic patients, is diagnosed by the condition of retinal multiple lesions. As a difficult task in medical image segmentation, DR multi-lesion segmentation faces the main concerns as follows. On the one hand, retinal lesions vary in location, shape, and size. On the other hand, because some lesions occupy only a very small part of the entire fundus image, the high proportion of background leads to difficulties in lesion segmentation. To solve the above problems, we propose a heterogeneous-aware convolutional network (HACDR-Net) that composes heterogeneous cross-convolution, heterogeneous modulated deformable convolution, and optional near-far-aware convolution. Our network introduces an adaptive aggregation module to summarize the heterogeneous feature maps and get diverse lesion areas in the heterogeneous receptive field along the channels and space. In addition, to solve the problem of the highly imbalanced proportion of focal areas, we design a new medical image segmentation loss function, Noise Adjusted Loss (NALoss). NALoss balances the predictive feature distribution of background and lesion by jointing Gaussian noise and hard example mining, thus enhancing awareness of lesions. We conduct the experiments on the public datasets IDRiD and DDR, and the experimental results show that the proposed method achieves better performance than other state-of-the-art methods. The code is open-sourced on github.com/xqh180110910537/HACDR-Net.

Incomplete multi-view clustering (IMVC) aims to reveal shared clustering structures within multi-view data, where only partial views of the samples are available. Existing IMVC methods primarily suffer from two issues: 1) Imputation-based methods inevitably introduce inaccurate imputations, which in turn degrade clustering performance; 2) Imputation-free methods are susceptible to unbalanced information among views and fail to fully exploit shared information. To address these issues, we propose a novel method based on variational autoencoders. Specifically, we adopt multiple view-specific encoders to extract information from each view and utilize the Product-of-Experts approach to efficiently aggregate information to obtain the common representation. To enhance the shared information in the common representation, we introduce a coherence objective to mitigate the influence of information imbalance. By incorporating the Mixture-of-Gaussians prior information into the latent representation, our proposed method is able to learn the common representation with clustering-friendly structures. Extensive experiments on four datasets show that our method achieves competitive clustering performance compared with state-of-the-art methods.

Few-shot font generation (FFG) produces stylized font images with a limited number of reference samples, which can significantly reduce labor costs in manual font designs. Most existing FFG methods follow the style-content dis-entanglement paradigm and employ the Generative Adver-sarial Network (GAN) to generate target fonts by combining the decoupled content and style representations. The complicated structure and detailed style are simultaneously generated in those methods, which may be the sub-optimal solutions for FFG task. Inspired by most manual font design processes of expert designers, in this paper, we model font generation as a multi-stage generative process. Specifically, as the injected noise and the data distribution in diffusion models can be well-separated into different sub-spaces, we are able to incorporate the font transfer process into these models. Based on this observation, we generalize diffusion methods to modelfont generative process by separating the reverse diffusion process into three stages with different functions: The structure construction stage first generates the structure information for the target character based on the source image, and the font transfer stage subsequently transforms the source font to the target font. Finally, the font refinement stage enhances the appearances and local details of the target font images. Based on the above multi-stage generative process, we construct our font generation framework. named MSD-Font, with a dual-network approach to generate font images. The superior performance demonstrates the effectiveness of our model. The code is available at: https://github.com/fubinfbIMSD-Font.

As a branch of clustering, multi-view clustering has received much attention in recent years. In practical applications, a common phenomenon is that partial views of some samples may be missing in the collected multi-view data, which poses a severe challenge to design the multi-view learning model and explore complementary and consistent information. Currently, most of the incomplete multi-view clustering methods only focus on exploring the information of available views while few works study the missing view recovery for incomplete multi-view learning. To this end, we propose an innovative diffusion-based missing view generation (DMVG) network. Moreover, for the scenarios with high missing rates, we further propose an incomplete multi-view data augmentation strategy to enhance the recovery quality for the missing views. Extensive experimental results show that the proposed DMVG can not only accurately predict missing views, but also further enhance the subsequent clustering performance in comparison with several state-of-the-art incomplete multi-view clustering methods.

The difficulty of partial multi-view multi-label learning lies in coupling the consensus of multi-view data with the task relevance of multi-label classification, under the condition where partial views and labels are unavailable. In this paper, we seek to compress cross-view representation to maximize the proportion of shared information to better predict semantic tags. To achieve this, we establish a model consistent with the information bottleneck theory for learning cross-view shared representation, minimizing non-shared information while maintaining feature validity to help increase the purity of task-relevant information. Furthermore, we model multi-label prototype instances in the latent space and learn label correlations in a data-driven manner. Our method outperforms existing state-of-the-art methods on multiple public datasets while exhibiting good compatibility with both partial and complete data. Finally, we experimentally reveal the importance of condensing shared information under the premise of information balancing, in the process of multi-view information encoding and compression.

Large-scale pre-trained vision-language models (e.g., CLIP) have shown powerful zero-shot transfer capabilities in image recognition tasks. Recent approaches typically employ supervised fine-tuning methods to adapt CLIP for zero-shot multi-label image recognition tasks. However, obtaining sufficient multi-label annotated image data for training is challenging and not scalable. In this paper, we propose a new language-driven framework for zero-shot multi-label recognition that eliminates the need for annotated images during training. Leveraging the aligned CLIP multi-modal embedding space, our method utilizes language data generated by LLMs to train a cross-modal classifier, which is subsequently transferred to the visual modality. During inference, directly applying the classifier to visual inputs may limit performance due to the modality gap. To address this issue, we introduce a cross-modal mapping method that maps image embeddings to the language modality while retaining crucial visual information. Comprehensive experiments demonstrate that our method outperforms other zero-shot multi-label recognition methods and achieves competitive results compared to few-shot methods.

As a more challenging domain adaptation setting, universal domain adaptation (UniDA) introduces category shift on top of domain shift, which needs to identify unknown category in the target domain and avoid misclassifying target samples into source private categories. To this end, we propose a novel UniDA approach named Batch Singular value Polarization and Weighted Semantic Augmentation (BSP-WSA). Specifically, we adopt an adversarial classifier to identify the target unknown category and align feature distributions between the two domains. Then, we propose to perform SVD on the classifier's outputs to maximize larger singular values while minimizing those smaller ones, which could prevent target samples from being wrongly assigned to source private classes. To better bridge the domain gap, we propose a weighted semantic augmentation approach for UniDA to generate data on common categories between the two domains. Extensive experiments on three benchmarks demonstrate that BSP-WSA could outperform existing state-of-the-art UniDA approaches.

Sketch Re-identification (Sketch Re-ID), which aims to retrieve target person from an image gallery based on a sketch query, is crucial for criminal investigation, law enforcement, and missing person searches. Existing methods aim to alleviate the modality gap by employing semantic metrics constraints or auxiliary modal guidance. However, they incur expensive labor costs and inevitably omit fine-grained modality-consistent information due to the abstraction of sketches. To address this issue, this paper proposes a novel $\textit{Optimal Transport-based Labor-free Text Prompt Modeling}$ (OLTM) network, which hierarchically extracts coarse- and fine-grained similarity representations guided by textual semantic information without any additional annotations. Specifically, multiple target attributes are flexibly obtained by a pre-trained visual question answering (VQA) model. Subsequently, a text prompt reasoning module employs learnable prompt strategy and optimal transport algorithm to extract discriminative global and local text representations, which serve as a bridge for hierarchical and multi-granularity modal alignment between sketch and image modalities. Additionally, instead of measuring the similarity of two samples by only computing their distance, a novel triplet assignment loss is further proposed, in which the whole data distribution also contributes to optimizing the inter/intra-class distances. Extensive experiments conducted on two public benchmarks consistently demonstrate the robustness and superiority of our OLTM over state-of-the-art methods.

Multi-view representation learning, aimed at uncovering the inherent structure within multi-view data, has developed rapidly in recent years. In practice, due to temporal and spatial desynchronization, it is common that only part of the data is aligned between views, which leads to thePartial View Alignment(PVA) problem. To address the challenge of representation learning on partially view-aligned multi-view data, we propose a new cross-view graph contrastive learning network, which integrates multi-view information to align data and learn latent representations. First, view-specific autoencoders are used to construct an end-to-end multi-view representation learning framework for learning specific view representations. Furthermore, to achieve cluster-level alignment, we introduce a cross-view graph contrastive learning module to guide the learning of discriminative representations. Compared to the existing methods, the proposed cluster-level alignment method successfully extends the view alignment to more than two views. Meanwhile, the results of clustering and classification experiments on several popular multi-view datasets can also illustrate the effectiveness and superiority of the proposed method.

In recent years, many incomplete multi-view clustering methods have been proposed to address the challenging and new clustering task on incomplete multi-view data whose part of view representations are not fully collected for some samples. Although extensive experiments have validated the effectiveness of these methods for handling the incomplete learning issue, a common issue exists, i.e., these methods all ignore the discriminative/important difference of discriminative features and noisy features. In this paper, to address the above issue, a new incomplete multi-view clustering model, called Graph Regularized and fEature Aware maTrix Factorization (GreatF), is proposed. Different from the existing methods, we introduce an adaptive feature weighting constraint to the matrix factorization-based multi-view representation learning model. With this weighting constraint, the effect of the discriminative features can be enhanced while the negative effect caused by the redundant and noisy features can be eliminated for the model optimization; thus, the robustness of the model can be enhanced. In addition, in this work, we designed a new graph-embedded consensus representation learning term in which consensus representation learning and structure information preservation are integrated into a joint model with one term. In particular, this term provides a more concise approach to obtain the structured consensus representation from incomplete multi-view data. Experimental results on four well-known datasets demonstrate that GreatF performs better than the state-of-the-art incomplete multi-view clustering methods.

Weakly supervised video anomaly detection is generally formulated as a multiple instance learning (MIL) problem, where an anomaly detector learns to generate frame-level anomaly scores under the supervision of MIL-based video-level classification. However, most previous works suffer from two drawbacks: 1) they lack ability to model temporal relationships between video segments and 2) they cannot extract sufficient discriminative features to separate normal and anomalous snippets. In this article, we develop a weakly supervised temporal discriminative (WSTD) paradigm, that aims to leverage both temporal relation and feature discrimination to mitigate the above drawbacks. To this end, we propose a transformer-styled temporal feature aggregator (TTFA) and a self-guided discriminative feature encoder (SDFE). Specifically, TTFA captures multiple types of temporal relationships between video snippets from different feature subspaces, while SDFE enhances the discriminative powers of features by clustering normal snippets and maximizing the separability between anomalous snippets and normal centers in embedding space. Experimental results on three public benchmarks indicate that WSTD outperforms state-of-the-art unsupervised and weakly supervised methods, which verifies the superiority of the proposed method.

Palmprint recognition has shown great value for biometric recognition due to its advantages of good hygiene, semi-privacy and low invasiveness. However, most existing palmprint recognition studies focus only on homogeneous palmprint recognition, where comparing palmprint images are collected under similar conditions with small domain gaps. To address the problem of matching heterogeneous palmprint images captured under the visible light (VIS) and the near-infrared (NIR) spectrum with large domain gaps, in this paper, we propose a Fourier-based feature learning network (FFLNet) for VIS-NIR heterogeneous palmprint recognition. First, we extract the multi-scale shallow representations of heterogeneous palmprint images via three vanilla convolution layers. Then, we convert the shallow palmprint feature maps into frequency-specific representations via Fourier transform to separate different layers of palmprint features, and exploit the underlying common and palmprint-specific frequency information of heterogeneous palmprint images. This effectively reduces the modality gap of heterogeneous palmprint images at the feature level. After that, we convert the common frequency-specific feature maps back to the spatial domain to learn the identity-invariant discriminative features via residual convolution for heterogeneous palmprint recognition. Extensive experimental results on three challenging heterogeneous palmprint databases clearly demonstrate the effectiveness of the proposed FFLNet for VIS-NIR heterogeneous palmprint recognition.

Consensus representation learning is one of the most popular approaches in the field of multi-view clustering. However, most of the existing methods cannot learn discriminative representations with a clustering-friendly structure since these methods ignore the separation among clusters and the compactness within each cluster. To tackle this issue, we propose a new deep multi-view clustering network with a dual contrastive mechanism to learn clustering-friendly representations. Specifically, our method employs dual contrasting losses: a dynamic cluster diffusion loss to maximize the distance between different clusters and a reliable neighbor-guided positive alignment loss to enhance compactness within each cluster. Our approach includes several key components: view-specific encoders to extract high-level features from each view, and an adaptive feature fusion strategy to obtain consensus representations across multiple views. The dynamic cluster diffusion module ensures inter-cluster separation by maximizing distances between different clusters in the consensus feature space. Simultaneously, the reliable neighbor-guided positive alignment module improves within-cluster compactness through a pseudo-label and nearest neighbor structure-driven contrastive loss. Experimental results on several datasets show that our method can acquire clustering-friendly representations with both good properties of inter-cluster separation and within-cluster compactness, and outperforms the existing state-of-the-art approaches in clustering performance. Our source code is available at https://github.com/tweety1028/DCMVC.

Unconstrained palmprint images have shown great potential for recognition applications due to their lower restrictions regarding hand poses and backgrounds during contactless image acquisition. However, they face two challenges: 1) unclear palm contours and finger-valley points of unconstrained palmprint images make it difficult to locate landmarks to crop the palmprint region of interest (ROI); and 2) large intra-class diversities of unconstrained palmprint images hinder the learning of intra-class-invariant palmprint features. In this paper, we propose to directly extract the complete palmprint region as the ROI (CROI) using the detection-style CenterNet without requiring the detection of any landmarks, and large intra-class diversities may occur. To address this, we further propose a palmprint feature alignment and learning hybrid network (PalmALNet) for unconstrained palmprint recognition. Specifically, we first exploit and align the multi-scale shallow representation of unconstrained palmprint images via deformable convolution and alignment-aware supervision, such that the pixel gaps of the intra-class palmprint CROIs can be minimized in shallow feature space. Then, we develop multiple triple-attention learning modules by integrating spatial, channel, and self-attention operations into convolution to adaptively learn and highlight the latent identity-invariant palmprint information, enhancing the overall discriminative power of the palmprint features. Extensive experimental results on four challenging palmprint databases demonstrate the promising effectiveness of both the proposed PalmALNet and CROI for unconstrained palmprint recognition.

Since hand-print recognition, i.e., palmprint, finger-knuckle-print (FKP), and hand-vein, have significant superiority in user convenience and hygiene, it has attracted greater enthusiasm from researchers. Seeking to handle the long-standing interference factors, i.e., noise, rotation, shadow, in hand-print images, multi-view hand-print representation has been proposed to enhance the feature expression by exploiting multiple characteristics from diverse views. However, the existing methods usually ignore the high-order correlations between different views or fuse very limited types of features. To tackle these issues, in this paper, we present a novel tensorized multi-view low-rank approximation based robust hand-print recognition method (TMLA_RHR), which can dexterously manipulate the multi-view hand-print features to produce a high-compact feature representation. To achieve this goal, we formulate TMLA_RHR by two key components, i.e., aligned structure regression loss and tensorized low-rank approximation, in a joint learning model. Specifically, we treat the low-rank representation matrices of different views as a tensor, which is regularized with a low-rank constraint. It models the across information between different views and reduces the redundancy of the learned sub-space representations. Experimental results on eight real-world hand-print databases prove the superiority of the proposed method in comparison with other state-of-the-art related works.

In order to reduce the negative effect of missing data on clustering, incomplete multiview clustering (IMVC) has become an important research content in machine learning. At present, graph-based methods are widely used in IMVC, but these methods still have some defects. First, some of the methods overlook potential relationships across views. Second, most of the methods depend on local structure information and ignore the global structure information. Third, most of the methods cannot use both global structure information and potential information across views to adaptively recover the incomplete relationship structure. To address the above issues, we propose a unified optimization framework to learn reasonable affinity relationships, called low-rank graph completion-based IMVC (LRGR_IMVC). 1) Our method introduces adaptive graph embedding to effectively explore the potential relationship among views; 2) we append a low-rank constraint to adequately exploit the global structure information among views; and 3) this method unites related information within views, potential information across views, and global structure information to adaptively recover the incomplete graph structure and obtain complete affinity relationships. Experimental results on several commonly used datasets show that the proposed method achieves better clustering performance significantly than some of the most advanced methods.

Due to the rapid development of multimedia technology and sensor technology, multi-view clustering (MVC) has become a research hotspot in machine learning, data mining, and other fields and has been developed significantly in the past decades. Compared with single-view clustering, MVC improves clustering performance by exploiting complementary and consistent information among different views. Such methods are all based on the assumption of complete views, which means that all the views of all the samples exist. It limits the application of MVC, because there are always missing views in practical situations. In recent years, many methods have been proposed to solve the incomplete MVC (IMVC) problem and a kind of popular method is based on matrix factorization (MF). However, such methods generally cannot deal with new samples and do not take into account the imbalance of information between different views. To address these two issues, we propose a new IMVC method, in which a novel and simple graph regularized projective consensus representation learning model is formulated for incomplete multi-view data clustering task. Compared with the existing methods, our method not only can obtain a set of projections to handle new samples but also can explore information of multiple views in a balanced way by learning the consensus representation in a unified low-dimensional subspace. In addition, a graph constraint is imposed on the consensus representation to mine the structural information inside the data. Experimental results on four datasets show that our method successfully accomplishes the IMVC task and obtain the best clustering performance most of the time. Our implementation is available at https://github.com/Dshijie/PIMVC.

Incomplete multiview clustering (IMC) is a hot and emerging topic. It is well known that unavoidable data incompleteness greatly weakens the effective information of multiview data. To date, existing IMC methods usually bypass unavailable views according to prior missing information, which is considered a second-best scheme based on evasion. Other methods that attempt to recover missing information are mostly applicable to specific two-view datasets. To handle these problems, in this article, we propose an information-recovery-driven-deep IMC network, termed as RecFormer. Concretely, a two-stage autoencoder network with self-attention structure is built to synchronously extract high-level semantic representations of multiple views and recover the missing data. Besides, we develop a recurrent graph reconstruction mechanism that cleverly leverages the restored views to promote representation learning and further data reconstruction. Visualization of recovery results are given and sufficient experimental results confirm that our RecFormer has obvious advantages over other top methods.

In this article, we propose a new linear regression (LR)-based multiclass classification method, called discriminative regression with adaptive graph diffusion (DRAGD). Different from existing graph embedding-based LR methods, DRAGD introduces a new graph learning and embedding term, which explores the high-order structure information between four tuples, rather than conventional sample pairs to learn an intrinsic graph. Moreover, DRAGD provides a new way to simultaneously capture the local geometric structure and representation structure of data in one term. To enhance the discriminability of the transformation matrix, a retargeted learning approach is introduced. As a result of combining the above-mentioned techniques, DRAGD can flexibly explore more unsupervised information underlying the data and the label information to obtain the most discriminative transformation matrix for multiclass classification tasks. Experimental results on six well-known real-world databases and a synthetic database demonstrate that DRAGD is superior to the state-of-the-art LR methods.

View missing and label missing are two challenging problems in the applications of multi-view multi-label classification scenery. In the past years, many efforts have been made to address the incomplete multi-view learning or incomplete multi-label learning problem. However, few works can simultaneously handle the challenging case with both the incomplete issues. In this article, we propose a new incomplete multi-view multi-label learning network to address this challenging issue. The proposed method is composed of four major parts: view-specific deep feature extraction network, weighted representation fusion module, classification module, and view-specific deep decoder network. By, respectively, integrating the view missing information and label missing information into the weighted fusion module and classification module, the proposed method can effectively reduce the negative influence caused by two such incomplete issues and sufficiently explore the available data and label information to obtain the most discriminative feature extractor and classifier. Furthermore, our method can be trained in both supervised and semi-supervised manners, which has important implications for flexible deployment. Experimental results on five benchmarks in supervised and semi-supervised cases demonstrate that the proposed method can greatly enhance the classification performance on the difficult incomplete multi-view multi-label classification tasks with missing labels and missing views.

Weakly supervised object detection (WSOD) has become an effective paradigm, which requires only class labels to train object detectors. However, WSOD detectors are prone to learn highly discriminative features corresponding to local objects rather than complete objects, resulting in imprecise object localization. To address the issue, designing backbones specifically for WSOD is a feasible solution. However, the redesigned backbone generally needs to be pretrained on large-scale ImageNet or trained from scratch, both of which require much more time and computational costs than fine-tuning. In this article, we explore to optimize the backbone without losing the availability of the original pretrained model. Since the pooling layer summarizes neighborhood features, it is crucial to spatial feature learning. In addition, it has no learnable parameters, so its modification will not change the pretrained model. Based on the above analysis, we further propose enhanced spatial feature learning (ESFL) for WSOD, which first takes full advantage of multiple kernels in a single pooling layer to handle multiscale objects and then enhances above-average activations within the rectangular neighborhood to alleviate the problem of ignoring unsalient object parts. The experimental results on the PASCAL VOC and the MS COCO benchmarks demonstrate that ESFL can bring significant performance improvement for the WSOD method and achieve state-of-the-art results.

Low-quality palmprint images will degrade the recognition performance, when they are captured under the open, unconstraint, and low-illumination conditions. Moreover, the traditional single-view palmprint representation methods have been difficult to express the characteristics of each palm strongly, where the palmprint characteristics become weak. To tackle these issues, in this article, we propose a structure suture learning-based robust multiview palmprint recognition method (SSL_RMPR), which comprehensively presents the salient palmprint features from multiple views. Unlike the existing multiview palmprint representation methods, SSL_RMPR introduces a structure suture learning strategy to produce an elastic nearest neighbor graph (ENNG) on the reconstruction errors that simultaneously exploit the label information and the latent consensus structure of the multiview data, such that the discriminant palmprint representation can be adaptively enhanced. Meanwhile, a low-rank reconstruction term integrating with the projection matrix learning is proposed, in such a manner that the robustness of the projection matrix can be improved. Particularly, since no extra structure capture term is imposed into the proposed model, the complexity of the model can be greatly reduced. Experimental results have proven the superiority of the proposed SSL_RMPR by achieving the best recognition performances on a number of real-world palmprint databases.

Palmprint has attracted increasing attention for biometric recognition in recent years due to its outstanding reliability, user-friendliness and hygiene. However, existing palmprint recognition methods usually require high-quality palmprint images with clear texture and line patterns; however, in practical applications palmprint images are usually of low quality. In this study, we propose a dense hybrid attention (DHA) network for palmprint image super-resolution (SR) by recovering the clear palmprint-specific characteristics. The proposed DHA network first obtains the high-dimensional shallow representation via a single convolution layer, and then jointly learns the local and global palmprint-specific features via parallel convolutional neural network (CNN)-and transformer-based branches. Particularly, we develop two enhanced spatial and channel attention (CA) modules to adaptively emphasize the local position-specific characteristics of palmprints, such that the SR palmprint images can be well recovered with clear texture and edge characteristics. Experimental results on three publicly used palmprint databases clearly show the effectiveness of the proposed method for palmprint image SR.

In recent years, multi-view multi-label learning has aroused extensive research enthusiasm. However, multi-view multi-label data in the real world is commonly incomplete due to the uncertain factors of data collection and manual annotation, which means that not only multi-view features are often missing, and label completeness is also difficult to be satisfied. To deal with the double incomplete multi-view multi-label classification problem, we propose a deep instance-level contrastive network, namely DICNet. Different from conventional methods, our DICNet focuses on leveraging deep neural network to exploit the high-level semantic representations of samples rather than shallow-level features. First, we utilize the stacked autoencoders to build an end-to-end multi-view feature extraction framework to learn the view-specific representations of samples. Furthermore, in order to improve the consensus representation ability, we introduce an incomplete instance-level contrastive learning scheme to guide the encoders to better extract the consensus information of multiple views and use a multi-view weighted fusion module to enhance the discrimination of semantic features. Overall, our DICNet is adept in capturing consistent discriminative representations of multi-view multi-label data and avoiding the negative effects of missing views and missing labels. Extensive experiments performed on five datasets validate that our method outperforms other state-of-the-art methods.

As we all know, multi-view data is more expressive than single-view data and multi-label annotation enjoys richer supervision information than single-label, which makes multi-view multi-label learning widely applicable for various pattern recognition tasks. In this complex representation learning problem, three main challenges can be characterized as follows: i) How to learn consistent representations of samples across all views? ii) How to exploit and utilize category correlations of multi-label to guide inference? iii) How to avoid the negative impact resulting from the incompleteness of views or labels? To cope with these problems, we propose a general multi-view multi-label learning framework named label-guided masked view- and category-aware transformers in this paper. First, we design two transformer-style based modules for cross-view features aggregation and multi-label classification, respectively. The former aggregates information from different views in the process of extracting view-specific features, and the latter learns subcategory embedding to improve classification performance. Second, considering the imbalance of expressive power among views, an adaptively weighted view fusion module is proposed to obtain view-consistent embedding features. Third, we impose a label manifold constraint in sample-level representation learning to maximize the utilization of supervised information. Last but not least, all the modules are designed under the premise of incomplete views and labels, which makes our method adaptable to arbitrary multi-view and multi-label data. Extensive experiments on five datasets confirm that our method has clear advantages over other state-of-the-art methods.

Diabetic retinopathy (DR) is the main cause of irreversible blindness for working-age adults. The previous models for DR detection have difficulties in clinical application. The main reason is that most of the previous methods only use single-view data, and the single field of view (FOV) only accounts for about 13% of the FOV of the retina, resulting in the loss of most lesion features. To alleviate this problem, we propose a multi-view model for DR detection, which takes full advantage of multi-view images covering almost all of the retinal field. To be specific, we design a Cross-Interaction Self-Attention based Module (CISAM) that interfuses local features extracted from convolutional blocks with long-range global features learned from transformer blocks. Furthermore, considering the pathological association in different views, we use the feature jigsaw to assemble and learn the features of multiple views. Extensive experiments on the latest public multi-view MFIDDR dataset with 34,452 images demonstrate the superiority of our method, which performs favorably against state-of-the-art models. To the best of our knowledge, this work is the first study on the public large-scale multi-view fundus images dataset for DR detection.

Most of the existing incomplete multi-view clustering (IMVC) methods focus on attaining a consensus representation from different views but ignore the important information hidden in the missing views and the latent intrinsic structures in each view. To tackle these issues, in this paper, a unified and novel framework, named tensorized incomplete multi-view clustering with intrinsic graph completion (TIMVC_IGC) is proposed. Firstly, owing to the effectiveness of the low-rank representation in revealing the inherent structure of the data, we exploit it to infer the missing instances and construct the complete graph for each view. Afterwards, inspired by the structural consistency, a between-view consistency constraint is imposed to guarantee the similarity of the graphs from different views. More importantly, the TIMVC_IGC simultaneously learns the low-rank structures of the different views and explores the correlations of the different graphs in a latent manifold sub-space using a low-rank tensor constraint, such that the intrinsic graphs of the different views can be obtained. Finally, a consensus representation for each sample is gained with a co-regularization term for final clustering. Experimental results on several real-world databases illustrates that the proposed method can outperform the other state-of-the-art related methods for incomplete multi-view clustering.

Graph-based multi-view clustering has attracted extensive attention because of the powerful clustering-structure representation ability and noise robustness. Considering the reality of a large amount of incomplete data, in this paper, we propose a simple but effective method for incomplete multi-view clustering based on consensus graph learning, termed as HCLS_CGL. Unlike existing methods that utilize graph constructed from raw data to aid in the learning of consistent representation, our method directly learns a consensus graph across views for clustering. Specifically, we design a novel confidence graph and embed it to form a confidence structure driven consensus graph learning model. Our confidence graph is based on an intuitive similar-nearest-neighbor hypothesis, which does not require any additional information and can help the model to obtain a high-quality consensus graph for better clustering. Numerous experiments are performed to confirm the effectiveness of our method.

In recent years, many incomplete multi-view clustering methods have been proposed to address the challenging unsupervised clustering issue on the multi-view data with missing views. However, most of the existing works are inapplicable to large-scale clustering task and their clustering results are unstable since these methods have high computational complexities and their results are produced by kmeans rather than their designed learning models. In this paper, we propose a new one-step incomplete multi-view clustering model, called Localized and Balanced Incomplete Multi-view Clustering (LBIMVC), to address these issues. Specifically, LBIMVC develops a new graph regularized incomplete multi-matrix-factorization model to obtain the unique clustering result by learning a consensus probability representation, where each element of the consensus representation can directly reflect the probability of the corresponding sample to the class. In addition, the proposed graph regularized model integrates geometric preserving and consensus representation learning into one term without introducing any extra constraint terms and parameters to explore the structure of data. Moreover, to avoid that samples are over divided into a few clusters, a balanced constraint is introduced to the model. Experimental results on four databases demonstrate that our method not only obtains competitive clustering performance, but also performs faster than some state-of-the-art methods.

Advanced deep multi-view subspace clustering methods are based on the self-expressive model, which has achieved impressive performance. However, most existing works have several limitations: 1) They endure high computational complexity when learning a consistent affinity matrix, impeding their capacity to handle large-scale multi-view data; 2) The global and local structure information of multi-view data remains under-explored. To tackle these challenges, we propose a simplistic but comprehensive framework called Multi-view Self-Expressive Subspace Clustering (MSESC) network. Specifically, we design a deep metric network to replace the conventional self-expressive model, which can directly and efficiently produce the intrinsic similarity values of any instance-pairs of all views. Moreover, our method explores global and local structure information from the connectivity of instance-pairs across views and the nearest neighbors of instance-pairs within the view, respectively. By integrating global and local structure information within a unified framework, MSESC can learn a high-quality shared affinity matrix for better clustering performance. Extensive experimental results indicate the superiority of MSESC compared to several state-of-the-art methods.

In this article, we propose a novel and effective incomplete multi-view clustering (IMVC) framework, referred to as incomplete multi-view clustering with regularized hierarchical graph (IMVC_RHG). Different from the existing graph learning-based IMVC methods, IMVC_RHG introduces a novel heterogeneous-graph learning and embedding strategy, which adopts the high-order structures between four tuples for each view, rather than a simple paired-sample intrinsic structure. Besides this, with the aid of the learned heterogeneous graphs, a between-view preserving strategy is designed to recover the incomplete graph for each view. Finally, a consensus representation for each sample is gained with a co-regularization term for final clustering. As a result of integrating these three learning strategies, IMVC_RHG can be flexibly applied to different types of IMVC tasks. Comparing with the other state-of-the-art methods, the proposed IMVC_RHG can achieve the best performances on real-world incomplete multi-view databases.

Multi-view learning has become a popular research topic in recent years, but research on the cross-application of classic multi-label classification and multi-view learning is still in its early stages. In this paper, we focus on the complex yet highly realistic task of incomplete multi-view weak multi-label learning and propose a masked two-channel decoupling framework based on deep neural networks to solve this problem. The core innovation of our method lies in decoupling the single-channel view-level representation, which is common in deep multi-view learning methods, into a shared representation and a view-proprietary representation. We also design a cross-channel contrastive loss to enhance the semantic property of the two channels. Additionally, we exploit supervised information to design a label-guided graph regularization loss, helping the extracted embedding features preserve the geometric structure among samples. Inspired by the success of masking mechanisms in image and text analysis, we develop a random fragment masking strategy for vector features to improve the learning ability of encoders. Finally, it is important to emphasize that our model is fully adaptable to arbitrary view and label absences while also performing well on the ideal full data. We have conducted sufficient and convincing experiments to confirm the effectiveness and advancement of our model.

Therapeutic peptide prediction is critical for drug development and therapeutic therapy. Researchers have developed several computational methods to identify different therapeutic peptide types. However, most computational methods focus on identifying the specific type of therapeutic peptides and fail to accurately predict all types of therapeutic peptides. Moreover, it is still challenging to utilize different properties features to predict the therapeutic peptides. In this study, a novel stacking framework PreTP-Stack is proposed for predicting different types of therapeutic peptide. PreTP-Stack is constructed based on ten different features and four predictors (Random Forest, Linear Discriminant Analysis, XGBoost and Support Vector Machine). Then the proposed method constructs an auto-weighted multi-view learning model as a final meta-classifier to enhance the performance of the basic models. Experimental results showed that the proposed method achieved better or highly comparable performance with the state-of-the-art methods for predicting eight types of therapeutic peptides A user-friendly web-server predictor is available at http://bliulab.net/PreTP-Stack.

In this paper, we consider the problem of multi-view clustering on incomplete views. Compared with complete multi-view clustering, the view-missing problem increases the difficulty of learning common representations from different views. To address the challenge, we propose a novel incomplete multi-view clustering framework, which incorporates cross-view relation transfer and multi-view fusion learning. Specifically, based on the consistency existing in multi-view data, we devise a cross-view relation transfer-based completion module, which transfers known similar inter-instance relationships to the missing view and infers the missing data via graph networks based on the transferred relationship graph. Then the view-specific encoders are designed to extract the recovered multi-view data, and an attention-based fusion layer is introduced to obtain the common representation. Moreover, to reduce the impact of the error caused by the inconsistency between views and obtain a better clustering structure, a joint clustering layer is introduced to optimize recovery and clustering simultaneously. Extensive experiments conducted on several real datasets demonstrate the effectiveness of the proposed method.

With the growing interest in convex and nonconvex low-rank matrix learning problems, the widely used singular value thresholding (SVT) operators associated with rank relaxation functions often face higher computational complexity, particularly for large-scale data matrices. To improve the efficacy of low-rank subspace clustering and overcome the issue of high computational complexity, this work proposes an efficient and effective method that avoids the need for singular value decomposition (SVD) computations in the iteration scheme. This can be achieved through the use of a computationally efficient and compact formulation, as well as automatic removal of the optimal mean, which reduces time consumption and enhances evaluation performance. A unified clustering framework based on Schatten-$p$norm regularized by$\ell _{2,q}$-norm can be formulated using this processing way, where inner element suppression can be achieved by choosing appropriate$p$,$q \in (0,1)$. Additionally, calculating the optimal mean enhances the robustness of the proposed method in the presence of outliers. Unlike the general iteration scheme of the alternating direction method of multiplier (ADMM) algorithms that introduce auxiliary splitting variables, the proposed alternating re-weighted least square (ARwLS) algorithm uses matrix inverse and multiplication computations to obtain analytic solutions, resulting in faster processing speeds for each sub-problem. To further investigate, we provide the computational complexity of each iteration and the theoretical analysis of the convergence property, where the derived solution is a stationary point. Experimental results on synthetic data and several benchmark datasets demonstrate the promising efficiency and efficacy of the proposed clustering method compared to classical and competing algorithms.

Latent low-rank representation (LatLRR) is a critical self-representation technique that improves low-rank representation (LRR) by using observed and unobserved samples. It can simultaneously learn the low-dimensional structure embedded in the data space and capture the salient features. However, LatLRR ignores the local geometry structure and can be affected by the noise and redundancy in the original data space. To solve the above problems, we propose a latent LRR with weighted distance penalty (LLRRWD) for clustering in this article. First, a weighted distance is proposed to enhance the original Euclidean distance by enlarging the distance among the unconnected samples, which can enhance the discriminitation of the distance among the samples. By leveraging on the weighted distance, a weighted distance penalty is introduced to the LatLRR model to enable the method to preserve both the local geometric information and global information, improving discrimination of the learned affinity matrix. Moreover, a weight matrix is imposed on the sparse error norm to reduce the effect of noise and redundancy. Experimental results based on several benchmark databases show the effectiveness of our method in clustering.

Weakly supervised object detection (WSOD) has received widespread attention since it requires only image-category annotations for detector training. Many advanced approaches solve this problem by a two-phase learning framework, that is, instance mining that classifies generated proposals via multiple instance learning, and instance refinement that iteratively refines bounding boxes using the supervision produced by the preceding stage. In this paper, we observe that the detection performance is usually limited by imprecise supervision, including part domination and untight boxes. To mitigate their adverse effects, we focus on selecting high-quality proposals as the supervision for WSOD. To be specific, for the issue of part domination, we propose bottom-up aggregated attention which incorporates low-level features from shallow layers to improve location representation of top-level features. In this manner, the proposals corresponding to entire objects can get high scores. Its advantage is that it can be flexibly plugged into the WSOD framework since there is no need to attach learnable parameters or learning branches. As regards the problem of untight boxes, we propose a phase-aware loss, which is the first work to measure supervision quality by the loss in the instance mining phase, to highlight correct boxes and suppress untight ones. In this work, we unify the proposed two modules into the framework of online instance classifier refinement. Extensive experiments on the PASCAL VOC and the MS COCO demonstrate that our method can significantly improve the performance of WSOD and achieve the state-of-the-art results. The code is available at https://github.com/Horatio9702/BUAA_PALoss.

Incomplete multi-view clustering, which aims to solve the clustering problem on the incomplete multi-view data with partial view missing, has received more and more attention in recent years. Although numerous methods have been developed, most of the methods either cannot flexibly handle the incomplete multi-view data with arbitrary missing views or do not consider the negative factor of information imbalance among views. Moreover, some methods do not fully explore the local structure of all incomplete views. To tackle these problems, this paper proposes a simple but effective method, named localized sparse incomplete multi-view clustering (LSIMVC). Different from the existing methods, LSIMVC intends to learn a sparse and structured consensus latent representation from the incomplete multi-view data by optimizing a sparse regularized and novel graph embedded multi-view matrix factorization model. Specifically, in such a novel model based on the matrix factorization, a norm based sparse constraint is introduced to obtain the sparse low-dimensional individual representations and the sparse consensus representation. Moreover, a novel local graph embedding term is introduced to learn the structured consensus representation. Different from the existing works, our local graph embedding term aggregates the graph embedding task and consensus representation learning task into a concise term. Furthermore, to reduce the imbalance factor of incomplete multi-view learning, an adaptive weighted learning scheme is introduced to LSIMVC. Comprehensive experimental results performed on six incomplete multi-view databases verify that the performance of our LSIMVC is superior to the state-of-the-art IMC approaches.

With the diversity of information acquisition, data is stored and transmitted in an increasing number of modalities. Nevertheless, it is not unusual for parts of the data to be lost in some views due to unavoidable acquisition, transmission or storage errors. In this paper, we propose an augmentation-free graph contrastive learning framework to solve the problem of partial multi-view clustering. Notably, we suppose that the representations of similar samples (i.e., belonging to the same cluster) should be similar. This is distinct from the general unsupervised contrastive learning that assumes an image and its augmentations share a similar representation. Specifically, relation graphs are constructed using the nearest neighbors to identify existing similar samples, then the constructed inter-instance relation graphs are transferred to the missing views to build graphs on the corresponding missing data. Subsequently, two main components, within-view graph contrastive learning and cross-view graph consistency learning, are devised to maximize the mutual information of different views within a cluster. The proposed approach elevates instance-level contrastive learning and missing data inference to the cluster-level, effectively mitigating the impact of individual missing data on clustering. Experiments on several challenging datasets demonstrate the superiority of our proposed methods.

Weakly supervised object detection (WSOD) aims to train object detectors by using only image-level annotations. Many recent works on WSOD adopt multiple instance detection networks (MIDN), which usually generate a certain number of proposals and regard proposal classification as a latent model learning within image classification. However, these methods tend to detect salient object, salient object parts and clustered objects due to lack of instance-level annotations during training. Thus a core issue is how to guarantee that the network learn as many objects with precise bounding boxes as possible. In this paper, we address this issue by exploiting the potential of proposal scores during training. We propose an adaptive instance refinement (AIR) framework with three novel designs, which can be integrated with MIDN into a single network. Specifically, adaptive instance mining attempts to discover all positive instances according to the score distribution of proposals and their spatial similarity. Adaptive score modulation dynamically adjusts proposal scores to make the network focus more on instances with different difficulties in different training iterations. Adaptive knowledge refinement distills important information from all previous stages by the weighted average of proposal scores. The experimental results on the PASCAL VOC 2007 and 2012 benchmarks and the MS COCO benchmark demonstrate that AIR significantly improves the performance of the original MIDN and achieves the state-of-the-art results.

In the real-world, some views of samples are often missing for the collected multiview data. Faced with the incomplete multiview data, most of the existing clustering methods tended to learn a common graph from the available views, where the hidden information of the absent views was ignored. Furthermore, some methods filled the absent instances with the average vector of the available samples for each view, which could not reflect a real distribution of the data. To solve these problems, in this paper an intrinsic and complete structure learning based incomplete multiview clustering method (ICSL_IMC) is proposed. Firstly, we calculate the initial complete graphs for all views by exploring the available incomplete graphs, which are further taken as the constraints for the reconstruction of the absent data integrating the self-representation method. Afterwards, encouraged by the complete multiview data, a complete structure inferring strategy is proposed to learn the intrinsic and complete structures for all views, such that the real distribution of the absent instances can be reflected in the completed structure of each view. We integrate these three learning phases into a joint optimization model, which can promote each other in the iterative learning procedure, simultaneously. Comparing with the other state-of-the-art methods, the proposed ICSL_IMC can achieve the best performances on different databases.

In this article, we propose a collaborative palmprint-specific binary feature learning method and a compact network consisting of a single convolution layer for efficient palmprint feature extraction. Unlike most existing palmprint feature learning methods, such as deep-learning, which usually ignore the inherent characteristics of palmprints and learn features from raw pixels of a massive number of labeled samples, palmprint-specific information, such as the direction and edge of patterns, is characterized by forming two kinds of ordinal measure vectors (OMVs). Then, collaborative binary feature codes are jointly learned by projecting double OMVs into complementary feature spaces in an unsupervised manner. Furthermore, the elements of feature projection functions are integrated into OMV extraction filters to obtain a collection of cascaded convolution templates that form a single-layer convolution network (SLCN) to efficiently obtain the binary feature codes of a new palmprint image within a single-stage convolution operation. Particularly, our proposed method can easily be extended to a general version that can efficiently perform feature extraction with more than two types of OMVs. Experimental results on five benchmark databases show that our proposed method achieves very promising feature extraction efficiency for palmprint recognition.

Video anomaly detection (VAD) refers to the discrimination of unexpected events in videos. The deep generative model (DGM)-based method learns the regular patterns on normal videos and expects the learned model to yield larger generative errors for abnormal frames. However, DGM cannot always do so, since it usually captures the shared patterns between normal and abnormal events, which results in similar generative errors for them. In this article, we propose a novel self-supervised framework for unsupervised VAD to tackle the above-mentioned problem. To this end, we design a novel self-supervised attentive generative adversarial network (SSAGAN), which is composed of the self-attentive predictor, the vanilla discriminator, and the self-supervised discriminator. On the one hand, the self-attentive predictor can capture the long-term dependences for improving the prediction qualities of normal frames. On the other hand, the predicted frames are fed to the vanilla discriminator and self-supervised discriminator for performing true-false discrimination and self-supervised rotation detection, respectively. Essentially, the role of the self-supervised task is to enable the predictor to encode semantic information into the predicted normal frames via adversarial training, in order for the angles of rotated normal frames can be detected. As a result, our self-supervised framework lessens the generalization ability of the model to abnormal frames, resulting in larger detection errors for abnormal frames. Extensive experimental results indicate that SSAGAN outperforms other state-of-the-art methods, which demonstrates the validity and advancement of SSAGAN.

Palmprint recognition provides a potential solution for noninvasive personal authentication due to its excellent contactless property and user-security, and it has attracted tremendous research interest in recent years. However, most existing methods focus on intraspectral palmprint recognition, which requires gallery and probe images to be captured under similar illumination, and thus significantly limit its practical applications in open environments with variant illuminations. In this study, we present a spectrum-invariant feature learning method for cross-spectral palmprint recognition to address the problem that gallery and probe samples are captured under different spectra. First, the blockwise direction-based ordinal measure vectors are formed to represent the intrinsic information of palmprint images. Then, a unified feature projection is jointly learned to map two different spectra of palmprint images into a common feature space, in which the different spectral features have enhanced discriminative power by enlarging their variances while the intraclass features learned from different spectral images are similar. The proposed method can be easily extended to seek the unified spectrum-invariant representation of multiple spectral palmprint images, making it feasible to perform palmprint recognition crossing one spectrum to multiple spectra. Experimental results on two multispectral palmprint image databases demonstrate the promising effectiveness of the proposed method on cross-spectral palmprint recognition.

Conventional multiview clustering seeks to partition data into respective groups based on the assumption that all views are fully observed. However, in practical applications, such as disease diagnosis, multimedia analysis, and recommendation system, it is common to observe that not all views of samples are available in many cases, which leads to the failure of the conventional multiview clustering methods. Clustering on such incomplete multiview data is referred to as incomplete multiview clustering (IMC). In view of the promising application prospects, the research of IMC has noticeable advances in recent years. However, there is no survey to summarize the current progresses and point out the future research directions. To this end, we review the recent studies of IMC. Importantly, we provide some frameworks to unify the corresponding IMC methods and make an in-depth comparative analysis for some representative methods from theoretical and experimental perspectives. Finally, some open problems in the IMC field are offered for researchers. The related codes are released athttps://github.com/DarrenZZhang/Survey_IMC.

Deep object detectors suffer from the gradient contribution imbalance during training. In this paper, we point out that such imbalance can be ascribed to the imbalance in example attributes, e.g., difficulty and shape variation degree. We further propose example attribute based prediction modulation (EAPM) to address it. In EAPM, first, the attribute of an example is defined by the prediction and the corresponding ground truth. Then, a modulating factor w.r.t the example attribute is introduced to modulate the prediction error. Finally, the new prediction and the ground-truth are input into the loss function. Essentially, we adjust the gradients of examples with specific attributes to reweight their contribution on the global gradients. We apply EAPM with focal loss and balanced L1 loss to simultaneously solve the imbalance in classification and localization. The experimental results on MS COCO demonstrate that EAPM can bring substantial improvement for deep object detectors.

Abnormal behavior detection in surveillance video is a fundamental task in modern public security. Different from typical pixel-based solutions, pose-based approaches leverage low-dimensional and strongly-structured skeleton feature, which enables the anomaly detector to be immune to complex background noise and obtain higher efficiency. However, existing pose-based methods only utilize the pose of each individual independently while ignore the important interactions between individuals. In this paper, we present a hierarchical graph embedded pose regularity learning framework via spatio-temporal transformer, which leverages the strength of graph representation in encoding strongly-structured skeleton feature. Specifically, skeleton feature is encoded as the hierarchical graph representation, which jointly models the interactions among multiple individuals and the correlations among body joints within the same individual. Furthermore, a novel task-specific spatial-temporal graph transformer is designed to encode the hierarchical spatio-temporal graph embeddings of human skeletons and learn the regular patterns within normal training videos. Experimental results indicate that our method obtains superior performance over state-of-the-art methods on several challenging datasets.

Pixel-level visual anomaly detection, which aims to recognize the abnormal areas from images, plays an important role in industrial fault detection and medical diagnosis. However, it is a challenging task due to the following reasons: i) the large variation of anomalies; and ii) the ambiguous boundary between anomalies and their normal surroundings. In this work, we present an uncertainty-aware prototypical transformer (UPformer), which takes into account both the diversity and uncertainty of anomaly to achieve accurate pixel-level visual anomaly detection. To this end, we first design a memory-guided prototype learning transformer encoder to learn and memorize the prototypical representations of anomalies for enabling the model to capture the diversity of anomalies. Additionally, an anomaly detection uncertainty quantizer is designed to learn the distributions of anomaly detection for measuring the anomaly detection uncertainty. Furthermore, an uncertainty-aware transformer decoder is proposed to leverage the detection uncertainties to guide the model to focus on the uncertain areas and generate the final detection results. As a result, our method achieves more accurate anomaly detection by combining the benefits of prototype learning and uncertainty estimation. Experimental results on five datasets indicate that our method achieves state-of-the-art anomaly detection performance.

Existing low-rank representation-based methods adopt a two-step framework, which must employ an extra clustering method to gain labels after representation learning. In this paper, a novel one-step representation-based method, i.e., One-step Low-Rank Representation (OLRR), is proposed to capture multi-subspace structures for clustering. OLRR integrates the low-rank representation model and clustering into a unified framework. Thus it can jointly learn the low-rank subspace structure embedded in the database and gain the clustering results. In particular, by approximating the representation matrix with two same clustering indicator matrices, OLRR can directly show the probability of samples belonging to each cluster. Further, a probability penalty is introduced to ensure that the samples with smaller distances are more inclined to be in the same cluster, thus enhancing the discrimination of the clustering indicator matrix and resulting in a more favorable clustering performance. Moreover, to enhance the robustness against noise, OLRR uses the probability to guide denoising and then performs representation learning and clustering in a recovered clean space. Extensive experiments well demonstrate the robustness and effectiveness of OLRR. Our code is publicly available at: https://github.com/fuzhiqiang1230/OLRR.

Center loss is widely used as a supervision tool in deep learning method. However, the center loss also has some shortcomings, the most important of which is that it must be combined with softmax loss to run well. In this article, we sum up five shortcomings of center loss and solve all of them by proposing a dual distance center loss (DDCL). Compared with center loss, DDCL can run without the combination of softmax to supervise training the model. In addition, we verify the inconsistency between the proposed DDCL and softmax loss in the feature space. To be specifically, we add the Pearson distance on the basis of the Euclidean distance to the same center, which makes all features of the same class be confined to the intersection of a hypersphere and a hypercone in the feature space, strengthens the intraclass compactness of the center loss, and enhances the generalization ability of center loss. Moreover, by designing a Euclidean distance threshold between all center pairs, we not only strengthen the interclass separability of center loss, but also make the center loss (or DDCL) works well without the combination of softmax loss. We verify the effectiveness of DDCL in four datasets, two of which are widely used in the field of vehicle re-identification named VeRi-776 dataset and VehicleID dataset, and two other datasets are widely used in the field of person re-identification named Market1501 dataset and MSMT17 dataset. The experimental results of the proposed DDCL exceed that of the softmax loss in all the four datasets, indicating that our proposed method not only can run without the combination of softmax, but also has a high accuracy.

Deep autoencoder (AE) has demonstrated promising performances in visual anomaly detection (VAD). Learning normal patterns on normal data, deep AE is expected to yield larger reconstruction errors for anomalous samples, which is utilized as the criterion for detecting anomalies. However, this hypothesis cannot be always tenable since the deep AE usually captures the low-level shared features between normal and abnormal data, which leads to similar reconstruction errors for them. To tackle this problem, we propose a self-supervised representation-augmented deep AE for unsupervised VAD, which can enlarge the gap of anomaly scores between normal and abnormal samples by introducing autoencoding transformation (AT). Essentially, AT is introduced to facilitate AE to learn the high-level visual semantic features of normal images by introducing a self-supervision task (transformation reconstruction). In particular, our model inputs the original and transformed images into the encoder for obtaining latent representations; afterward, they are fed to the decoder for reconstructing both the original image and applied transformation. In this way, our model can utilize both image and transformation reconstruction errors to detect anomaly. Extensive experiments indicate that the proposed method outperforms other state-of-the-art methods, which demonstrates the validity and advancement of our model.

The continuous developments of urban and industrial environments have increased the demand for intelligent video surveillance. Deep learning has achieved remarkable performance for anomaly detection in surveillance videos. Previous approaches achieve anomaly detection with a single-pretext task (image reconstruction or prediction) and detect anomalies by larger reconstruction error or poor prediction. However, they cannot fully exploit the discriminative semantics and temporal context information. Moreover, tackling anomaly detection with a single pretext task is suboptimal due to the nonalignment between the pretext task and anomaly detection. In this article, we propose a temporal-aware contrastive network (TAC-Net) to address the abovementioned problems of anomaly detection for intelligence video surveillance. TAC-Net is an unsupervised method that utilizes deep contrastive self-supervised learning to capture the high-level semantic features and tackles anomaly detection with multiple self-supervised tasks. During inference phase, the multiple task losses and contrastive similarity are utilized to calculate the anomaly score. Experimental results show that our method is superior to state-of-the-art approaches on three benchmarks, which demonstrates the validity and advancement of TAC-Net.

With the dramatic increase in the amount of multimedia data, cross-modal similarity retrieval has become one of the most popular yet challenging problems. Hashing offers a promising solution for large-scale cross-modal data searching by embedding the high-dimensional data into the low-dimensional similarity preserving Hamming space. However, most existing cross-modal hashing usually seeks a semantic representation shared by multiple modalities, which cannot fully preserve and fuse the discriminative modal-specific features and heterogeneous similarity for cross-modal similarity searching. In this paper, we propose a joint specifics and consistency hash learning method for cross-modal retrieval. Specifically, we introduce an asymmetric learning framework to fully exploit the label information for discriminative hash code learning, where 1) each individual modality can be better converted into a meaningful subspace with specific information, 2) multiple subspaces are semantically connected to capture consistent information, and 3) the integration complexity of different subspaces is overcome so that the learned collaborative binary codes can merge the specifics with consistency. Then, we introduce an alternatively iterative optimization to tackle the specifics and consistency hashing learning problem, making it scalable for large-scale cross-modal retrieval. Extensive experiments on five widely used benchmark databases clearly demonstrate the effectiveness and efficiency of our proposed method on both one-cross-one and one-cross-two retrieval tasks.

In this paper, we propose a novel method, referred to as incomplete multi-view tensor spectral clustering with missing-view inferring (IMVTSC-MVI) to address the challenging multi-view clustering problem with missing views. Different from the existing methods which commonly focus on exploring the certain information of the available views while ignoring both of the hidden information of the missing views and the intra-view information of data, IMVTSC-MVI seeks to recover the missing views and explore the full information of such recovered views and available views for data clustering. In particular, IMVTSC-MVI incorporates the feature space based missing-view inferring and manifold space based similarity graph learning into a unified framework. In such a way, IMVTSC-MVI allows these two learning tasks to facilitate each other and can well explore the hidden information of the missing views. Moreover, IMVTSC-MVI introduces the low-rank tensor constraint to capture the high-order correlations of multiple views. Experimental results on several datasets demonstrate the effectiveness of IMVTSC-MVI for incomplete multi-view clustering.

In recent years, incomplete multi-view clustering has drawn increasing attention due to the existence of large amounts of unlabeled incomplete data whose views are not fully observed in the practical applications. Although many traditional methods have been extended to address the incomplete learning problem, most of them exploit the shallow models and ignore the geometric structure. To address these issues, we proposed a structural deep incomplete multi-view clustering network. Specifically, the proposed method can simultaneously explore the high-level features and high-order geometric structure information of data with several view-specific graph convolutional encoder networks and can directly obtain the optimal clustering indicator matrix in one stage. Experimental results on several datasets with the comparison of state-of-the-art methods validate the superiority of the proposed method.

Cross-modal hashing has received increasing research attentions due to its less storage and efficient retrieval. However, most existing cross-modal hashing methods focus only on exploring multi-modal information, while underestimate the significance of local and Euclidean structure information on the hashing learning procedure. In this paper, we propose a supervised discrete-based cross-modal hashing method, named Scalable Discriminative Discrete Hashing (SDDH), for cross-modal retrieval, where 1) the discrete hash codes are directly obtained by multi-modal features and semantic labels so that the quantization errors are dramatically reduced, and 2) the discrete hash codes simultaneously preserve the heterogeneous similarity and manifold information in the original space by employing matrix factoring with orthogonal and balanced constraints. Moreover, an efficient optimization is introduced to tackle the discrete solution, which makes the SDDH scalable to large-scale cross-modal retrieval. Empirical results on three widely-used benchmark databases clearly demonstrate the effectiveness and efficiency of the proposed method in comparison with state-of-the-arts.

Cross-modal hashing has attracted much attention for cross-modal retrieval and achieved promising performance due to its powerful capacity. Some existing cross-modal hashing methods construct pairwise similarities to represent the relationship of heterogeneous data, which require much computation time and storage space, making them unscalable for large-scale retrieval tasks. In this paper, we propose a novel supervised Discrete Semantic Embedding Hashing (DSEH) for cross-modal retrieval. Specifically, we first learn the common representation of heterogeneous data by embedding the semantic labels into a collective matrix factorization, such that both intra- and inter-modality similarities can be well captured. Then, we learn the hash codes in the discrete space based on the learned common representation via an orthogonal rotation technique. Moreover, we learn the multi-modal hash functions that can efficiently convert out-of-sample instances into unified hash codes. Extensive experimental results on three widely used benchmark databases demonstrate the superiority of the proposed DSEH compared with previous state-of-the-arts.

Protein fold recognition is one of the most essential steps for protein structure prediction, aiming to classify proteins into known protein folds. There are two main computational approaches: one is the template-based method based on the alignment scores between query-template protein pairs and the other is the machine learning method based on the feature representation and classifier. These two approaches have their own advantages and disadvantages. Can we combine these methods to establish more accurate predictors for protein fold recognition? In this study, we made an initial attempt and proposed two novel algorithms: TSVM-fold and ESVM-fold. TSVM-fold was based on the Support Vector Machines (SVMs), which utilizes a set of pairwise sequence similarity scores generated by three complementary template-based methods, including HHblits, SPARKS-X, and DeepFR. These scores measured the global relationships between query sequences and templates. The comprehensive features of the attributes of the sequences were fed into the SVMs for the prediction. Then the TSVM-fold was further combined with the HHblits algorithm so as to improve its generalization ability. The combined method is called ESVM-fold. Experimental results in two rigorous benchmark datasets (LE and YK datasets) showed that the proposed methods outperform some state-of-the-art methods, indicating that the TSVM-fold and ESVM-fold are efficient predictors for protein fold recognition.

Protein fold recognition is critical for studies of the protein structure prediction and drug design. Several methods have been proposed to obtain discriminative features from the protein sequences for fold recognition. However, the ensemble methods that combine the various features to improve predictive performance remain the challenge problems. In this study, we proposed two novel algorithms: AWMG and EMfold. AWMG used a novel predictor based on the multi-view learning framework for fold recognition. Each view was treated as the intermediate representation of the corresponding data source of proteins, including the evolutionary information and the retrieval information. AWMG calculated the auto-weight for each view respectively and constructed the latent subspace which contains the common information shared by different views. The marginalized constraint was employed to enlarge the margins between different folds, improving the predictive performance of AWMG. Furthermore, we proposed a novel ensemble method called EMfold, which combines two complementary methods AWMG and DeepSS. The later method was a template-based algorithm using the SPARKS-X and DeepFR programs. EMfold integrated the advantages of template-based assignment and machine learning classifier. Experimental results on the two widely datasets (LE and YK) showed that the proposed methods outperformed some state-of-the-art methods, indicating that AWMG and EMfold are useful tools for protein fold recognition.

An important underlying assumption that guides the success of the existing multiview learning algorithms is the full observation of the multiview data. However, such rigorous precondition clearly violates the common-sense knowledge in practical applications, where in most cases, only incomplete fractions of the multiview data are given. The presence of the incomplete settings generally disables the conventional multiview clustering methods. In this article, we propose a simple but effective incomplete multiview clustering (IMC) framework, which simultaneously considers the local geometric information and the unbalanced discriminating powers of these incomplete multiview observations. Specifically, a novel graph-regularized matrix factorization model, on the one hand, is developed to preserve the local geometric similarities of the learned common representations from different views. On the other hand, the semantic consistency constraint is introduced to stimulate these view-specific representations toward a unified discriminative representation. Moreover, the importance of different views is adaptively determined to reduce the negative influence of the unbalanced incomplete views. Furthermore, an efficient learning algorithm is proposed to solve the resulting optimization problem. Extensive experimental results performed on several incomplete multiview datasets demonstrate that the proposed method can achieve superior clustering performance in comparison with some state-of-the-art multiview learning methods.

In this article, we propose a multifeature learning method to jointly learn compact multifeature codes (LCMFCs) for palmprint recognition with a single training sample per palm. Unlike most existing hand-crafted methods that extract single-type features from raw pixels, we first form the multi-type data vectors such as the direction-data, and texture-data to completely sample the multiple information of a palmprint image. Then, we learn the discriminative multifeatures from multi-type data vectors by maximizing the inter-palm distance, and minimizing the energy loss between the learned codes, and the original data. Moreover, our LCMFC method adaptively learns the optimal weights of multi-type features to jointly learn the compact multifeature codes. Finally, we cluster the nonoverlapping blockwise histograms of the compact multifeature codes into a feature vector for palmprint representation. Extensive experimental results on six benchmark palmprint databases are presented to show the effectiveness of the proposed method.

In real-world applications, it is often that the collected multi-view data are incomplete, i.e., some views of samples are absent. Existing clustering methods for incomplete multi-view data all focus on obtaining a common representation or graph from the available views but neglect the hidden information of missing views and information imbalance of different views. To solve these problems, a novel method, called adaptive graph completion based incomplete multi-view clustering (AGC_IMC), is proposed in this paper. Specifically, AGC_IMC develops a joint framework for graph completion and consensus representation learning, which mainly contains three components, i.e., within-view preservation, between-view inferring, and consensus representation learning. To reduce the negative influence of information imbalance, AGC_IMC introduces some adaptive weights to balance the importance of different views during the consensus representation learning. Importantly, AGC_IMC has the potential to recover the similarity graphs of all views with the optimal cluster structure, which encourages it to obtain a more discriminative consensus representation. Experimental results on five well-known datasets show that AGC_IMC significantly outperforms the state-of-the-art methods.

3D palmprint-based biometric recognition has drawn growing research attention due to its several merits over 2D counterpart such as robust structural measurement of a palm surface and high anti-counterfeiting capability. However, most existing 3D palmprint descriptors are hand-crafted that usually extract stationary features from 3D palmprint images. In this paper, we propose a feature learning method to jointly learn compact curvature feature descriptor for 3D palmprint recognition. We first form multiple curvature data vectors to completely sample the intrinsic curvature information of 3D palmprint images. Then, we jointly learn a feature projection function that project curvature data vectors into binary feature codes, which have the maximum inter-class variances and minimum intra-class distance so that they are discriminative. Moreover, we learn the collaborative binary representation of the multiple curvature feature codes by minimizing the information loss between the final representation and the multiple curvature features, so that the proposed method is more compact in feature representation and efficient in matching. Experimental results on the baseline 3D palmprint database demonstrate the superiority of the proposed method in terms of recognition performance in comparison with state-of-the-art 3D palmprint descriptors.

In recent years, incomplete multi-view clustering, which studies the challenging multi-view clustering problem on missing views, has received growing research interests. Although a series of methods have been proposed to address this issue, the following problems still exist: 1) Almost all of the existing methods are based on shallow models, which is difficult to obtain discriminative common representations. 2) These methods are generally sensitive to noise or outliers since the negative samples are treated equally as the important samples. In this paper, we propose a novel incomplete multi-view clustering network, called Cognitive Deep Incomplete Multi-view Clustering Network (CDIMC-net), to address these issues. Specifically, it captures the high-level features and local structure of each view by incorporating the view-specific deep encoders and graph embedding strategy into a framework. Moreover, based on the human cognition, \emph{i.e.}, learning from easy to hard, it introduces a self-paced strategy to select the most confident samples for model training, which can reduce the negative influence of outliers. Experimental results on several incomplete datasets show that CDIMC-net outperforms the state-of-the-art incomplete multi-view clustering methods.

In this paper, a new deep incomplete multi-view clustering network, called DIMC-net, is proposed to address the challenge of multi-view clustering on missing views. In particular, DIMC-net designs several view-specific encoders to extract the high-level information of multiple views and introduces a fusion graph based constraint to explore the local geometric information of data. To reduce the negative influence of missing views, a weighted fusion layer is introduced to obtain the consensus representation shared by all views. Moreover, a clustering layer is introduced to guarantee that the obtained consensus representation is the best one for the clustering task. Compared with the existing deep learning based approaches, DIMC-net is more flexible and efficient since it can handle all kinds of incomplete cases and directly produce the clustering results. Experimental results show that DIMC-net achieves significant improvement over state-of-the-art incomplete multi-view clustering methods.

Direction-based methods are the most powerful and popular palmprint recognition methods. However, there is no existing work that completely analyzes the essential differences among different direction-based methods and explores the most discriminant direction representation of a palmprint. In this paper, we attempt to establish the connection between the direction feature extraction model and the discriminability of direction features, and we propose a novel exponential and Gaussian fusion model (EGM) to characterize the discriminative power of different directions. The EGM can provide us with a new insight into the optimal direction feature selection of palmprints. Moreover, we propose a local discriminant direction binary pattern (LDDBP) to completely represent the direction features of a palmprint. Guided by the EGM, the most discriminant directions can be exploited to form the LDDBP-based descriptor for palmprint representation and recognition. Extensive experiment results conducted on four widely used palmprint databases demonstrate the superiority of the proposed LDDBP method over the state-of-the-art direction-based methods.

This paper proposes a novel discriminative regression method, called adaptive locality preserving regression (ALPR) for classification. In particular, ALPR aims to learn a more flexible and discriminative projection that not only preserves the intrinsic structure of data, but also possesses the properties of feature selection and interpretability. To this end, we introduce a target learning technique to adaptively learn a more discriminative and flexible target matrix rather than the pre-defined strict zero-one label matrix for regression. Then, a locality preserving constraint regularized by the adaptive learned weights is further introduced to guide the projection learning, which is beneficial to learn a more discriminative projection and avoid overfitting. Moreover, we replace the conventional `Frobenius norm' with the special l2,1norm to constrain the projection, which enables the method to adaptively select the most important features from the original high-dimensional data for feature extraction. In this way, the negative influence of the redundant features and noises residing in the original data can be greatly eliminated. Besides, the proposed method has good interpretability for features owing to the row-sparsity property of the l2,1norm. Extensive experiments conducted on the synthetic database with manifold structure and many real-world databases prove the effectiveness of the proposed method.

Graph based classification methods have been widely applied in the fields of computer vision and machine learning. The quality of the graph highly affects the performance of these methods. The same object is commonly represented by different features, i.e., multi-view features, which leads to multiple graphs corresponding to different features in multi-view learning. However, what kind of graph is important for the task is unknown in advance. Moreover, existing multi-view learning methods become weak in dealing with noisy graphs when the data is corrupted by the noise. In this paper, we address this problem by observing that the noise of each graph has specific structure. Then, based on this observation we propose a robust multi-view graph recovery (RMGR) method in which the specific structure is used to clean the multiple input noisy graphs and these cleaned graphs are simultaneously aggregated into a consensus graph by adaptively assigning great weighted coefficients for important graphs. To make the consensus graph suit classification, the clustering structure is introduced to restrain the rank of Laplacian matrix of the consensus graph such that the number of its connected components is equal to that of clustering. In doing so, the graph is adaptively adjusted during optimization to more accurately partition data. The optimization problem is solved by proposed the iterative update algorithm. Extensive experiments on synthetic and several benchmark data sets show the effectiveness of the proposed method.

In this paper, a novel least square regression method, named group low-rank representation-based discriminant linear regression (GLRRDLR), is proposed for multi-class classification. Unlike the conventional linear regression methods, the proposed method aims to learn a more discriminative projection. Specially, two main techniques are adopted to improve the discriminability of the projection. The first approach is to make the transformed samples locate in their own subspace by introducing a group low-rank constraint to the model, such that the distance between samples from the same class can be decreased greatly. The second approach is to simultaneously learn a discriminative target matrix for regression. The extensive experimental results show that the proposed method performs much better than the state-of-the-art methods, which proves the effectiveness of the above two approaches in improving the discriminability of the projection.

In this paper, we propose a general framework for incomplete multiview clustering. The proposed method is the first work that exploits the graph learning and spectral clustering techniques to learn the common representation for incomplete multiview clustering. First, owing to the good performance of low-rank representation in discovering the intrinsic subspace structure of data, we adopt it to adaptively construct the graph of each view. Second, a spectral constraint is used to achieve the low-dimensional representation of each view based on the spectral clustering. Third, we further introduce a co-regularization term to learn the common representation of samples for all views, and then use the k -means to partition the data into their respective groups. An efficient iterative algorithm is provided to optimize the model. Experimental results conducted on seven incomplete multiview datasets show that the proposed method achieves the best performance in comparison with some state-of-the-art methods, which proves the effectiveness of the proposed method in incomplete multiview clustering.

Linear discriminant analysis (LDA) has been widely used as the technique of feature exaction. However, LDA may be invalid to address the data from different domains. The reasons are as follows: 1) the distribution discrepancy of data may disturb the linear transformation matrix so that it cannot extract the most discriminative feature and 2) the original design of LDA does not consider the unlabeled data so that the unlabeled data cannot take part in the training process for further improving the performance of LDA. To address these problems, in this brief, we propose a novel transferable LDA (TLDA) method to extend LDA into the scenario in which the data have different probability distributions. The whole learning process of TLDA is driven by the philosophy that the data from the same subspace have a low-rank structure. The matrix rank in TLDA is the key learning criterion to conduct local and global linear transformations for restoring the low-rank structure of data from different distributions and enlarging the distances among different subspaces. In doing so, the variations of distribution discrepancy within the same subspace can be reduced, i.e., data can be aligned well and the maximally separated structure can be achieved for the data from different subspaces. A simple projected subgradient-based method is proposed to optimize the objective of TLDA, and a strict theory proof is provided to guarantee a quick convergence. The experimental evaluation on public data sets demonstrates that our TLDA can achieve better classification performance and outperform the state-of-the-art methods.

Multi-view clustering aims to partition data collected from diverse sources based on the assumption that all views are complete. However, such prior assumption is hardly satisfied in many real-world applications, resulting in the incomplete multi-view learning problem. The existing attempts on this problem still have the following limitations: 1) the underlying semantic information of the missing views is commonly ignored; 2) The local structure of data is not well explored; 3) The importance of different views is not effectively evaluated. To address these issues, this paper proposes a Unified Embedding Alignment Framework (UEAF) for robust incomplete multi-view clustering. In particular, a locality-preserved reconstruction term is introduced to infer the missing views such that all views can be naturally aligned. A consensus graph is adaptively learned and embedded via the reverse graph regularization to guarantee the common local structure of multiple views and in turn can further align the incomplete views and inferred views. Moreover, an adaptive weighting strategy is designed to capture the importance of different views. Extensive experimental results show that the proposed method can significantly improve the clustering performance in comparison with some state-of-the-art methods.

Linear discriminant analysis (LDA) is a very popular supervised feature extraction method and has been extended to different variants. However, classical LDA has the following problems: 1) The obtained discriminant projection does not have good interpretability for features; 2) LDA is sensitive to noise; and 3) LDA is sensitive to the selection of number of projection directions. In this paper, a novel feature extraction method called robust sparse linear discriminant analysis (RSLDA) is proposed to solve the above problems. Specifically, RSLDA adaptively selects the most discriminative features for discriminant analysis by introducing the$l_{2,1}$norm. An orthogonal matrix and a sparse matrix are also simultaneously introduced to guarantee that the extracted features can hold the main energy of the original data and enhance the robustness to noise, and thus RSLDA has the potential to perform better than other discriminant methods. Extensive experiments on six databases demonstrate that the proposed method achieves the competitive performance compared with other state-of-the-art feature extraction methods. Moreover, the proposed method is robust to the noisy data.

Preserving global and local structures during projection learning is very important for feature extraction. Although various methods have been proposed for this goal, they commonly introduce an extra graph regularization term and the corresponding regularization parameter that needs to be tuned. However, tuning the parameter manually not only is time-consuming, but also is difficult to find the optimal value to obtain a satisfactory performance. This greatly limits their applications. Besides, projections learned by many methods do not have good interpretability and their performances are commonly sensitive to the value of the selected feature dimension. To solve the above problems, a novel method named low-rank preserving projection via graph regularized reconstruction (LRPP_GRR) is proposed. In particular, LRPP_GRR imposes the graph constraint on the reconstruction error of data instead of introducing the extra regularization term to capture the local structure of data, which can greatly reduce the complexity of the model. Meanwhile, a low-rank reconstruction term is exploited to preserve the global structure of data. To improve the interpretability of the learned projection, a sparse term with${l_{2,1}}$norm is imposed on the projection. Furthermore, we introduce an orthogonal reconstruction constraint to make the learned projection hold main energy of data, which enables LRPP_GRR to be more flexible in the selection of feature dimension. Extensive experimental results show the proposed method can obtain competitive performance with other state-of-the-art methods.

Palmprint directions have been proved to be one of the most effective features for palmprint recognition. However, most existing direction-based palmprint descriptors are hand-craft designed and require strong prior knowledge. In this paper, we propose a discriminant direction binary code (DDBC) learning method for palmprint recognition. Specifically, for each palmprint image, we first calculate the convolutions of the direction-based templates and palmprint and form the informative convolution difference vectors by computing the convolution difference between the neighboring directions. Then, we propose a simple yet effective model to learn feature mapping functions that can project these convolution difference vectors into DDBCs. For all training samples: (1) the variance of the learned binary codes is maximized; (2) the intra-class distance of the binary codes is minimized; and (3) the inter-class distance of the binary codes is maximized. Finally, we cluster the block-wise histograms of DDBC forming the discriminant direction binary palmprint descriptor for palmprint recognition. The experimental results on four challenging contactless palmprint databases clearly demonstrate the effectiveness of the proposed method.

Classical linear regression not only lacks of the flexibility in fitting the label, but also ignores to preserve the intrinsic local geometric structure of data, which leads to overfitting. In this paper, we propose a novel discriminative regression method, called adaptive locality preserving based discriminative regression (ALPDR), to address these problems. Firstly, a locality preserving constraint regularized by the adaptive weight is introduced to preserve the intrinsic geometric structures of data, in which the similar points of the same class are adaptively pulled together by the projection. Secondly, ALPDR directly learns the discriminative target matrix from data based on the given label information, which allows more freedom in label fitting and simultaneously enlarges the margins between different classes. Thirdly, ALPDR imposes a row-sparsity constraint on the projection, which enables the method to adaptively select the most discriminative features from data such that the negative influence of noises and redundant features can be eliminated. Finally, an efficient iterative algorithm is provided to optimize the model. Extensive experiments show that the proposed method outperforms the other state-of-art methods, which proves the effectiveness of the proposed method.

Various palmprint recognition methods have been proposed based on orientation features of palmprints. Among them, the competitive code method using the dominant orientation of palmprint images achieves promising performance in palmprint recognition. In this paper, we propose a discriminative and robust competitive code based method, which uses a more accurate dominant orientation representation of palmprint images for palmprint authentication. Moreover, we propose to weight the orientation information of a neighbor area to improve the precision and stability of the discriminative and robust dominant orientation code. Experiments performed on three types of palmprint databases and a noisy dataset validate the effectiveness of the proposed method.

Lines are the most essential and discriminative features of palmprint images, which motivate researches to propose various line direction based methods for palmprint recognition. Conventional methods usually capture the only one of the most dominant direction of palmprint images. However, a number of points in palmprint images have double or even more than two dominant directions because of a plenty of crossing lines of palmprint images. In this paper, we propose a local multiple directional pattern (LMDP) to effectively characterize the multiple direction features of palmprint images. LMDP can not only exactly denote the number and positions of dominant directions but also effectively reflect the confidence of each dominant direction. Then, a simple and effective coding scheme is designed to represent the LMDP and a block-wise LMDP descriptor is used as the feature space of palmprint images in palmprint recognition. Extensive experimental results demonstrate the superiority of the LMDP over the conventional powerful descriptors and the state-of-the-art direction based methods in palmprint recognition.