Aiming to estimate the full extent of partially occluded objects, amodal segmentation is a critical capability for visual intelligence. Existing methods suffer from limitations in efficiency and precision, due to their reliance on auxiliary information or two-stage architectures. Furthermore, they lack generalizability, failing to meet practical requirements. To overcome these challenges, we proposed a new paradigm, CondDiff-AMO, that interprets amodal segmentation as a denoising problem by leveraging diffusion models. Methodologically, the designed novel framework consists of three key innovations to adapt the task characteristics and unlocks the diffusion models’ potential in amodal segmentation, including a masking strategy in the forward process, an adaptive transformer for conditional feature extraction, and visual-guided sampling. In the forward process, progressive masking strategy converts ground-truth masks to visible masks, simulating amodal segmentation process to enhance reasoning regarding occluded areas. For architectural design, a pyramid network with feature refinement extracts adaptive and representative conditional priors, improving the guidance in the denoising process of diffusion models. As for the sampling stage, a visible mask is incorporated with an ensemble strategy, restricting the prediction on occluded part. Experiments were conducted on five well-known datasets under supervised and zero-shot learning, with the results confirming that CondDiff-AMO outperforms state-of-the-art methods.

Multi-view data encompasses various data types, including multi-feature, multi-sequence, and multi-modal data. Multi-view multi-label classification aims to leverage the rich semantic information contained in multiple views to achieve enhanced multi-label classification performance. In practical applications, the absence of views and labels poses a significant challenge to multi-view multi-label classification tasks. Premised on the assumption that shared semantic information across multiple views is sufficient to support the downstream task, we propose CTRL, a novel incomplete multi-view multi-label classification framework to address the multi-view learning challenge on the data with partially missing views and missing labels in this paper. The core mechanism of CTRL lies in learning a high-purity, low-redundancy condensed representation that adequately captures the essential information of the original data. Specifically, we design a new objective loss to enhance the semantic information of shared cross-view within the joint representation learning process while simultaneously suppressing intra-view redundant information that is irrelevant to the downstream task. This enables CTRL to extract task-relevant representations even when views are incomplete. Furthermore, we employ the Beta Evidential Neural Network to model the label distribution. This network is then integrated with Dempster-Shafer theory, enabling our model to perform label-level classification uncertainty estimation. This also allows us to use the estimated uncertainty and belief mass to create high-reliability pseudo-labels, resulting in further gains in model performance. Experimental results on multiple benchmark datasets demonstrate the superior performance of our proposed model in terms of accuracy, robustness, and reliability.

Palmprint recognition systems experience a significant performance decline in cross-domain scenarios due to domain shift caused by non-identity factors such as capture devices and lighting conditions. To address this issue, this paper introduces a novel deep decoupling framework, the Identity and Style Feature Decoupling Network (ISFDNet), designed to improve the model’s cross-domain generalization. ISFDNet explicitly separates stable identity-related information from variable domain-related style information within palmprint features. The framework incorporates two innovative mechanisms: at the feature level, the Spatially-Aware Separation Module (SASM) adaptively produces complementary spatial attention masks to decouple mixed features into identity and style components; at the image level, the Low-Frequency Disturbance Module (LFDM) creates stylized training samples by perturbing the low-frequency parts of images, encouraging the network to learn identity representations that are insensitive to style variations. Additionally, a carefully designed collaborative supervision strategy combines multiple losses to ensure effective decoupling. Extensive experiments on four publicly available palmprint datasets demonstrate that ISFDNet achieves top performance in both cross-domain and in-domain tests, while significantly enhancing the generalization capabilities of existing networks. The code is released at https://github.com/20201422/ISFDNet.

Current deep learning (DL)-based palmprint verification models rely on centralized training with large datasets, which raises significant privacy concerns due to the sensitive and immutable nature of biometric data. Federated learning (FL), a privacy-preserving distributed learning paradigm, offers a compelling alternative by enabling collaborative model training without the need for data sharing. However, FL-based palmprint verification faces critical challenges, including data heterogeneity from diverse identities and the absence of standardized evaluation benchmarks. This paper addresses these gaps by establishing a comprehensive benchmark for FL-based palmprint verification, which explicitly defines and evaluates two practical scenarios: closed-set and open-set verification. We propose FedPalm, a unified FL framework that balances local adaptability with global generalization. Each client trains a personalized textural expert tailored to local data and collaboratively contributes to a shared global textural expert for extracting generalized features. To further enhance verification performance, we introduce a Textural Expert Interaction Module that dynamically routes textural features among experts to generate refined side textural features. Learnable parameters are employed to model relationships between original and side features, fostering cross-texture-expert interaction and improving feature discrimination. Extensive experiments validate the effectiveness of FedPalm, demonstrating robust performance across both scenarios and providing a promising foundation for advancing FL-based palm-print verification research. The related code has been publicly available at https://github.com/Zi-YuanYang/FedPalm.

Multimodal perception and fusion play a vital role in uncrewed aerial vehicle (UAV) object detection. Existing methods typically adopt global fusion strategies across modalities. However, due to illumination variation, the effectiveness of RGB and infrared modalities may differ across local regions within the same image, particularly in UAV perspectives where occlusions and dense small objects are prevalent, leading to suboptimal performance of global fusion methods. To address this issue, we propose an adaptive fine-grained fusion network for multimodal UAV object detection. First, we design a local feature consistency-based modality fusion module, which adaptively assigns local fusion weights according to the structural consistency of high-response regions across modalities, thereby enabling more effective aggregation of object-relevant features. Second, we introduce a mutual information-guided feature contrastive loss to encourage the preservation of modality-specific information during the early training phase. Experimental results demonstrate that the proposed method effectively addresses the issue of object occlusion in UAV perspectives, achieving state-of-the-art performance on multimodal UAV object detection benchmarks. Code will be available at https://github.com/lingf5877/AFFNet.

Deep convolutional neural networks can use hierarchical information to progressively extract structural information to recover high-quality images. However, preserving the effectiveness of the obtained structural information is important in image super-resolution. In this paper, we propose a cosine network for image super-resolution (CSRNet) by improving a network architecture and optimizing the training strategy. To extract complementary homologous structural information, odd and even heterogeneous blocks are designed to enlarge the architectural differences and improve the performance of image super-resolution. Combining linear and non-linear structural information can overcome the drawback of homologous information and enhance the robustness of the obtained structural information in image super-resolution. Taking into account the local minimum of gradient descent, a cosine annealing mechanism is used to optimize the training procedure by performing warm restarts and adjusting the learning rate. Experimental results illustrate that the proposed CSRNet is competitive with state-of-the-art methods in image super-resolution.

Unrestrained palmprint recognition refers to a comprehensive identity authentication technology, that performs personal authentication based on the palmprint images captured in uncontrolled environments, i.e., smartphone cameras, surveillance footage, or near-infrared scenarios. However, unrestrained palmprint recognition faces significant challenges due to the variability in image quality, lighting conditions, and hand poses present in such settings. We observed that many existing methods utilize the subspace structure as a prior, where the block diagonal property of the data has been proved. In this paper, we consider a unified learning model to guarantee the consensus block diagonal property for all views, named high-confident block diagonal analysis for multi-view palmprint recognition (HCBDA_MPR). Particularly, this paper proposed a multi-view block diagonal regularizer to guide that all views learn a consensus block diagonal structure. In such a manner, the main discriminant features from each view can be preserved while the learning of the strict block diagonal structure across all views. Experimental results on a number of real-world unrestrained palmprint databases proved the superiority of the proposed method, where the highest recognition accuracies were obtained in comparison with the other state-of-the-art related methods.

Owing to its remarkable convenience, weak invasiveness, and strong private security, palmprint recognition has become one of the most promising biometric methods and has attracted increasing attention in both academia and industry. Although considerable recognition performance has been achieved by existing palmprint learning methods, they generally require the use of substantial labeled datasets and involve substantial computational overhead for feature learning. In this article, we propose a novel multilayer projection learning (MLPL) method to achieve efficient palmprint feature learning and recognition. First, we transform the palmprint images into their direction-specific representations by computing the difference in the multiple directional responses. Then, we learn three layers of feature projections for robust feature learning, including low-rank projection for image noise decoupling, feature projection for discriminative feature exploration, and quantization projection for information preservation during feature encoding. With multilayer feature projections, palmprint images can be transformed into discriminative feature representations through a single-step process for efficient palmprint recognition. Moreover, we extend the proposed MLPL, referred to as E-MLPL, by minimizing the representation discrepancy between heterogeneous palmprint images to make it applicable for heterogeneous palmprint recognition. The results obtained from five widely adopted databases confirm the superior performance of the proposed method in terms of both accuracy and efficiency.

Palmprint recognition has emerged as a prominent biometric technology, widely applied in diverse scenarios. Traditional handcrafted methods for palmprint recognition often fall short in representation capability, as they heavily depend on researchers’ prior knowledge. Deep learning (DL) has been introduced to address this limitation, leveraging its remarkable successes across various domains. While existing surveys focus narrowly on specific tasks within palmprint recognition—often grounded in traditional methodologies—there remains a significant gap in comprehensive research exploring DL-based approaches across all facets of palmprint recognition. This article bridges that gap by thoroughly reviewing recent advancements in DL-powered palmprint recognition. This article systematically examines progress across key tasks, including region-of-interest (ROI) segmentation, feature extraction, and security and privacy-oriented challenges. Beyond highlighting these advancements, this article identifies current challenges and uncovers promising opportunities for future research. By consolidating state-of-the-art progress, this review serves as a valuable resource for researchers, enabling them to stay abreast of cutting-edge technologies and drive innovation in palmprint recognition.

Solutions to time-varying problems are crucial for research areas such as predicting changes in human body shape over time. While recurrent neural networks have made significant advancements in this field, their reliance on centralized processing has led to challenges such as model silos and data isolation. In response, distributed AI systems like federated learning have emerged to facilitate dynamic collaboration among models; however, they still depend on central coordinators, which pose risks to system security and efficiency. Moreover, traditional federated learning primarily supports homogeneous models and lacks effective strategies for the interaction of heterogeneous models. To address these limitations, we propose a novel method called Dynamic Collaboration of Heterogeneous Models (DCHM), based on Isomerism Learning, which leverages a consortium blockchain network to enhance model credibility and facilitate coordination among heterogeneous models. Additionally, we introduce a Distributed Hierarchical Aggregation (DHA) algorithm that enables permissioned nodes within each group to aggregate local model results and share them for standardized processing. After several iterative cycles, these nodes perform secondary integration of local results to produce global outcomes. Experimental results demonstrate that DCHM effectively analyzes the temporal variability of body shape changes with high efficiency.

The need for robust machine learning models is particularly evident in the realm of biological pattern recognition. Traditional centralized methods often struggle, as they frequently depend on large datasets that are challenging to gather due to stringent data privacy regulations. To address these limitations while maintaining classification accuracy, we propose an innovative approach that unifies models with diverse architectures within a federated learning framework built on a blockchain network. This decentralized and trustworthy system fosters effective collaboration among various models. Furthermore, we have implemented a weight distribution mechanism designed to maximize the individual strengths of each model. By leveraging blockchains inherent transparency and auditability, this approach also ensures secure and traceable data exchanges among participants. Additionally, the adaptability of the framework allows it to be extended to other domains where privacy-preserving data sharing is critical. Our experimental results showcase that the proposed methodology significantly enhances performance in classification tasks compared to existing alternatives.

Due to its rich characteristic and high accuracy, palmprint patterns have played an important role in identity recognition. However, current palmprint recognition techniques often concentrate on extracting salient features, neglecting the interactions between local and global textural attributes in palmprint images. To address this issue, we propose a novel multi-scale network capable of integrating local and global features for palmprint recognition, dubbed Multi-Scale Parallel Hybrid Network (MSPHNet). Specifically, we designed a Parallel Hybrid Feature Extraction Block (PHEB), which includes a parallel Convolutional Neural Network (CNN) -based branch for local feature extraction and a Transformer-based branch for capturing global features. Furthermore, acknowledging the uneven distribution of critical texture and its details in palmprint images, we address the current research shortfall in pixel relationship analysis by introducing the Comprehensive Attention Block (CAB). This block integrates Spatial Attention (SA) and Pixel Attention (PA) to effectively leverage the imbalanced pixel distributions within the CNN branch. Extensive experimental results on numerous public palmprint datasets demonstrated that our proposed method achieves remarkable results.

Facial age estimation has been widely adopted in the last decade, yet it presents critical privacy issues, such as susceptibility to unauthorized exposure. In contrast, tongue imagery has emerged as a promising alternative to traditional facial age estimation methods, offering improved privacy through reduced identifiability compared to facial imagery. This paper introduces a novel framework for age estimation using tongue imagery, addressing privacy concerns associated with facial recognition technologies. Our method takes advantage of a three-module approach: segmentation, quality control, and prediction, to estimate the age from tongue images. The segmentation module employs a zero-shot inference technique using the Large Language Instructed Segmentation Assistant (LISA) to accurately segment the tongue from its oral cavity images. Subsequent quality assessment of these segmentations is performed in the quality control module, ensuring only high-quality images proceed to the prediction stage. The age prediction module then employs an ensemble of EfficientNet B2 models, fine-tuned and optimized for age prediction from tongue imagery. Our experimental results derived from a newly collected dataset consisting of 9,767 tongue and facial images spanning diverse age groups demonstrate that the proposed method achieves an accuracy comparable to traditional facial age estimation techniques. This study not only provides an alternative approach to facial biometrics for age estimation but also enhances user privacy, making it particularly suitable for privacy-sensitive applications in healthcare and biometric monitoring.

Palmprint recognition has become one of the most popular approaches in the application of biometric recognition. However, palmprint recognition security has not been thoroughly studied. In this paper, we reveal a potential attack method against palmprint recognition convolutional neural networks (CNNs) by planting a human-imperceptible trigger to poison the training data. We extract the fine lines in the palmprints and use these features as the backdoor trigger. The backdoor trigger is fused with clean palmprint images by a constrained generative adversarial network (GAN). Afterwards, the poisoned dataset can be widely spread on the internet by the adversary. Here, the CNN trained on this poisoned dataset will be extremely vulnerable to the implanted trigger. Experimental results show that attackers with the right trigger can achieve on average 82% and 84% attack success rate (ASR) on CASIA and IITD public palmprint datasets against various palmprint recognition CNNs, respectively.

Pixel-level anomaly detection is indispensable in industrial defect detection and medical diagnosis. Recently, Segment Anything Model (SAM) has achieved promising results in many vision tasks. However, direct application of the SAM to pixel-level anomaly detection tasks results in unsatisfactory performance, meanwhile SAM needs the manual prompt. Although some automatically prompt-based SAM has been proposed, these automated prompting approaches merely utilize partial image features as prompts and fail to incorporate crucial features such as multi-scale image features to generate more suitable prompts. In this paper, we propose a novel Omni Dimensional State Space Model-driven SAM (ODS-SAM) for pixel-level anomaly detection. Specifically, the proposed method adopts the SAM architecture, ensuring easy implementation and avoiding the need for fine-tuning. A State-Space Model-based residual Omni Dimensional module is designed to automatically generate suitable prompts. This module can effectively leverage multi-scale and global information, facilitating an iterative search for optimal prompts in the prompt space. The identified optimal prompts are then fed into SAM as high-dimensional tensors. Experimental results demonstrate that the proposed ODS-SAM outperforms state-of-the-art models on both industrial and medical image datasets.

Zero-Shot Industrial Anomaly Detection (ZSIAD) aims to identify and localize anomalies in industrial images from unseen categories. Owing to the powerful generalization capabilities, Vision-Language Models (VLMs) have achieved growing interest in ZSIAD. To guide the model toward understanding and localizing the semantically complex industrial anomalies, existing VLM-based methods have attempted to provide additional prompts to the model through learnable text prompt templates. However, these zero-shot methods lack detailed descriptions of specific anomalies, making it difficult to classify and segment the diverse range of industrial anomalies accurately. To address the aforementioned issue, we firstly propose the multi-stage prompt generation agent for ZSIAD. Specifically, we leverage the Multi-modal Language Large Model (MLLM) to articulate the detailed differential information between normal and test samples, which can provide detailed text prompts to the model through further refinement and anti-false alarm constraint. Moreover, we introduce the Visual Fundamental Model (VFM) to generate anomaly-related attention prompts for more accurate localization of anomalies with varying sizes and shapes. Extensive experiments on seven real-world industrial anomaly detection datasets have shown that the proposed method not only outperforms recent SOTA methods, but also its explainable prompts provide the model with a more intuitive basis for anomaly identification.

The dynamic variations of food quality across spatial and temporal scales pose significant challenges for global food safety and nutrition research, requiring comprehensive analysis of diverse, multi-modal, and distributed data while preserving privacy. Existing centralized approaches suffer from data silos and limited collaboration, and although federated learning and blockchain technologies have shown promise independently, their combined potential for incentivized, privacy-preserving, and heterogeneous model collaboration remains underexplored. In this paper, we propose the concept of a Global Spatial-Temporal Food Memory-a novel research paradigm that envisions secure, decentralized, and incentivized collaboration among multiple stakeholders worldwide, leveraging blockchain-enabled token-based rewards integrated with federated learning of heterogeneous models. We discuss the scientific challenges and opportunities inherent in this vision, including multi-modal data fusion, trustworthy incentive mechanisms, and scalable long-term temporal analysis. This work aims to open new avenues in multimedia research by bridging decentralized AI, blockchain, and spatiotemporal food quality monitoring, providing a foundation for future explorations in privacy-preserving, collaborative, and large-scale multimedia data analysis.

Palmprint recognition is a pivotal biometric modality, renowned for its numerous advantages and applications in the field of biometrics. The Gabor filter is a classic and efficient texture feature extractor abstracted from the nervous system. The existing palmprint texture coding methods only focus on first-order texture features (1TFs), while neglecting discriminative second-order texture features (2TFs). Therefore, this paper proposes multi-order extensions for state-of-the-art (SOTA) palmprint texture coding methods, which makes full usage of 1TFs and 2TFs. A filter is used to extract 1TFs from the palmprint image, and the same filter is applied to extract 2TFs from 1TFs. Here, different methods employ various filters to extract diverse textures. Due to the simultaneous participations of 1TFs and 2TFs in multi-order extension codes, more discriminative features are extracted and fused. The experimental results on three public databases, including contact, noncontact and multispectral acquisition types, show that the accuracies of all the palmprint texture coding methods are remarkably improved by multi-order extension, establishing it as a general framework extendable to other texture-based recognition tasks.

As a cross-topic of multi-view learning and multi-label classification, multi-view multi-label classification has gradually gained traction in recent years. The application of multi-view contrastive learning has further facilitated this process; however, the existing multi-view contrastive learning methods crudely separate the so-called negative pair, which largely results in the separation of samples belonging to the same category or similar ones. Besides, plenty of multi-view multi-label learning methods ignore the possible absence of views and labels. To address these issues, in this paper, we propose an incomplete multi-view missing multi-label classification network named RANK. In this network, a label-driven multi-view contrastive learning strategy is proposed to leverage supervised information to preserve the intra-view structure and perform the cross-view consistency alignment. Furthermore, we break through the view-level weights inherent in existing methods and propose a quality-aware subnetwork to dynamically assign quality scores to each view of each sample. The label correlation information is fully utilized in the final multi-label cross-entropy classification loss, effectively improving the discriminative power. Last but not least, our model is not only able to handle complete multi-view multi-label data, but also works on datasets with missing instances and labels. Extensive experiments confirm that our RANK outperforms existing state-of-the-art methods.

Palmprint recognition has received considerable attention due to its inherent discriminative characteristics. However, conventional methods largely rely on static features extracted from individual images, which limits their representational richness. To address this, we propose a dynamic palmprint verification framework that harnesses generative models to enhance feature representations through dynamic construction and matching strategies. During training, a classifier-guided generative model synthesizes class-aware pairs, and a regularization term is introduced to expand the feature space, while mitigating overfitting. For matching, we reformulate the process as a subspace projection within a locally adaptive feature space, where the original and class-conditioned generated features form the basis of the subspace. This enables the model to capture latent inter-individual relationships and achieve stronger discriminability. Extensive experiments across multiple backbones and public benchmarks validate the effectiveness and robustness of the proposed framework.

Hand-based multimodal biometrics has garnered significant attention in information security and identity authentication. However, prevalent multimodal recognition techniques often extract the discriminant features from different modalities separately, ignoring the structural consistency between various modalities of the same class. Moreover, these methods generally focus on specific-scenarios, where recognition performance will be compromised when faced with different databases or multiple application scenarios. To solve these limitations, we present an innovative Dual-Cohesion Metric Learning (DCML) framework embedded in noise decomposition for few-shot hand multimodal biometrics. This approach comprehensively exploits multimodal features from both intra-modal and inter-modal structural consistency to improve its robustness across multiple applications. Specifically, DCML imposes a dual-cohesion mechanism to pull in the cross-modal distance of the same label and the within-class distance for each modal, while concurrently pushing away the between-class distance in the projected space. Furthermore, in the procedure of feature learning, the proposed DCML incorporates the low-rank constraint to mitigate the interference of noise in the raw data and enforces a sparsity constraint to extract more salient and compact features. Notably, our DCML can be flexibly extended to other multimodal biometrics. Extensive experimental results on six multimodal datasets demonstrated that our DCML outperforms the latest approaches in multiple multimodal recognition scenarios and has strong generalization ability even when the training samples are small.

Currently global features are usually extracted directly from local patterns in palmprint verification. Furthermore, sequence features for palmprint verification are only used as local features, but the properties of sequence features are not fully utilized. To solve this issue, this paper introduces Sequence Feature Fusion Network (SF2Net) for palmprint verification. SF2Net proposes a new paradigm: using stable and spatially correlated sequence features as an intermediate bridge to generate robust global representations. SF2Net’s core mechanism is to first extract fine-grained local features that are then converted into sequence features by a sequence feature extractor (SFE). Finally, the sequence features are used as a superior input to capture high-quality global features. By fusing multi-order texture-based local features with globally extracted sequence features, SF2Net achieves superior discrimination. To ensure high accuracy even with limited training data, a hybrid loss function is proposed, which integrate a cross-entropy loss and a triplet loss. Triplet loss effectively optimizes feature separation by explicitly considering negative samples. Extensive experiments on multiple publicly available palmprint datasets demonstrate that SF2Net achieves state-of-the-art (SOTA) performance. Remarkably, even with a small training-to-testing ratio (1:9), SF2Net achieves 100% accuracy, surpassing SOTA methods under several benchmark datasets. The code is released at https://github.com/20201422/SF2Net.

Three significant challenges have been limiting the stable palmprint recognition via mobile devices: 1) rotations and unconsensus scales of the unconstrait hand; 2) noises generated in the open imaging environments; and 3) low quality images captured in the low-illumination conditions. Current palmprint representation methods rely on rich prior knowledge and lack any adaptability to its environment. In this paper, we propose a multi-view hierarchical graph learning based palmprint recognition (MVHG_PR) method, which comprehensively presents the discriminant palmprint features from multiple views. Fully exploiting different types of characteristics, it aims to adaptively perform multi-view feature description and feature selection. To this end, a novel regularized heterogeneous graph learning strategy is proposed for construction of the intra- and inter-class relationships, learning high-order structures for different views between four tuples, rather than just pair-wise intrinsic structures. In the proposed model, the learned hierarchical graph is given an elastic power from the label information to precisely reflect the intra-class and the inter-class relationships in each view, such that the projected structures can be aligned locally and globally. Besides this, we constructed a mobile palmprint dataset to simulate as many open application circumstance as possible to verify the effectiveness of contactless palmprint recognition methods. Experimental results have proven the superiority of the proposed MVHG_PR by achieving the best recognition performances on a number of real-world palmprint databases. The proposed mobile palmprint database and the code of the proposed MVHG_PR are available athttps://github.com/ShupingZhao/MVHG_PR-for-contactless-palmprint-recognition.

Palmprint recognition has recently garnered attention due to its high accuracy, strong robustness, and high security. Existing deep learning-based palmprint recognition methods usually require large amounts of data for centralized training, facing the challenge of privacy disclosure. In addition, the non-independent and identically distributed (non-IID) issue in the multi-spectral palmprint images generally leads to the degradation of recognition performance. To tackle these problems, this paper proposes a dynamic personalized federated learning model for cross-spectral palmprint recognition, called DPFed-Palm. Specifically, for each client's local training, we present a new combination of loss functions to enforce the constraints of local models and effectively enhance the feature representation capability of models. Subsequently, DPFed-Palm aggregates the above-trained local models by using the combined aggregation strategies of the Federated Averaging (FedAvg) and Personalized Federated Learning (PFL) to obtain the best personalized global model of each client. For the selection of the best personalized global model, we develop a dynamic weight selection strategy to obtain the optimal weights of the local and global models by cross-spectral (cross-client) testing. Extensive experimental results on three public PolyU multispectral, IITD, and CASIA datasets show that the proposed method outperforms the existing techniques in privacy-preserving and recognition performance.

Anomaly detection can significantly aid doctors in interpreting chest X-rays. The commonly used strategy involves utilizing the pre-trained network to extract features from normal data to establish feature representations. However, when a pre-trained network is applied to more detailed X-rays, differences of similarity can limit the robustness of these feature representations. Therefore, we propose an intra- and inter-correlation learning framework for chest X-ray anomaly detection. Firstly, to better leverage the similar anatomical structure information in chest X-rays, we introduce the Anatomical-Feature Pyramid Fusion Module for feature fusion. This module aims to obtain fusion features with both local details and global contextual information. These fusion features are initialized by a trainable feature mapper and stored in a feature bank to serve as centers for learning. Furthermore, to Facing Differences of Similarity (FDS) introduced by the pre-trained network, we propose an intra- and inter-correlation learning strategy: 1) We use intra-correlation learning to establish intra-correlation between mapped features of individual images and semantic centers, thereby initially discovering lesions; 2) We employ inter-correlation learning to establish inter-correlation between mapped features of different images, further mitigating the differences of similarity introduced by the pre-trained network, and achieving effective detection results even in diverse chest disease environments. Finally, a comparison with 18 state-of-the-art methods on three datasets demonstrates the superiority and effectiveness of the proposed method across various scenarios.

Efforts in weakly-supervised video anomaly detection center on detecting abnormal events within videos by coarse-grained labels, which has been successfully applied to many real-world applications. However, a significant limitation of most existing methods is that they are only effective for specific objects in specific scenarios, which makes them prone to misclassification or omission when confronted with previously unseen anomalies. Relative to conventional anomaly detection tasks, Open-world Weakly-supervised Video Anomaly Detection (OWVAD) poses greater challenges due to the absence of labels and fine-grained annotations for unknown anomalies. To address the above problem, we propose a multi-scale evidential vision-language model to achieve open-world video anomaly detection. Specifically, we leverage generalized visual-language associations derived from CLIP to harness the full potential of large pre-trained models in addressing the OWVAD task. Subsequently, we integrate a multi-scale temporal modeling module with a multimodal evidence collector to achieve precise frame-level detection of both seen and unseen anomalies. Extensive experiments on two widely-utilized benchmarks have conclusively validated the effectiveness of our method. The code will be made publicly available.

Palmprint-based biometric recognition has gained widespread attention due to its rich features, contactless acquisition, and low invasiveness. However, most existing methods neglect image quality, making them less effective for low-quality, noisy palmprint images. In this paper, we propose a palm intrinsic features learning selective state space model (PalmMamba) for palmprint image denoising, which consists of shallow feature representation, noise-insensitive palmprint-specific feature learning, and sharp palmprint image restoration modules. First, we convert the degraded noisy palmprint image into a high-dimensional shallow feature representation through a single-layer convolution backbone. Then, we develop parallel learning branches, including a second-order attention-based selective state space model and a mixed difference convolution module, to exploit diverse palmprint-specific features with both global and local details. Finally, we map the fine-grained palmprint-intrinsic feature map into the identity-preserved sharp palmprint image via a commonly used convolution layer. Extensive experimental results on five public palmprint databases demonstrate the encouraging performance of the proposed PalmMamba in palmprint image denoising.

The dissemination of public opinion in the social media network is driven by public sentiment, which can be used to promote the effective resolution of social incidents. However, public sentiments for incidents are often affected by environmental factors such as geography, politics, and ideology, which increases the complexity of the sentiment acquisition task. Therefore, a hierarchical mechanism is designed to reduce complexity and utilize processing at multiple phases to improve practicality. Through serial processing between different phases, the task of public sentiment acquisition can be decomposed into two subtasks, which are the classification of report text to locate incidents and sentiment analysis of individuals' reviews. Performance has been improved through improvements to the model structure, such as embedding tables and gating mechanisms. That being said, the traditional centralized structure model is not only easy to form model silos in the process of performing tasks but also faces security risks. In this article, a novel distributed deep learning model called isomerism learning based on blockchain is proposed to address these challenges, the trusted collaboration between models can be realized through parallel training. In addition, for the problem of text heterogeneity, we also designed a method to measure the objectivity of events to dynamically assign the weights of models to improve aggregation efficiency. Extensive experiments demonstrate that the proposed method can effectively improve performance and outperform the state-of-the-art methods significantly.

Exploring the structure information is crucial for data clustering task, particularly for the sceneries of incomplete multiview clustering (IMVC) when some views are missing. However, almost all of the existing graph-based IMVC methods either introduce the Laplacian constraint with fixed graphs or simply fuse the graphs of all views, which are vulnerable to the quality of the constructed graphs. To address this issue, we propose a new graph-based method, called confident local structure-aware incomplete multiview spectral clustering. Different from existing works, our method seeks to adaptively uncover the inherent similarity structure among the available instances in each view and learn the optimal consensus graph within a unified learning framework. Moreover, to mitigate the adverse effects of imbalance information across incomplete views and improve the quality of consensus graph, we further impose some adaptive weights on the consensus graph learning model w.r.t. each view and introduce some confident structure graphs to explore the most confident similarity information in the model. In contrast to existing works, our approach simultaneously takes into account the pairwise similarity information and neighbor group-based confident structure information. This dual consideration makes our method more effective in achieving the optimal consensus graph and delivering superior IMVC performance. Experimental results on several datasets demonstrate that our method effectively learns a high-quality and clustering-friendly graph from incomplete multiview data, and it outperforms many state-of-the-art IMVC methods in terms of clustering performance.

Vision-language models (VLMs) have emerged as formidable tools, showing their strong capability in handling various open-vocabulary tasks in image recognition, text-driven visual content generation, and visual chatbots, to name a few. In recent years, considerable efforts and resources have been devoted to adaptation methods for improving downstream performance of VLMs, particularly on parameter-efficient fine-tuning methods like prompt learning. However, a crucial aspect that has been largely overlooked is the confidence calibration problem in fine-tuned VLMs, which could greatly reduce reliability when deploying such models in the real world. This paper bridges the gap by systematically investigating the confidence calibration problem in the context of prompt learning and reveals that existing calibration methods are insufficient to address the problem, especially in the open-vocabulary setting. To solve the problem, we present a simple and effective approach called Distance-Aware Calibration (DAC), which is based on scaling the temperature using as guidance the distance between predicted text labels and base classes. The experiments with 7 distinct prompt learning methods applied across 11 diverse downstream datasets demonstrate the effectiveness of DAC, which achieves high efficacy without sacrificing the inference speed.

In this work, we focus on studying the differentiable relaxations of several linear regression problems, where the original formulations are usually both nonsmooth with one nonconvex term. Unfortunately, in most cases, the standard alternating direction method of multipliers (ADMM) cannot guarantee global convergence when addressing these kinds of problems. To address this issue, by smoothing the convex term and applying a linearization technique before designing the iteration procedures, we employ nonconvex ADMM to optimize challenging nonconvex-convex composite problems. In our theoretical analysis, we prove the boundedness of the generated variable sequence and then guarantee that it converges to a stationary point. Meanwhile, a potential function is derived from the augmented Lagrange function, and we further verify that the objective function is monotonically nonincreasing. Under the Kurdyka-Łojasiewicz (KŁ) property, the global convergence is analyzed step by step. Finally, experiments on face reconstruction, image classification, and subspace clustering tasks are conducted to show the superiority of our algorithms over several state-of-the-art ones.

Low-rank matrix recovery is a major challenge in machine learning and computer vision, particularly for large-scale data matrices, as popular methods involving nuclear norm and singular value decomposition (SVD) are associated with high computational costs and biased estimators. To overcome this challenge, we propose a novel approach to learning low-rank matrices based on the matrix volume and a nonconvex logarithmic function. The matrix volume is the product of all the nonzero singular values of a matrix and has unique geometric properties and connections with other convex and nonconvex functions. We establish a generalized nonconvex regularization problem using the penalty function strategy and introduce an accelerated proximal alternating linearized minimization (AccPALM) algorithm with double acceleration, which combines Nesterov’s acceleration and power strategy. The algorithm reduces computational costs and has provable convergence results under the Kurdyka-Łojasiewicz (KŁ) inequality with mild conditions. Our approach shows superior accuracy, efficiency, and convergence behavior compared to other low-rank matrix learning methods on robust matrix completion (RMC) and low-rank representation (LRR) tasks. We analyze the impact of algorithm parameters on convergence and performance and present visually appealing results to further demonstrate the effectiveness of our approach. The proposed methodology represents a promising advance in the field of low-rank matrix recovery, and its effectiveness has been validated via extensive numerical experiments. The source code for the proposed algorithms is accessible at https://github.com/ZhangHengMin/AccPALMcodes.

Palmprint recognition has shown great value for biometric recognition due to its advantages of good hygiene, semi-privacy and low invasiveness. However, most existing palmprint recognition studies focus only on homogeneous palmprint recognition, where comparing palmprint images are collected under similar conditions with small domain gaps. To address the problem of matching heterogeneous palmprint images captured under the visible light (VIS) and the near-infrared (NIR) spectrum with large domain gaps, in this paper, we propose a Fourier-based feature learning network (FFLNet) for VIS-NIR heterogeneous palmprint recognition. First, we extract the multi-scale shallow representations of heterogeneous palmprint images via three vanilla convolution layers. Then, we convert the shallow palmprint feature maps into frequency-specific representations via Fourier transform to separate different layers of palmprint features, and exploit the underlying common and palmprint-specific frequency information of heterogeneous palmprint images. This effectively reduces the modality gap of heterogeneous palmprint images at the feature level. After that, we convert the common frequency-specific feature maps back to the spatial domain to learn the identity-invariant discriminative features via residual convolution for heterogeneous palmprint recognition. Extensive experimental results on three challenging heterogeneous palmprint databases clearly demonstrate the effectiveness of the proposed FFLNet for VIS-NIR heterogeneous palmprint recognition.

Over the last decades, finger vein biometric recognition has generated increasing attention because of its high security, accuracy, and natural anti-counterfeiting. However, most of the existing finger vein recognition approaches rely on image enhancement or require much prior knowledge, which limits their generalization ability to different databases and different scenarios. Additionally, these methods rarely take into account the interference of noise elements in feature representation, which is detrimental to the final recognition results. To tackle these problems, we propose a novel jointly embedding model, called Joint Discriminative Analysis with Low-Rank Projection (JDA-LRP), to simultaneously extract noise component and salient information from the raw image pixels. Specifically, JDA-LRP decomposes the input image into noise and clean components via low-rank representation and transforms the clean data into a subspace to adaptively learn salient features. To further extract the most representative features, the proposed JDA-LRP enforces the discriminative class-induced constraint of the training samples as well as the sparse constraint of the embedding matrix to aggregate the embedded data of each class in their respective subspace. In this way, the discriminant ability of the jointly embedding model is greatly improved, such that JDA-LRP can be adapted to multiple scenarios. Comprehensive experiments conducted on three commonly used finger vein databases and four palm-based biometric databases illustrate the superiority of our proposed model in recognition accuracy, computational efficiency, and domain adaptation.

Due to their high reliability, security, and anti-counterfeiting, finger-based biometrics (such as finger vein and finger knuckle print) have recently received considerable attention. Despite recent advances in finger-based biometrics, most of these approaches leverage much prior information and are non-robust for different modalities or different scenarios. To address this problem, we propose a structured Robust and Sparse Least Square Regression (RSLSR) framework to adaptively learn discriminative features for personal identification. To achieve the powerful representation capacity of the input data, RSLSR synchronously integrates robust projection learning, noise decomposition, and discriminant sparse representation into a unified learning framework. Specifically, RSLSR jointly learns the most discriminative information from the original pixels of the finger images by introducing the$l_{2,1}$norm. A sparse transformation matrix and reconstruction error are simultaneously enforced to enhance its robustness to noise, thus making RSLSR adaptable to multi-scenarios. Extensive experiments on five contact-based and contactless-based finger databases demonstrate the clear superiority of the proposed RSLSR in terms of recognition accuracy and computational efficiency.

In recent years, palmprints have been extensively utilized for individual verification. The abundance of sensitive information in palmprint data necessitates robust protection to ensure security and privacy without compromising system performance. Existing systems frequently use cancelable transformations to protect palmprint templates. However, if an adversary gains access to the stored database, they could initiate a replay attack before the system detects the breach and can revoke and replace the reference template. To address replay attacks while meeting template protection criteria, we propose a dual-level cancelable palmprint verification framework. In this framework, the reference template is initially transformed using a cancelable competition hashing network with a first-level token, enabling the end-to-end generation of cancelable templates. During enrollment, the system creates a negative database (NDB) using a second-level token for further protection. Due to the unique NDB-to-vector matching characteristic, a replay attack involving the matching between the reference template and a compromised instance in NDB form is infeasible. This approach effectively addresses the replay attack problem at its root. Furthermore, the dual-level protected reference template enjoys heightened security, as reversing the NDB is NP-hard. We also propose a novel NDB-to-vector matching algorithm based on matrix operations to expedite the matching process, addressing the inefficiencies of previous NDB methods reliant on dictionary-based matching rules. Extensive experiments conducted on public palmprint datasets confirm the effectiveness and generality of the proposed framework. Upon acceptance of the paper, the code will be accessible athttps://github.com/Zi-YuanYang/DCPV.

Image denoising is a critical problem in industrial information applications since noisy images can have adverse effects on the performance of many industrial tasks. Currently, Transformer structures and graph convolutional networks (GCNs) have been widely employed in image denoising to capture long-range dependencies for the performance promotion. These methods, however, severely suffer from three major problems. Initially, the long-range dependencies captured by Transformers and GCNs are only focused on the pixel level and patch level, respectively. This leads to the coarse retrieved feature, hindering further performance promotion. In addition, due to the limited training data, especially for the noisy images with highly diverse and complex noise, the denoising process may lack sufficient feature for reconstructing denoised images. Eventually, the limited training data may also results in over-fitting, leading to poor generalization in the denoising process. This article first proposes adaptive graph prior network (AGP-Net) using a novel graph construction method to capture the long-range dependencies on both the pixel and patch levels. Then, we propose graph supplementary prior and graph noise prior in AGP-Net to adaptively generate supplementary feature and regularization noise for improving the performance and generalization of image denoising. Extensive ablation and benchmark tests show our AGP-Net achieve the most advanced image denoising performance.

Recent advancements in deep learning techniques have pushed forward the frontiers of real photograph denoising. However, due to the inherent pooling operations in the spatial domain, current CNN-based denoisers are biased towards focusing on low-frequency representations, while discarding the high-frequency components. This will induce a problem for suboptimal visual quality as the image denoising tasks target completely eliminating the complex noises and recovering all fine-scale and salient information. In this work, we tackle this challenge from the frequency perspective and present a new solution pipeline, coined as frequency attention denoising network (FADNet). Our key idea is to build a learning-based frequency attention framework, where the feature correlations on a broader frequency spectrum can be fully characterized, thus enhancing the representational power of the network across multiple frequency channels. Based on this, we design a cascade of adaptive instance residual modules (AIRMs). In each AIRM, we first transform the spatial-domain features into the frequency space. Then, a learning-based frequency attention framework is devised to explore the feature inter-dependencies converted in the frequency domain. Besides this, we introduce an adaptive layer by leveraging the guidance of the estimated noise map and intermediate features to meet the challenges of model generalization in the noise discrepancy. The effectiveness of our method is demonstrated on several real camera benchmark datasets, with superior denoising performance, generalization capability, and efficiency versus the state-of-the-art.

Unconstrained palmprint images have shown great potential for recognition applications due to their lower restrictions regarding hand poses and backgrounds during contactless image acquisition. However, they face two challenges: 1) unclear palm contours and finger-valley points of unconstrained palmprint images make it difficult to locate landmarks to crop the palmprint region of interest (ROI); and 2) large intra-class diversities of unconstrained palmprint images hinder the learning of intra-class-invariant palmprint features. In this paper, we propose to directly extract the complete palmprint region as the ROI (CROI) using the detection-style CenterNet without requiring the detection of any landmarks, and large intra-class diversities may occur. To address this, we further propose a palmprint feature alignment and learning hybrid network (PalmALNet) for unconstrained palmprint recognition. Specifically, we first exploit and align the multi-scale shallow representation of unconstrained palmprint images via deformable convolution and alignment-aware supervision, such that the pixel gaps of the intra-class palmprint CROIs can be minimized in shallow feature space. Then, we develop multiple triple-attention learning modules by integrating spatial, channel, and self-attention operations into convolution to adaptively learn and highlight the latent identity-invariant palmprint information, enhancing the overall discriminative power of the palmprint features. Extensive experimental results on four challenging palmprint databases demonstrate the promising effectiveness of both the proposed PalmALNet and CROI for unconstrained palmprint recognition.

Since hand-print recognition, i.e., palmprint, finger-knuckle-print (FKP), and hand-vein, have significant superiority in user convenience and hygiene, it has attracted greater enthusiasm from researchers. Seeking to handle the long-standing interference factors, i.e., noise, rotation, shadow, in hand-print images, multi-view hand-print representation has been proposed to enhance the feature expression by exploiting multiple characteristics from diverse views. However, the existing methods usually ignore the high-order correlations between different views or fuse very limited types of features. To tackle these issues, in this paper, we present a novel tensorized multi-view low-rank approximation based robust hand-print recognition method (TMLA_RHR), which can dexterously manipulate the multi-view hand-print features to produce a high-compact feature representation. To achieve this goal, we formulate TMLA_RHR by two key components, i.e., aligned structure regression loss and tensorized low-rank approximation, in a joint learning model. Specifically, we treat the low-rank representation matrices of different views as a tensor, which is regularized with a low-rank constraint. It models the across information between different views and reduces the redundancy of the learned sub-space representations. Experimental results on eight real-world hand-print databases prove the superiority of the proposed method in comparison with other state-of-the-art related works.

Domain adaptation person re-identification (Re-ID) is a challenging task, which aims to transfer the knowledge learned from the labeled source domain to the unlabeled target domain. Recently, some clustering-based domain adaptation Re-ID methods have achieved great success. However, these methods ignore the inferior influence on pseudo-label prediction due to the different camera styles. The reliability of the pseudo-label plays a key role in domain adaptation Re-ID, while the different camera styles bring great challenges for pseudo-label prediction. To this end, a novel method is proposed, which bridges the gap of different cameras and extracts more discriminative features from an image. Specifically, an intra-to-intermechanism is introduced, in which samples from their own cameras are first grouped and then aligned at the class level across different cameras followed by our logical relation inference (LRI). Thanks to these strategies, the logical relationship between simple classes and hard classes is justified, preventing sample loss caused by discarding the hard samples. Furthermore, we also present a multiview information interaction (MvII) module that takes features of different images from the same pedestrian as patch tokens, obtaining the global consistency of a pedestrian that contributes to the discriminative feature extraction. Unlike the existing clustering-based methods, our method employs a two-stage framework that generates reliable pseudo-labels from the views of the intracamera and intercamera, respectively, to differentiate the camera styles, subsequently increasing its robustness. Extensive experiments on several benchmark datasets show that the proposed method outperforms a wide range of state-of-the-art methods. The source code has been released at https://github.com/lhf12278/LRIMV.

In this article, we propose a new linear regression (LR)-based multiclass classification method, called discriminative regression with adaptive graph diffusion (DRAGD). Different from existing graph embedding-based LR methods, DRAGD introduces a new graph learning and embedding term, which explores the high-order structure information between four tuples, rather than conventional sample pairs to learn an intrinsic graph. Moreover, DRAGD provides a new way to simultaneously capture the local geometric structure and representation structure of data in one term. To enhance the discriminability of the transformation matrix, a retargeted learning approach is introduced. As a result of combining the above-mentioned techniques, DRAGD can flexibly explore more unsupervised information underlying the data and the label information to obtain the most discriminative transformation matrix for multiclass classification tasks. Experimental results on six well-known real-world databases and a synthetic database demonstrate that DRAGD is superior to the state-of-the-art LR methods.

To enhance the effectiveness and efficiency of subspace clustering in visual tasks, this work introduces a novel approach that automatically eliminates the optimal mean, which is embedded in the subspace clustering framework of low-rank representation (LRR) methods, along with the computationally factored formulation of Schatten p -norm. By addressing the issues related to meaningful computations involved in some LRR methods and overcoming biased estimation of the low-rank solver, we propose faster nonconvex subspace clustering methods through joint Schatten p -norm factorization with optimal mean (JS p NFOM), forming a unified framework for enhancing performance while reducing time consumption. The proposed approach employs tractable and scalable factor techniques, which effectively address the disadvantages of higher computational complexity, particularly when dealing with large-scale coefficient matrices. The resulting nonconvex minimization problems are reformulated and further iteratively optimized by multivariate weighting algorithms, eliminating the need for singular value decomposition (SVD) computations in the developed iteration procedures. Moreover, each subproblem can be guaranteed to obtain the closed-form solver, respectively. The theoretical analyses of convergence properties and computational complexity further support the applicability of the proposed methods in real-world scenarios. Finally, comprehensive experimental results demonstrate the effectiveness and efficiency of the proposed nonconvex clustering approaches compared to existing state-of-the-art methods on several publicly available databases. The demonstrated improvements highlight the practical significance of our work in subspace clustering tasks for visual data analysis. The source code for the proposed algorithms is publicly accessible at https://github.com/ZhangHengMin/TRANSUFFC.

Low-quality palmprint images will degrade the recognition performance, when they are captured under the open, unconstraint, and low-illumination conditions. Moreover, the traditional single-view palmprint representation methods have been difficult to express the characteristics of each palm strongly, where the palmprint characteristics become weak. To tackle these issues, in this article, we propose a structure suture learning-based robust multiview palmprint recognition method (SSL_RMPR), which comprehensively presents the salient palmprint features from multiple views. Unlike the existing multiview palmprint representation methods, SSL_RMPR introduces a structure suture learning strategy to produce an elastic nearest neighbor graph (ENNG) on the reconstruction errors that simultaneously exploit the label information and the latent consensus structure of the multiview data, such that the discriminant palmprint representation can be adaptively enhanced. Meanwhile, a low-rank reconstruction term integrating with the projection matrix learning is proposed, in such a manner that the robustness of the projection matrix can be improved. Particularly, since no extra structure capture term is imposed into the proposed model, the complexity of the model can be greatly reduced. Experimental results have proven the superiority of the proposed SSL_RMPR by achieving the best recognition performances on a number of real-world palmprint databases.

Graph convolutional networks (GCNs) are a popular approach to learn the feature embedding of graph-structured data, which has shown to be highly effective as well as efficient in performing node classification in an inductive way. However, with massive nongraph-organized data existing in application scenarios nowadays, it is critical to exploit the relationships behind the given groups of data, which makes better use of GCN and broadens the application field. In this article, we propose the f uzzy g raph s ubspace c onvolutional n etwork (FGSCN) to provide a brand-new paradigm for feature embedding and node classification with graph convolution (GC) when given an arbitrary collection of data. The FGSCN performs GC on the f uzzy s ubspace ($\mathcal {F}$-space), which simultaneously learns from the underlying subspace information in the low-dimensional space as well as its neighborliness information in the high-dimensional space. In particular, we construct the fuzzy homogenous graph$\mathcal {G}_{\mathcal {F}}$on the$\mathcal {F}$-space by fusing the homogenous graph of neighborliness$\mathcal {G}_{\mathcal {N}}$and homogenous graph of subspace$\mathcal {G}_{\mathcal {S}}$(defined by the affinity matrix of the low-rank representation). Here, it is proven that the GC on$\mathcal {F}$-space will propagate both the local and global information through fuzzy set theory. We evaluated FGSCN on 15 unique datasets with different tasks (e.g., feature embedding, visual recognition, etc.). The experimental results showed that the proposed FGSCN has significant superiority compared with current state-of-the-art methods.

Most of the existing incomplete multi-view clustering (IMVC) methods focus on attaining a consensus representation from different views but ignore the important information hidden in the missing views and the latent intrinsic structures in each view. To tackle these issues, in this paper, a unified and novel framework, named tensorized incomplete multi-view clustering with intrinsic graph completion (TIMVC_IGC) is proposed. Firstly, owing to the effectiveness of the low-rank representation in revealing the inherent structure of the data, we exploit it to infer the missing instances and construct the complete graph for each view. Afterwards, inspired by the structural consistency, a between-view consistency constraint is imposed to guarantee the similarity of the graphs from different views. More importantly, the TIMVC_IGC simultaneously learns the low-rank structures of the different views and explores the correlations of the different graphs in a latent manifold sub-space using a low-rank tensor constraint, such that the intrinsic graphs of the different views can be obtained. Finally, a consensus representation for each sample is gained with a co-regularization term for final clustering. Experimental results on several real-world databases illustrates that the proposed method can outperform the other state-of-the-art related methods for incomplete multi-view clustering.

In this article, we propose a novel and effective incomplete multi-view clustering (IMVC) framework, referred to as incomplete multi-view clustering with regularized hierarchical graph (IMVC_RHG). Different from the existing graph learning-based IMVC methods, IMVC_RHG introduces a novel heterogeneous-graph learning and embedding strategy, which adopts the high-order structures between four tuples for each view, rather than a simple paired-sample intrinsic structure. Besides this, with the aid of the learned heterogeneous graphs, a between-view preserving strategy is designed to recover the incomplete graph for each view. Finally, a consensus representation for each sample is gained with a co-regularization term for final clustering. As a result of integrating these three learning strategies, IMVC_RHG can be flexibly applied to different types of IMVC tasks. Comparing with the other state-of-the-art methods, the proposed IMVC_RHG can achieve the best performances on real-world incomplete multi-view databases.

Deep Neural Networks (DNNs) have achieved impressive results on the task of image denoising, but there are two serious problems. First, the denoising ability of DNNs-based image denoising models using traditional training strategies heavily relies on extensive training on clean-noise image pairs. Second, image denoising models based on DNNs usually have large parameters and high computational complexity. To address these issues, this paper proposes a two-stage Few-Shot Learning for Image Denoising (FSLID). Our FSLID is a two-stage denoising strategy integrating Basic Feature Learner (BFL), Denoising Feature Inducer (DFI), and Shared Image Reconstructor (SIR). BFL and SIR are first jointly unsupervised to train on the base image dataset$\mathcal {D}_{base}$consisting of easily collected high-quality clean images. Following this, the trained BFL extracts the guided features and constraint features for the noisy and corresponding clean images in the novel image dataset$\mathcal {D}_{novel}$, respectively. Furthermore, DFI encodes the noisy features of the noisy images in$\mathcal {D}_{novel}$. Then, inducing both the guided features and noisy features, DFI can generate the denoising prior features for the SIR with frozen weights to adaptively denoise the noisy images. Furthermore, we propose refined, low-channel-count, recursive multi-branch Multi-Scale Feature Recursive (MSFR) to modularly formulate an efficient DFI to capture more diverse contextual features information under a limited number of feature channels. Thus, compared with the baseline models, the FSLID composed of the proposed MSFR can significantly reduce the number of model parameters and computational complexity. Extensive experimental results demonstrate our FSLID significantly outperforms well-established baselines on multiple datasets and settings. We hope that our work will encourage further research to explore the field of few-shot image denoising.

Cross-modal retrieval tasks, which are more natural and challenging than traditional retrieval tasks, have attracted increasing interest from researchers in recent years. Although different modalities with the same semantics have some potential relevance, the feature space heterogeneity still seriously weakens the performance of cross-modal retrieval models. To solve this problem, common space-based methods in which multimodal data is projected into a learned common space for similarity measurement have become the mainstream approach for cross-modal retrieval tasks. However, current methods entangle the modality style and semantic content in the common space and neglect to fully explore the semantic and discriminative representation/reconstruction of the semantic content. This often results in an unsatisfactory retrieval performance. To solve these issues, this paper proposes a new Deep Supervised Dual Cycle Adversarial Network (DSDCAN) model based on common space learning. It is composed of two cross-modal cycle GANs, one for the image and one for the text. The proposed cycle GAN model disentangles the semantic content and modality style features by making the data of one modality well reconstructed from the extracted modal style feature and the content feature of the other modality. Then, a discriminative semantic and label loss is proposed by fully considering the category, sample contrast, and label supervision to enhance the semantic discrimination of the common space representation. Besides this, to make the data distribution between two modalities similar, a second-order similarity is presented as a distance measurement of the cross-modal representation in the common space. Extensive experiments have been conducted on the Wikipedia, Pascal Sentence, NUS-WIDE-10k, PKU XMedia, MSCOCO, NUS-WIDE, Flickr30k and MIRFlickr datasets. The results demonstrate that the proposed method can achieve a higher performance than the state-of-the-art methods.

Supervised deep learning techniques have been widely explored in real photograph denoising and achieved noticeable performances. However, being subject to specific training data, most current image denoising algorithms can easily be restricted to certain noisy types and exhibit poor generalizability across testing sets. To address this issue, we propose a novel flexible and well-generalized approach, coined as dual meta attention network (DMANet). The DMANet is mainly composed of a cascade of the self-meta attention blocks (SMABs) and collaborative-meta attention blocks (CMABs). These two blocks have two forms of advantages. First, they simultaneously take both spatial and channel attention into account, allowing our model to better exploit more informative feature interdependencies. Second, the attention blocks are embedded with the meta-subnetwork, which is based on metalearning and supports dynamic weight generation. Such a scheme can provide a beneficial means for self and collaborative updating of the attention maps on-the-fly. Instead of directly stacking the SMABs and CMABs to form a deep network architecture, we further devise a three-stage learning framework, where different blocks are utilized for each feature extraction stage according to the individual characteristics of SMAB and CMAB. On five real datasets, we demonstrate the superiority of our approach against the state of the art. Unlike most existing image denoising algorithms, our DMANet not only possesses a good generalization capability but can also be flexibly used to cope with the unknown and complex real noises, making it highly competitive for practical applications.

The sparsity is an attractive property that has been widely and intensively utilized in various image processing fields (e.g., robust image representation, image compression, image analysis, etc.). Its actual success owes to the exhaustive mining of the intrinsic (or homogenous) information from the whole data carrying redundant information. From the perspective of image representation, the sparsity can successfully find an underlying homogenous subspace from a collection of training data to represent a given test sample. The famous sparse representation (SR) and its variants embed the sparsity by representing the test sample using a linear combination of training samples with $L_{0}$ -norm regularization and $L_{1}$ -norm regularization. However, although these state-of-the-art methods achieve powerful and robust performances, the sparsity is not fully exploited on the image representation in the following three aspects: 1) the within-sample sparsity, 2) the between-sample sparsity, and 3) the image structural sparsity. In this paper, to make the above-mentioned multi-context sparsity properties agree and simultaneously learned in one model, we propose the concept of consensus sparsity (Con-sparsity) and correspondingly build a multi-context sparse image representation (MCSIR) framework to realize this. We theoretically prove that the consensus sparsity can be achieved by the $L_{\infty }$ -induced matrix variate based on the Bayesian inference. Extensive experiments and comparisons with the state-of-the-art methods (including deep learning) are performed to demonstrate the promising performance and property of the proposed consensus sparsity.

The Collaborative Representation (CR) framework has provided various effective and efficient solutions to pattern analysis. By leveraging between discriminative coefficient coding (l 2 regularization) and the best reconstruction quality (collaboration), the CR framework can exploit discriminative patterns efficiently in high-dimensional space. Due to the limitations of its linear representation mechanism, the CR must sacrifice its superior efficiency for capturing the non-linear information with the kernel trick. Besides this, even if the coding is indispensable, there is no mechanism designed to keep the CR free from inevitable noise brought by real-world information systems. In addition, the CR only emphasizes exploiting discriminative patterns on coefficients rather than on the reconstruction. To tackle the problems of primitive CR with a unified framework, in this article we propose the Euler Collaborative Representation (E-CR) framework. Inferred from the Euler formula, in the proposed method, we map the samples to a complex space to capture discriminative and non-linear information without the high-dimensional hidden kernel space. Based on the proposed E-CR framework, we form two specific classifiers: the Euler Collaborative Representation based Classifier (E-CRC) and the Euler Probabilistic Collaborative Representation based Classifier (E-PROCRC). Furthermore, we specifically designed a robust algorithm for E-CR (termed as R-E-CR ) to deal with the inevitable noises in real-world systems. Robust iterative algorithms have been specially designed for solving E-CRC and E-PROCRC. We correspondingly present a series of theoretical proofs to ensure the completeness of the theory for the proposed robust algorithms. We evaluated E-CR and R-E-CR with various experiments to show its competitive performance and efficiency.

Compared with uni-modal biometrics systems, multimodal biometrics systems using multiple sources of information for establishing an individual’s identity have received considerable attention recently. However, most traditional multimodal biometrics techniques generally extract features from each modality independently, ignoring the implicit associations between different modalities. In addition, most existing work uses hand-crafted descriptors that are difficult to capture the latent semantic structure. This paper proposes to learn the sparse and discriminative multimodal feature codes (SDMFCs) for multimodal finger recognition, which simultaneously takes into account the specific and common information among inter-modality and intra-modality. Specifically, given the multimodal finger images, we first establish the local difference matrix to capture informative texture features in local patches. Then, we aim to jointly learn discriminative and compact binary codes by constraining the observations from multiple modalities. Finally, we develop a novel SDMFC-based multimodal finger recognition framework, which integrates the local histograms of each division block in the learned binary codes together for classification. Experimental results on three commonly used finger databases demonstrate the effectiveness and robustness of the proposed framework in multimodal biometrics tasks.

Accompanied with the increasing popularity of linear regression approaches, most of the existing minimization problems are related with several convex measurements, e.g.,$\ell_1$/$\ell_2$/$\ell_{2,1}$-norm of a vector and$L_1$/$L_{2,1}$/Frobenius/nuclear norm of a matrix, where the regularized function and the loss function are usually studied for two objective terms case by case, respectively. To address this issue, this work combines these linear regression problems into a unified expression framework by employing an adaptive and flexible function, in which we need to choose different variable elements and adjust an inner parameter, properly. Besides this, they are equipped with some corresponding relationships and their interesting properties. Intuitively speaking, the proposed framework can generalize several traditional linear regression formulations and even more complex ones into an extended representation. For further optimizations, an iteratively re-weighted penalty solution (IRwPS) is devised without any inner loops, making the iteration programming easy to perform. Meanwhile, the theoretical results are provided for guaranteeing that the mathematical convergence analysis is solid and meaningful. Finally, by performing real-world applications in supervised, unsupervised, and semi-supervised tasks, numerical experiments are conducted to validate the theoretical properties and the superiority over some of the state-of-the-art.

In the real-world, some views of samples are often missing for the collected multiview data. Faced with the incomplete multiview data, most of the existing clustering methods tended to learn a common graph from the available views, where the hidden information of the absent views was ignored. Furthermore, some methods filled the absent instances with the average vector of the available samples for each view, which could not reflect a real distribution of the data. To solve these problems, in this paper an intrinsic and complete structure learning based incomplete multiview clustering method (ICSL_IMC) is proposed. Firstly, we calculate the initial complete graphs for all views by exploring the available incomplete graphs, which are further taken as the constraints for the reconstruction of the absent data integrating the self-representation method. Afterwards, encouraged by the complete multiview data, a complete structure inferring strategy is proposed to learn the intrinsic and complete structures for all views, such that the real distribution of the absent instances can be reflected in the completed structure of each view. We integrate these three learning phases into a joint optimization model, which can promote each other in the iterative learning procedure, simultaneously. Comparing with the other state-of-the-art methods, the proposed ICSL_IMC can achieve the best performances on different databases.

In this article, we propose a collaborative palmprint-specific binary feature learning method and a compact network consisting of a single convolution layer for efficient palmprint feature extraction. Unlike most existing palmprint feature learning methods, such as deep-learning, which usually ignore the inherent characteristics of palmprints and learn features from raw pixels of a massive number of labeled samples, palmprint-specific information, such as the direction and edge of patterns, is characterized by forming two kinds of ordinal measure vectors (OMVs). Then, collaborative binary feature codes are jointly learned by projecting double OMVs into complementary feature spaces in an unsupervised manner. Furthermore, the elements of feature projection functions are integrated into OMV extraction filters to obtain a collection of cascaded convolution templates that form a single-layer convolution network (SLCN) to efficiently obtain the binary feature codes of a new palmprint image within a single-stage convolution operation. Particularly, our proposed method can easily be extended to a general version that can efficiently perform feature extraction with more than two types of OMVs. Experimental results on five benchmark databases show that our proposed method achieves very promising feature extraction efficiency for palmprint recognition.

Dictionary-based Classification (DC) has been a promising learning theory in multimedia computing. Previous studies focused on learning a discriminative dictionary as well as the sparsest representation based on the dictionary, to cope with the complex conditions in real-world applications. However, robustness by learning only one single dictionary is far from the optimal level. What is worse, it cannot take advantage of the available techniques proven in modern machine learning, like data augmentation, to mitigate the same problem. In this work, we propose a novel method that utilizes joint Augmented and Compressed Dictionaries for Robust Dictionary-based Classification (ACD-RDC). For optimization under the noise model introduced by real-world conditions, the objective function of ACD-RDC incorporates only two simple, but well-designed constraints, including one enhanced sparsity constraint by the general data augmentation, which requires less case-by-case and sophisticated tuning, and another discriminative constraint solved by a jointly learned dictionary. The optimization of the objective function is then deduced theoretically to an approximate linear problem. The sparsity and discrimination enhanced by data augmentation guarantees the robustness for image classification under various conditions, which constructs the first positive case using data augmentation to obtain robust dictionary-based classification. Numerous experiments have been conducted on popular facial and object image datasets. The results demonstrate that ACD-RDC obtains more promising classification on diversely collected images than the current dictionary-based classification methods. ACD-RDC is also confirmed to be a state-of-the-art classification method when using deep features as inputs.

Conventional multiview clustering seeks to partition data into respective groups based on the assumption that all views are fully observed. However, in practical applications, such as disease diagnosis, multimedia analysis, and recommendation system, it is common to observe that not all views of samples are available in many cases, which leads to the failure of the conventional multiview clustering methods. Clustering on such incomplete multiview data is referred to as incomplete multiview clustering (IMC). In view of the promising application prospects, the research of IMC has noticeable advances in recent years. However, there is no survey to summarize the current progresses and point out the future research directions. To this end, we review the recent studies of IMC. Importantly, we provide some frameworks to unify the corresponding IMC methods and make an in-depth comparative analysis for some representative methods from theoretical and experimental perspectives. Finally, some open problems in the IMC field are offered for researchers. The related codes are released athttps://github.com/DarrenZZhang/Survey_IMC.

Preserving projection learning has been widely used in feature extraction and selection for unsupervised image classification. Generally, some related methods constructed a graph to represent the nearest neighbor relationships of the data based on the Euclidean distances among different samples, which used 0 or 1 to predefine whether two samples are from the same class. Since a simple Euclidean distance is sensitive to noise, the predefined graph cannot produce exact correlations between the two samples. What is more, the predefined graph cannot reflect the structure of the projected data on a latent subspace when the projection matrix is learned. To solve these problems, in this article a novel adaptive graph embedded preserving projection learning (AGE_PPL) method is proposed, first combining the sparsity-based graph learning and the projection learning as an integral framework for feature extraction and feature selection. In particular, a sparse representation term with$l_{1}$-norm is exploited in AGE_PPL to achieve the adaptive graph of the data to preserve the local structures among different samples while the projection matrix is learned. Meanwhile, a global-scale constraint is imposed to preserve the global structure of the data on a latent subspace. Therefore, the transformed samples will be more discriminative, allowing margins of the same class to be reduced, and margins among different classes to be enlarged. Experimental results proved the effectiveness of the proposed algorithm by obtaining competitive performances over other baseline and state-of-the-art methods. In addition, the proposed method is very flexible for feature selection and dimensionality reduction.

Recently, deep learning techniques are soaring and have shown dramatic improvements in real-world noisy image denoising. However, the statistics of real noise generally vary with different camera sensors and in-camera signal processing pipelines. This will induce problems of most deep denoisers for the overfitting or degrading performance due to the noise discrepancy between the training and test sets. To remedy this issue, we propose a novel flexible and adaptive denoising network, coined as FADNet. Our FADNet is equipped with a plane dynamic filter module, which generates weight filters with flexibility that can adapt to the specific input and thereby impedes the FADNet from overfitting to the training data. Specifically, we exploit the advantage of the spatial and channel attention, and utilize this to devise a decoupling filter generation scheme. The generated filters are conditioned on the input and collaboratively applied to the decoded features for representation capability enhancement. We additionally introduce the Fourier transform and its inverse to guide the predicted weight filters to adapt to the noisy input with respect to the image contents. Experimental results demonstrate the superior denoising performances of the proposed FADNet versus the state-of-the-art. In contrast to the existing deep denoisers, our FADNet is not only flexible and efficient, but also exhibits a compelling generalization capability, enjoying tremendous potential for practical usage.

The human breath has various components to show an individual’s health status or represent a concrete disease, such as kidney disease or diabetes mellitus. Moreover, electronic nose (e-nose) is a widely used method to evaluate the breath sample of people with various chemical sensors to reflect their current health situation or predict various illnesses. Thus, e-nose breath analysis is a convenient, low-cost approach at present. So far, many works focus on various disease detection, such as diabetes, lung cancer, and kidney disease via breath sample analysis. However, few studies aim to investigate multi-feature fatty liver disease detection by evaluating breath samples via the e-nose. In this study, we propose a multi-feature representation method to extract multiple features for diagnosing fatty liver from healthy candidates via breath sample analysis. In particular, two external features, i.e., low-dimensional and latent as well as one internal feature, i.e., channel are extracted from the breath sample, which are further concatenated before being applied to various classifiers for diagnosis. Experimental results indicate that our proposed approach can obtain solid performances (Accuracy of 72.39% with SVM) in detecting fatty liver disease compared to only applying single feature methods.

Binary feature representation methods have received increasing attention due to their high efficiency and great robustness to illumination variation. However, most of them are hand-designed feature descriptors that generally require much prior knowledge in their design. This paper introduces a Row-sparsity Binary Feature Learning (Rs-BFL) method to adaptively learn and encode palmprint features for open-set palmprint recognition. Given the training palmprint images, RsBFL jointly learns a bank of linear projection functions that transform the informative texture features into discriminative binary codes. Afterwards, we calculate the block-wise histograms of each feature map and concatenate them as the final feature representation. Based on the pre-trained projection matrix, we mapped the palmprint texture features of the test samples into binary features for matching. For RsBFL, we enforce three criteria: 1) the quantization error between the projected real-valued features and the binary features is minimized, at the same time, the projection noise is minimized; 2) the latent label semantic information is utilized to minimize the distance of the within-class samples and simultaneously maximize the distance of the between-class samples; 3) the$l_{2,1}$norm is used to make the projection matrix to extract more discriminative features. Extensive experimental results on two publicly accessible palmprint datasets demonstrated the effectiveness and powerful learning capability of the proposed method.

Latent subspace learning aims to produce a latent representation for better reconstruction and classification from high-dimensional data through exploiting the optimal subspace. Current latent subspace learning methods commonly have three problems: 1) The discriminative property is ignored when learning the latent subspace, 2) The redundancy exists between the latent subspace and the prediction space, 3) There is no unified latent subspace that exploits knowledge jointly from the raw space, latent subspace, and label space. In this paper, we formulate theJointDiscriminativeLatentSubspaceLearning (JDLSL) problem to address these issues, and provide its optimization solution. JDLSL learns image representation from two aspects: a) the joint learning of latent subspaces for data reconstruction and prediction, b) the joint learning of label space and latent subspace for data reconstruction. To integrate knowledge from the joint learning, we organize the sparsity-induced latent subspace, where row-sparsity and column sparsity are simultaneously imposed. We provide the theoretical proof for the discriminativity learning ability of the sparsity-induced latent subspace. Extensive experiments and comparisons with the state-of-the-art showed that the proposed method has better performance. JDLSL shows a competitive performance with deep features compared to deep learning architectures, reflecting it potential integrating with deep learning.

Image classification is a fundamental component in modern computer vision systems, where sparse representation-based classification has drawn a lot of attention due to its robustness. However, on the optimization of sparse learning systems, regularization and data augmentation are both powerful, but currently isolated. We believe that regularization and data augmentation can cooperate to generate a breakthrough in robust image classification. In this article, we propose a novel framework, regularization on augmented data (READ), which creates diversification in the data using the generic augmentation techniques to implement robust sparse representation-based image classification. When the training data are augmented, READ applies a distinct regularizer,$l_{1}$or$l_{2}$, in particular, on the augmented training data apart from the original data, so that regularization and data augmentation are utilized and enhanced synchronously. We introduce an elaborate theoretical analysis on how to optimize the sparse representation by both$l_{1}$-norm and$l_{2}$-norm with the generic data augmentation and demonstrate its performance in extensive experiments. The results obtained on several facial and object datasets show that READ outperforms many state-of-the-art methods when using deep features.

The use of deep learning for water extraction requires precise pixel-level labels. However, it is very difficult to label high-resolution remote-sensing images at the pixel level. Therefore, we study how to utilize point labels to extract water bodies and propose a novel method called the neighbor feature aggregation network (NFANet). Compared with pixel-level labels, point labels are much easier to obtain, but they will lose much information. In this article, we take advantage of the similarity between the adjacent pixels of a local water body, and propose a neighbor sampler to resample remote-sensing images. Then, the sampled images are sent to the network for feature aggregation. In addition, we use an improved recursive training algorithm to further improve the extraction accuracy, making the water boundary more natural. Furthermore, our method utilizes neighboring features instead of global or local features to learn more representative features. The experimental results show that the proposed NFANet method not only outperforms other studied weakly supervised approaches, but also obtains similar results as the state-of-the-art ones.

The low resolution of remote sensing images often limits the land cover classification (LCC) performance. Super resolution (SR) can improve the image resolution, while greatly increasing the computational burden for the LCC due to the larger size of the input image. In this article, the SR-guided deep network (SRGDN) framework is proposed, which can generate meaningful structures from higher resolution images to improve the LCC performance without consuming more computational costs. In general, the SRGDN consists of two branches (i.e., SR branch and LCC branch) and a guidance module. The SR branch aims to increase the resolution of remote sensing images. Since high- and low-resolution image pairs cannot be directly provided by imaging sensors to train the SR branch, we introduce a self-supervised generative adversarial network (GAN) to estimate the downsampling kernel that can produce these image pairs. The LCC branch adopts the high-resolution network (HRNet) to retain as much resolution information with a few downsampling operations as possible. The guidance module teaches the LCC branch to learn the high-resolution information from the SR branch without the utilization of the higher-resolution images as the inputs. Furthermore, the guidance module introduces spatial pyramid pooling (SPP) to match the feature maps of different sizes in the two branches. In the testing stage, the guidance module and SR branch can be removed, and therefore do not create additional computational costs. Experimental results on three real datasets demonstrate the superiority of the proposed method over several well-known LCC approaches.

Due to its high anti-counterfeiting and universality, the use of finger-vein pattern for identity authentication has recently attracted extensive attention in academia and industry. Despite recent advances in finger-vein recognition, most of the hand-crafted descriptors require strong prior knowledge, which may be ineffective in expressing its distinctiveness. In this paper, we present a novel compact multi-representation feature descriptor (CMrFD) with visual and semantic consistency, for finger-vein feature representation. Given the finger-vein images, we first form two-view representations to describe the informative vein features in local patches. Then, we jointly learn a feature transformation to map the two-view representations into discriminative binary codes. For the projection function, we linearly combine multi-view information and minimize the quantization error between the projected binary features and the original real-valued features. In terms of visual consistency, we minimize the Euclidean distance of each representation from the same class, at the same time, maximize the Euclidean distance from different classes in the projected space. Semantic consistency is used to ensure that similar images have compact multi-representation combined projection features. Lastly, we calculate the block-wise histograms as the final extracted features for finger-vein recognition. Experimental results on four widely used finger-vein databases demonstrate that the proposed method outperforms the state-of-the-art finger-vein recognition methods.

Domain adaptation leverages rich knowledge from a related source domain so that it can be used to perform tasks in a target domain. For more knowledge to be obtained under relaxed conditions, domain adaptation methods have been widely used in pattern recognition and image classification. However, most of the existing domain adaptation methods only consider how to minimize different distributions of the source and target domains, which neglects what should be transferred for a specific task and suffers negative transfer by distribution outliers. To address these problems, in this paper, we propose a novel domain adaptation method called weighted correlation embedding learning (WCEL) for image classification. In the WCEL approach, we seamlessly integrated correlation learning, graph embedding, and sample reweighting into a unified learning model. Specifically, we extracted the maximum correlated features from the source and target domains for image classification tasks. In addition, two graphs were designed to preserve the discriminant information from interclass samples and neighborhood relations in intraclass samples. Furthermore, to prevent the negative transfer problem, we developed an efficient sample reweighting strategy to predict the target with different confidence levels. To verify the performance of the proposed method in image classification, extensive experiments were conducted with several benchmark databases, verifying the superiority of the WCEL method over other state-of-the-art domain adaptation algorithms.

Recent deep convolutional neural networks for real-world noisy image denoising have shown a huge boost in performance by training a well-engineered network over external image pairs. However, most of these methods are generally trained with supervision. Once the testing data is no longer compatible with the training conditions, they can exhibit poor generalization and easily result in severe overfitting or degrading performances. To tackle this barrier, we propose a novel denoising algorithm, dubbed as Meta PID Attention Network (MPA-Net). Our MPA-Net is built based upon stacking Meta PID Attention Modules (MPAMs). In each MPAM, we utilize a second-order attention module (SAM) to exploit the channel-wise feature correlations with second-order statistics, which are then adaptively updated via a proportional-integral-derivative (PID) guided meta-learning framework. This learning framework exerts the unique property of the PID controller and meta-learning scheme to dynamically generate filter weights for beneficial update of the extracted features within a feedback control system. Moreover, the dynamic nature of the framework enables the generated weights to be flexibly tweaked according to the input at test time. Thus, MPAM not only achieves discriminative feature learning, but also facilitates a robust generalization ability on distinct noises for real images. Extensive experiments on ten datasets are conducted to inspect the effectiveness of the proposed MPA-Net quantitatively and qualitatively, which demonstrates both its superior denoising performance and promising generalization ability that goes beyond those of the state-of-the-art denoising methods.

Semantic segmentation methods based on deep neural networks have achieved great success in recent years. However, training such deep neural networks relies heavily on a large number of images with accurate pixel-level labels, which requires a huge amount of human effort, especially for large-scale remote sensing images. In this paper, we propose a point-based weakly supervised learning framework called the deep bilateral filtering network (DBFNet) for the semantic segmentation of remote sensing images. Compared with pixel-level labels, point annotations are usually sparse and cannot reveal the complete structure of the objects; they also lack boundary information, thus resulting in incomplete prediction within the object and the loss of object boundaries. To address these problems, we incorporate the bilateral filtering technique into deeply learned representations in two respects. First, since a target object contains smooth regions that always belong to the same category, we perform deep bilateral filtering (DBF) to filter the deep features by a nonlinear combination of nearby feature values, which encourages the nearby and similar features to become closer, thus achieving a consistent prediction in the smooth region. In addition, the DBF can distinguish the boundary by enlarging the distance between the features on different sides of the edge, thus preserving the boundary information well. Experimental results on two widely used datasets, the ISPRS 2-D semantic labeling Potsdam and Vaihingen datasets, demonstrate that our proposed DBFNet can achieve a highly competitive performance compared with state-of-the-art fully-supervised methods. Code is available at https://github.com/Luffy03/DBFNet.

With the development of deep learning technologies, recent research on real-world noisy image denoising has achieved a considerable improvement in performance. However, a common limitation for existing approaches is the imbalanced trade-off between denoising accuracy and efficiency. To address this problem, we propose a robust and efficient denoiser, called a hierarchical-based PID-attention denoising network (HPDNet), to flexibly deal with the sophisticated noise. The core of our algorithm is the PID-attentive recurrent network (PAR-Net) whose framework mainly consists of the LSTM network and PID controller. PAR-Net inherits the advantages of both the attentive recurrent network and control action, which can encourage more discriminatory feature representations. This learning procedure is implemented within a feedback control system, allowing a faster and more robust means to enhance feature discriminability. Furthermore, by decomposing the noisy image and stacking the PAR-Nets, our PAR-Net can work on a progressively hierarchical framework, and hence obtain multi-scale features and manageable successive refinements. On several widely used datasets, the proposed HPDNet demonstrates high efficiency, while delivering a better perceptually appealing image quality over state-of-the-art image denoising methods.

Objects often have different appearances because of viewpoint changes or part deformation. How to reasonably model these variations is still a big challenge for object detection. In this paper, we propose a novel Deformable Template Network (DTN), which exploits the pictorial structure to model possible variations of an object. DTN represents an object by virtue of a generated template in a deformable way. It has two key modules: the template generating module and the part matching module. The template generating module produces a template for a given object which defines the anchor positions of the$k{\times }k$parts. Based on such a template, the part matching module aims to perform part alignment around the anchor positions. In terms of each part, the matching process makes a trade-off between maximizing the detection score and minimizing the deformation cost relative to the anchor position. Moreover, DTN is a fully convolutional network which means it is competitive in terms of detection efficiency. We evaluate DTN on both the PASCAL VOC and MSCOCO datasets, achieving the state-of-the-art results, an accuracy of 82.7% for PASCAL VOC and of 44.9% for MSCOCO.

Heterogeneous palmprint recognition has attracted considerable research attention in recent years because it has the potential to greatly improve the recognition performance for personal authentication. In this article, we propose a simultaneous heterogeneous palmprint feature learning and encoding method for heterogeneous palmprint recognition. Unlike existing hand-crafted palmprint descriptors that usually extract features from raw pixels and require strong prior knowledge to design them, the proposed method automatically learns the discriminant binary codes from the informative direction convolution difference vectors of palmprint images. Differing from most heterogeneous palmprint descriptors that individually extract palmprint features from each modality, our method jointly learns the discriminant features from heterogeneous palmprint images so that the specific discriminant properties of different modalities can be better exploited. Furthermore, we present a general heterogeneous palmprint discriminative feature learning model to make the proposed method suitable for multiple heterogeneous palmprint recognition. Experimental results on the widely used PolyU multispectral palmprint database clearly demonstrate the effectiveness of the proposed method.

Real photograph denoising is extremely challenging in low-level computer vision since the noise is sophisticated and cannot be fully modeled by explicit distributions. Although deep-learning techniques have been actively explored for this issue and achieved convincing results, most of the networks may cause vanishing or exploding gradients, and usually entail more time and memory to obtain a remarkable performance. This article overcomes these challenges and presents a novel network, namely, PID controller guide attention neural network (PAN-Net), taking advantage of both the proportional-integral-derivative (PID) controller and attention neural network for real photograph denoising. First, a PID-attention network (PID-AN) is built to learn and exploit discriminative image features. Meanwhile, we devise a dynamic learning scheme by linking the neural network and control action, which significantly improves the robustness and adaptability of PID-AN. Second, we explore both the residual structure and share-source skip connections to stack the PID-ANs. Such a framework provides a flexible way to feature residual learning, enabling us to facilitate the network training and boost the denoising performance. Extensive experiments show that our PAN-Net achieves superior denoising results against the state-of-the-art in terms of image quality and efficiency.

Recently, contactless bimodal palmprint recognition technology has attracted increased attention due to the COVID-19 pandemic. Many dual-camera-based sensors have been proposed to capture palm vein and palmprint images synchronously. However, translations between captured palmprint and palm vein images differ depending on the distance between the hand and the sensors. To address this issue, we designed a low-cost method to align the bimodal palm regions for current dual-camera systems. In this study, we first implemented a contactless palm image acquisition device with a dual-camera module and a single-point time of flight (TOF) ranging sensor. Using this device, we collected a dataset named DCPD under different distances and light source intensities from 271 different palms. Then, a bimodal palm image alignment method is proposed based on the imaging and ranging models. After the system model is calibrated, the translation between the visible light and infrared light palm regions can be estimated quickly based on the palm distance. Finally, we designed a convolutional neural network (CNN) to effectively extract the fine- and coarse-grained palm features. Compared to widely used existing methods, the proposed networks achieved the lowest equal error rate (EER) on the Tongji, IITD, and DCPD datasets, and the average time cost of the system to perform one-time identification is approximately 0.15 s. The experimental results indicate that the proposed methods achieved high efficiency and comparable accuracy. In addition, the system’s EER and rank-1 on the DCPD dataset were 0.304% and 98.66%, respectively.

Various palmprint recognition methods have been proposed and applied in security, particularly authentication. However, improving the performance of palmprint recognition with insufficient training samples per person is still a challenging task. The undersampling problem limits the application and popularization of palmprint recognition. In this article, by regularly sampling different local regions of the palmprint image, we learn complete and discriminative convolution features by using deep convolutional neural networks (DCNNs). With this powerful palmprint description, a joint constrained least-square regression (JCLSR) framework, which performs representation for each local region of the same palmprint image requiring all regular local regions of the palmprint image to have similar projected target matrices, is presented to exploit the commonality of different patches. The proposed method can well solve the undersampling classification problem in palmprint recognition. Experiments were conducted on the IITD, CASIA, noisy IITD, and PolyU multispectral palmprint databases. It can be seen from the experimental results that the proposed JCLSR consistently outperformed the classical palmprint recognition methods and some subspace learning-based methods for palmprint recognition.

In this paper, we propose a novel method, referred to as incomplete multi-view tensor spectral clustering with missing-view inferring (IMVTSC-MVI) to address the challenging multi-view clustering problem with missing views. Different from the existing methods which commonly focus on exploring the certain information of the available views while ignoring both of the hidden information of the missing views and the intra-view information of data, IMVTSC-MVI seeks to recover the missing views and explore the full information of such recovered views and available views for data clustering. In particular, IMVTSC-MVI incorporates the feature space based missing-view inferring and manifold space based similarity graph learning into a unified framework. In such a way, IMVTSC-MVI allows these two learning tasks to facilitate each other and can well explore the hidden information of the missing views. Moreover, IMVTSC-MVI introduces the low-rank tensor constraint to capture the high-order correlations of multiple views. Experimental results on several datasets demonstrate the effectiveness of IMVTSC-MVI for incomplete multi-view clustering.

In recent years, incomplete multi-view clustering has drawn increasing attention due to the existence of large amounts of unlabeled incomplete data whose views are not fully observed in the practical applications. Although many traditional methods have been extended to address the incomplete learning problem, most of them exploit the shallow models and ignore the geometric structure. To address these issues, we proposed a structural deep incomplete multi-view clustering network. Specifically, the proposed method can simultaneously explore the high-level features and high-order geometric structure information of data with several view-specific graph convolutional encoder networks and can directly obtain the optimal clustering indicator matrix in one stage. Experimental results on several datasets with the comparison of state-of-the-art methods validate the superiority of the proposed method.

The use of finger-vein (FV) trait for the purpose of identity authentication has attracted much attention in recent years. However, most of the conventional FV recognition methods are hand-crafted by design and require strong prior knowledge, which are ineffective at expressing the distinctiveness of the FV images. In this paper, we propose an adaptive discriminant and sparsity feature descriptor (DSFD) for FV feature extraction and recognition. Specifically, we first form a direction difference vector (DDV) to better represent the direction feature of the FV images. Afterwards, the DSFD adaptively projects the DDVs into a feature space with discriminative binary codes in which the distance of the within-class samples is minimized and simultaneously the distance of the between-class samples is maximized. Lastly, we concatenate the block-wise histograms into a global histogram as the final feature descriptor for FV recognition. Experimental results on two publicly available FV databases demonstrate the effectiveness of the proposed method.

In a typical Generative Adversarial Network (GAN), a noise is sampled to generate fake samples via a series of convolutional operations after random initialization. However, current GANs merely rely on the pixel space to sample the noise, in-creasing the difficulty of approaching the target distribution. Fortunately, the long proven wavelet transformation is able to decompose multiple spectral information from the images. In this work, we propose a novel multi-channel wavelet-based filtering method for GANs, to cope with this problem. The proposed WaveletNet embeds a wavelet deconvolution layer in the generator to take advantage of the wavelet deconvolution. By learning a filter with multiple channels, or multiple convolutional filters, it can efficiently homogenize the sampled noise via an averaging operation, to generate high-fidelity samples. We conducted benchmark experiments on the Fashion-MNIST, KMNIST, and SVHN datasets through an open GAN bench-mark tool. The results showed that WaveletGAN has excellent performance in generating high-fidelity samples.

Image brightness in color representation has caught active research interests, while its influence to texture features is relatively rarely studied. In this paper, we address the issue of illuminance, or brightness, interference to texture descriptors, especially to the Gabor filter based approaches. Firstly, we reveal the fact that the Gabor filter response linearly drifts with the illuminance. Secondly, the Hodge decomposition is introduced, and a linear regression model is presented to interpret the linear drift phenomenon. An interesting connection with the intrinsic image decomposition is established. Finally, a partial differential equation based method is proposed to compensate the drift issue. Consequently, the texture of the compensated image is enhanced. Extensive experiments are conducted to verify the proposed statements. The proposed linear regression model is verified empirically, and the compensation performances are demonstrated. Classification results on various databases indicate that the compensation can effectively improve the capacity of many existing texture description methods. Especially, the results show that the boosting effect on some deep learning based methods is also valid.

Multiview learning has been widely studied in various fields and achieved outstanding performances in comparison to many single-view-based approaches. In this paper, a novel multiview learning method based on the Gaussian process latent variable model (GPLVM) is proposed. In contrast to existing GPLVM methods which only assume that there are transformations from the latent variable to the multiple observed inputs, our proposed method simultaneously takes a back constraint into account, encoding multiple observations to the latent variable by enjoying the Gaussian process (GP) prior. Particularly, to overcome the difficulty of the covariance matrix calculation in the encoder, a linear projection is designed to map different observations to a consistent subspace first. The obtained variable in this subspace is then projected to the latent variable in the manifold space with the GP prior. Furthermore, different from most GPLVM methods which strongly assume that the covariance matrices follow a certain kernel function, for example, radial basis function (RBF), we introduce a multikernel strategy to design the covariance matrix, being more reasonable and adaptive for the data representation. In order to apply the presented approach to the classification, a discriminative prior is also embedded to the learned latent variables to encourage samples belonging to the same category to be close and those belonging to different categories to be far. Experimental results on three real-world databases substantiate the effectiveness and superiority of the proposed method compared with state-of-the-art approaches.

Multimodal biometrics recognition has recently attracted much interest for its higher security and effectiveness compared with unimodal biometrics recognition. However, most of the conventional multimodal recognition approaches generally focus on extracting semantic information from different modalities independently, while ignoring the implicit correlations among inter-modality. In this paper, we propose a simple yet effective supervised multimodal feature learning method, called joint discriminative sparse coding (JDSC), which is applied for hand-based multimodal recognition including finger-vein and finger-knuckle-print fusion, palm-vein and palmprint fusion, as well as palm-vein and dorsal-hand-vein fusion. Considering that relevant samples from different modalities have semantic correlations, JDSC projects the raw data into a shared space in which the distance of the between-class is maximized and the distance of the within-class is minimized, at the same time, the correlation among the inter-modality of the within-class is maximized. Therefore, sparse binary codes quantified by the obtained projection matrix can have more discriminative power for multimodal recognition tasks. Thorough experiments on six commonly used multimodal datasets demonstrate the superiority of our proposed method over several state-of-the-art techniques.

Palmprint direction patterns have been widely and successfully used in palmprint recognition methods. Most existing direction-based methods utilize the pre-defined filters to achieve the genuine line responses in the palmprint image, which requires rich prior knowledge and usually ignores the vital direction information. In addition, some line responses influenced by noise will degrade the recognition accuracy. Furthermore, how to extract the discriminative features to make the palmprint more separable is also a dilemma for improving the recognition performance. To solve these problems, we propose to learn complete and discriminative direction patterns in this study. We first extract the complete and salient local direction patterns, which contains a complete local direction feature (CLDF) and a salient convolution difference feature (SCDF) extracted from the palmprint image. Afterwards, two learning models are proposed to learn sparse and discriminative directions from CLDF and to achieve the underlying structure for the SCDFs in the training samples, respectively. Lastly, the projected CLDF and the projected SCDF are concatenated forming the complete and discriminative direction feature for palmprint recognition. Experimental results on seven palmprint databases, as well as three noisy datasets clearly demonstrates the effectiveness of the proposed method.

It is well known in Traditional Chinese Medicine (TCM) that a person's wrist pulse signal can reflect their health condition. Recently, many computerized wrist pulse AI systems have been proposed to simulate a practitioner's three fingers in order to acquire the wrist pulse signals (three positions/channels) from a candidate's wrist dynamically, before evaluating their health status based on the various feature extraction and detection methods. However, few works have investigated the correlation of the extracted features from the three wrist channels and comprehensively fused the various features together, which can improve the performance of wrist pulse diagnosis. In this paper, we propose a graph based multichannel feature fusion (GBMFF) method to utilize the multichannel features of the wrist pulse signals effectively. In detail, two different sensors, i.e., pressure and photoelectricity are used to capture the three channels of the wrist pulse signals. These are used to generate two different features by applying the stacked sparse autoencoder and wavelet scattering. Each feature of one wrist pulse sample is regarded as a node associated with its corresponding feature vector, and used to construct a graph for one candidate. A novel algorithm is implemented to construct different graphs for different candidates, which are used for wrist pulse diagnosis by developing graph convolutional networks. Experimental results indicate that our proposed AI-based method can obtain superior performances compared to other state-of-the-art approaches.

Dictionary-based classification has been promising in knowledge discovery from image data, due to its good performance and interpretable theoretical system. Dictionary learning effectively supports both small- and large-scale datasets, while its robustness and performance depends on the atoms of the dictionary most of the time. Empirically, using a large number of atoms is helpful to obtain a robust classification, while robustness cannot be ensured when setting a small number of atoms. However, learning a huge dictionary dramatically slows down the speed of classification, which is especially worse on the large-scale datasets. To address the problem, we propose a Fast and Robust Dictionary-based Classification (FRDC) framework, which fully utilizes the learned dictionary for classification by staging - and -norms to obtain a robust sparse representation. The new objective function, on the one hand, introduces an additional -norm term upon the conventional -norm optimization, which generates a more robust classification. On the other hand, the optimization based on both - and -norms is solved in two stages, which is much easier and faster than current solutions. In this way, even when using a limited size of dictionary, which makes sure the classification runs very fast, it still can gain higher robustness for multiple types of image data. The optimization is then theoretically analyzed in a new formulation, close but distinct to elastic-net, to prove it is crucial to improve the performance under the premise of robustness. According to our extensive experiments conducted on four image datasets for face and object classification, FRDC keeps generating a robust classification no matter whether using a small or large number of atoms. This guarantees a fast and robust dictionary-based image classification. Furthermore, when simply using deep features extracted via some popular pre-trained neural networks, it outperforms many state-of-the-art methods on the specific datasets.

In this article, we propose a multifeature learning method to jointly learn compact multifeature codes (LCMFCs) for palmprint recognition with a single training sample per palm. Unlike most existing hand-crafted methods that extract single-type features from raw pixels, we first form the multi-type data vectors such as the direction-data, and texture-data to completely sample the multiple information of a palmprint image. Then, we learn the discriminative multifeatures from multi-type data vectors by maximizing the inter-palm distance, and minimizing the energy loss between the learned codes, and the original data. Moreover, our LCMFC method adaptively learns the optimal weights of multi-type features to jointly learn the compact multifeature codes. Finally, we cluster the nonoverlapping blockwise histograms of the compact multifeature codes into a feature vector for palmprint representation. Extensive experimental results on six benchmark palmprint databases are presented to show the effectiveness of the proposed method.

Deep convolutional neural networks (CNNs) have been popularly adopted in image super-resolution (SR). However, deep CNNs for SR often suffer from the instability of training, resulting in poor image SR performance. Gathering complementary contextual information can effectively overcome the problem. Along this line, we propose a coarse-to-fine SR CNN (CFSRCNN) to recover a high-resolution (HR) image from its low-resolution version. The proposed CFSRCNN consists of a stack of feature extraction blocks (FEBs), an enhancement block (EB), a construction block (CB) and, a feature refinement block (FRB) to learn a robust SR model. Specifically, the stack of FEBs learns the long- and short-path features, and then fuses the learned features by expending the effect of the shallower layers to the deeper layers to improve the representing power of learned features. A compression unit is then used in each FEB to distill important information of features so as to reduce the number of parameters. Subsequently, the EB utilizes residual learning to integrate the extracted features to prevent from losing edge information due to repeated distillation operations. After that, the CB applies the global and local LR features to obtain coarse features, followed by the FRB to refine the features to reconstruct a high-resolution image. Extensive experiments demonstrate the high efficiency and good performance of our CFSRCNN model on benchmark datasets compared with state-of-the-art SR models. The code of CFSRCNN is accessible onhttps://github.com/hellloxiaotian/CFSRCNN.

In real-world applications, it is often that the collected multi-view data are incomplete, i.e., some views of samples are absent. Existing clustering methods for incomplete multi-view data all focus on obtaining a common representation or graph from the available views but neglect the hidden information of missing views and information imbalance of different views. To solve these problems, a novel method, called adaptive graph completion based incomplete multi-view clustering (AGC_IMC), is proposed in this paper. Specifically, AGC_IMC develops a joint framework for graph completion and consensus representation learning, which mainly contains three components, i.e., within-view preservation, between-view inferring, and consensus representation learning. To reduce the negative influence of information imbalance, AGC_IMC introduces some adaptive weights to balance the importance of different views during the consensus representation learning. Importantly, AGC_IMC has the potential to recover the similarity graphs of all views with the optimal cluster structure, which encourages it to obtain a more discriminative consensus representation. Experimental results on five well-known datasets show that AGC_IMC significantly outperforms the state-of-the-art methods.

Multiview learning has shown its superiority in visual classification compared with the single-view-based methods. Especially, due to the powerful representation capacity, the Gaussian process latent variable model (GPLVM)-based multiview approaches have achieved outstanding performances. However, most of them only follow the assumption that the shared latent variables can be generated from or projected to the multiple observations but fail to exploit the harmonization in the back constraint and adaptively learn a classifier according to these learned variables, which would result in performance degradation. To tackle these two issues, in this article, we propose a novel harmonization shared autoencoder GPLVM with a relaxed Hamming distance (HSAGP-RHD). Particularly, an autoencoder structure with the Gaussian process (GP) prior is first constructed to learn the shared latent variable for multiple views. To enforce the agreement among various views in the encoder, a harmonization constraint is embedded into the model by making consistency for the view-specific similarity. Furthermore, we also propose a novel discriminative prior, which is directly imposed on the latent variable to simultaneously learn the fused features and adaptive classifier in a unit model. In detail, the centroid matrix corresponding to the centroids of different categories is first obtained. A relaxed Hamming distance (RHD)-based measurement is subsequently presented to measure the similarity and dissimilarity between the latent variable and centroids, not only allowing us to get the closed-form solutions but also encouraging the points belonging to the same class to be close, while those belonging to different classes to be far. Due to this novel prior, the category of the out-of-sample is also allowed to be simply assigned in the testing phase. Experimental results conducted on three real-world data sets demonstrate the effectiveness of the proposed method compared with state-of-the-art approaches.

Diabetes mellitus and lung cancer are two of the most common fatal diseases in the world, causing considerable deaths every year. However, it is not easy to detect diabetes mellitus and lung cancer efficiently--needing professional medical instruments such as a CT and a qualified individual to perform the Fasting Plasma Glucose test. Considering the risks and various inconveniences with conventional diagnosis methods, noninvasive approaches based on computerized analysis are desired. The aim of this paper is to distinguish patients with diabetes mellitus, lung cancer from healthy people simultaneously by analyzing facial images through the stacked sparse autoencoder. Experimental results on a dataset containing 450 healthy samples, 284 diabetes and 175 lung cancer patients produced the F1-score of 93.57%, 97.54%, 81.56% for detecting healthy, diabetes and lung cancer, respectively, validating the effectiveness of our proposed method.

Least squares regression (LSR) has two main issues that greatly limits the improvement of performance: 1) The target matrix is too rigid leading to a large regression error; 2) the underlying geometric structure of the training data is often ignored to learn a more discriminative projection matrix. To solve these dilemmas, this paper presents a discriminant and sparsity based least squares regression with l1regularization (DS_LSR). In DS_LSR, the sparse coefficient matrix of the training data with l1regularization is jointly learned with the projection matrix to make the projection matrix discriminative. In addition, an orthogonal relaxed term is introduced to hold the structure of regression targets while relaxing the rigid label matrix. Extensive experimental results demonstrate the effectiveness of the proposed method in classification accuracy.

Currently, diabetes mellitus and breast cancer have become more widespread than ever before. Those suffering from these two types of diseases usually need a blood test or biopsy, where both extract fluids or tissues from the human body, which brings pain and a sense of discomfort. With the rise of medical biometrics, it is possible to perform non-invasive detection according to the biometric identifiers from the face of the patients. However, it is still difficult to simultaneously perform disease detection on both diabetes mellitus and breast cancer accurately. To resolve this issue, in this paper, we propose a progressive stack face-based network (PF-Net) to perform multi-class classification on diabetes mellitus, breast cancer, and healthy control using facial information. To perform diagnosis in a progressive way, a latent facial representation is first generated from a stacked sparse autoencoder. Later, the representation is fed into an ensemble layer containing several classifiers. Finally, only the effective classifiers are activated in the classification layer to make the final decision. The experiments showed our proposed method achieved an overall Accuracy of 92.94%, which outperforms a number of classification methods.

3D palmprint-based biometric recognition has drawn growing research attention due to its several merits over 2D counterpart such as robust structural measurement of a palm surface and high anti-counterfeiting capability. However, most existing 3D palmprint descriptors are hand-crafted that usually extract stationary features from 3D palmprint images. In this paper, we propose a feature learning method to jointly learn compact curvature feature descriptor for 3D palmprint recognition. We first form multiple curvature data vectors to completely sample the intrinsic curvature information of 3D palmprint images. Then, we jointly learn a feature projection function that project curvature data vectors into binary feature codes, which have the maximum inter-class variances and minimum intra-class distance so that they are discriminative. Moreover, we learn the collaborative binary representation of the multiple curvature feature codes by minimizing the information loss between the final representation and the multiple curvature features, so that the proposed method is more compact in feature representation and efficient in matching. Experimental results on the baseline 3D palmprint database demonstrate the superiority of the proposed method in terms of recognition performance in comparison with state-of-the-art 3D palmprint descriptors.

In recent years, incomplete multi-view clustering, which studies the challenging multi-view clustering problem on missing views, has received growing research interests. Although a series of methods have been proposed to address this issue, the following problems still exist: 1) Almost all of the existing methods are based on shallow models, which is difficult to obtain discriminative common representations. 2) These methods are generally sensitive to noise or outliers since the negative samples are treated equally as the important samples. In this paper, we propose a novel incomplete multi-view clustering network, called Cognitive Deep Incomplete Multi-view Clustering Network (CDIMC-net), to address these issues. Specifically, it captures the high-level features and local structure of each view by incorporating the view-specific deep encoders and graph embedding strategy into a framework. Moreover, based on the human cognition, \emph{i.e.}, learning from easy to hard, it introduces a self-paced strategy to select the most confident samples for model training, which can reduce the negative influence of outliers. Experimental results on several incomplete datasets show that CDIMC-net outperforms the state-of-the-art incomplete multi-view clustering methods.

In this paper, a new deep incomplete multi-view clustering network, called DIMC-net, is proposed to address the challenge of multi-view clustering on missing views. In particular, DIMC-net designs several view-specific encoders to extract the high-level information of multiple views and introduces a fusion graph based constraint to explore the local geometric information of data. To reduce the negative influence of missing views, a weighted fusion layer is introduced to obtain the consensus representation shared by all views. Moreover, a clustering layer is introduced to guarantee that the obtained consensus representation is the best one for the clustering task. Compared with the existing deep learning based approaches, DIMC-net is more flexible and efficient since it can handle all kinds of incomplete cases and directly produce the clustering results. Experimental results show that DIMC-net achieves significant improvement over state-of-the-art incomplete multi-view clustering methods.

Abstract Physically plausible fracture animation is a challenging topic in computer graphics. Most of the existing approaches focus on the fracture of isotropic materials. We proposed a frame‐field method for the design of anisotropic brittle fracture patterns. In this case, the material anisotropy is determined by two parts: anisotropic elastic deformation and anisotropic damage mechanics. For the elastic deformation, we reformulate the constitutive model of hyperelastic materials to achieve anisotropy by adding additional energy density functions in particular directions. For the damage evolution, we propose an improved phase‐field fracture method to simulate the anisotropy by designing a deformation‐aware second‐order structural tensor. These two parts can present elastic anisotropy and fractured anisotropy independently, or they can be well coupled together to exhibit rich crack effects. To ensure the flexibility of simulation, we further introduce a frame‐field concept to assist in setting local anisotropy, similar to the fiber orientation of textiles. For the discretization of the deformable object, we adopt a novel Material Point Method(MPM) according to its fracture‐friendly nature. We also give some design criteria for anisotropic models through comparative analysis. Experiments show that our anisotropic method is able to be well integrated with the MPM scheme for simulating the dynamic fracture behavior of anisotropic materials.

Abstract Physically based deformation simulation has been studied for many years in computer graphics. In order to simulate more complex geometric models and better meet the designer's requirements, many anisotropic approaches have been proposed in recent years. However, most of the approaches focus on simulating orthotropic models. In comparison with orthotropic models, nonorthotropic ones allow the objects to have anisotropic behaviors along nonorthogonal directions. In this paper, we introduce an improved approach to simulate nonorthotropic geometric models under large deformation. The improvements are mainly twofold. First, a frame field is specified on a given undeformed object, that is, each point of the object is equipped with a frame. In each local frame, we construct three independent vectors and form a nonorthogonal coordinate. Second, we design the deformation properties along each axis in the local nonorthogonal coordinate to get a local constitutive model. The final nonorthotropic model is generated by transforming the designed model from local nonorthogonal coordinates to the global standard Cartesian coordinate. To improve the stability, we introduce a time‐varying method to simultaneously track the local coordinates reorientation by pushing forward the original frame field to the deformed frame field. Experiments show that the deformation simulation using the designed nonorthotropic models exhibits anisotropic behaviors along different directions and are more stable than previous methods.

Direction-based methods are the most powerful and popular palmprint recognition methods. However, there is no existing work that completely analyzes the essential differences among different direction-based methods and explores the most discriminant direction representation of a palmprint. In this paper, we attempt to establish the connection between the direction feature extraction model and the discriminability of direction features, and we propose a novel exponential and Gaussian fusion model (EGM) to characterize the discriminative power of different directions. The EGM can provide us with a new insight into the optimal direction feature selection of palmprints. Moreover, we propose a local discriminant direction binary pattern (LDDBP) to completely represent the direction features of a palmprint. Guided by the EGM, the most discriminant directions can be exploited to form the LDDBP-based descriptor for palmprint representation and recognition. Extensive experiment results conducted on four widely used palmprint databases demonstrate the superiority of the proposed LDDBP method over the state-of-the-art direction-based methods.

In this paper, a novel deep network is proposed for multi-focus image fusion, named Deep Regression Pair Learning (DRPL). In contrast to existing deep fusion methods which divide the input image into small patches and apply a classifier to judge whether the patch is in focus or not, DRPL directly converts the whole image into a binary mask without any patch operation, subsequently tackling the difficulty of the blur level estimation around the focused/defocused boundary. Simultaneously, a pair learning strategy, which takes a pair of complementary source images as inputs and generates two corresponding binary masks, is introduced into the model, greatly imposing the complementary constraint on each pair and making a large contribution to the performance improvement. Furthermore, as the edge or gradient does exist in the focus part while there is no similar property for the defocus part, we also embed a gradient loss to ensure the generated image to be all-in-focus. Then the structural similarity index (SSIM) is utilized to make a trade-off between the reference and fused images. Experimental results conducted on the synthetic and real-world datasets substantiate the effectiveness and superiority of DRPL compared with other state-of-the-art approaches. The testing code can be found in https://github.com/sasky1/DPRL.

Subspace learning and reconstruction have been widely explored in recent transfer learning work. Generally, a specially designed projection and reconstruction transfer functions bridging multiple domains for heterogeneous knowledge sharing are wanted. However, we argue that the existing subspace reconstruction based domain adaptation algorithms neglect the class prior, such that the learned transfer function is biased, especially when data scarcity of some class is encountered. Different from those previous methods, in this paper, we propose a novel class-wise reconstruction-based adaptation method called Class-specific Reconstruction Transfer Learning (CRTL), which optimizes a well modeled transfer loss function by fully exploiting intra-class dependency and inter-class independency. The merits of the CRTL are three-fold. 1) Using a class-specific reconstruction matrix to align the source domain with the target domain fully exploits the class prior in modeling the domain distribution consistency, which benefits the cross-domain classification. 2) Furthermore, to keep the intrinsic relationship between data and labels after feature augmentation, a projected Hilbert-Schmidt Independence Criterion (pHSIC), that measures the dependency between data and label, is first proposed in transfer learning community by mapping the data from raw space to RKHS. 3) In addition, by imposing low-rank and sparse constraints on the class-specific reconstruction coefficient matrix, the global and local data structure that contributes to domain correlation can be effectively preserved. Extensive experiments on challenging benchmark datasets demonstrate the superiority of the proposed method over state-of-the-art representation-based domain adaptation methods. The demo code is available in https://github.com/wangshanshanCQU/CRTL.

Sparse representation based classification (SRC), nuclear-norm matrix regression (NMR), and deep learning (DL) have achieved a great success in face recognition (FR). However, there still exist some intrinsic limitations among them. SRC and NMR based coding methods belong to one-step model, such that the latent discriminative information of the coding error vector cannot be fully exploited. DL, as a multi-step model, can learn powerful representation, but relies on large-scale data and computation resources for numerous parameters training with complicated back-propagation. Straightforward training of deep neural networks from scratch on small-scale data is almost infeasible. Therefore, in order to develop efficient algorithms that are specifically adapted for small-scale data, we propose to derive the deep models of SRC and NMR. Specifically, in this paper, we propose an end-to-end deep cascade model (DCM) based on SRC and NMR with hierarchical learning, nonlinear transformation and multi-layer structure for corrupted face recognition. The contributions include four aspects. First, an end-to-end deep cascade model for small-scale data without back-propagation is proposed. Second, a multi-level pyramid structure is integrated for local feature representation. Third, for introducing nonlinear transformation in layer-wise learning, softmax vector coding of the errors with class discrimination is proposed. Fourth, the existing representation methods can be easily integrated into our DCM framework. Experiments on a number of small-scale benchmark FR datasets demonstrate the superiority of the proposed model over state-of-the-art counterparts. Additionally, a perspective that deep-layered learning does not have to be convolutional neural network with back-propagation optimization is consolidated. The demo code is available in https://github.com/liuji93/DCM.

In recent years, low-rank matrix recovery problems have attracted much attention in computer vision and machine learning. The corresponding rank minimization problems are both combinational and NP-hard in general, which are mainly solved by both nuclear norm and Schatten-p (0<p<1) norm based optimization algorithms. However, inspired by weighted nuclear norm and Schatten-p norm as the relaxations of rank function, the main merits of this work firstly provide a modified Schatten-p norm in the affine matrix rank minimization problem, denoted as the modified Schatten-p norm minimization (MSpNM). Secondly, its surrogate function is constructed and the equivalence relationship with the MSpNM is further achieved. Thirdly, the iterative singular value thresholding algorithm (ISVTA) is devised to optimize it, and its accelerated version, i.e., AISVTA, is also obtained to reduce the number of iterations through the well-known Nesterov's acceleration strategy. Most importantly, the convergence guarantees and their relationship with objective function, stationary point and variable sequence generated by the proposed algorithms are established under some specific assumptions, e.g., Kurdyka-Łojasiewicz (KŁ) property. Finally, numerical experiments demonstrate the effectiveness of the proposed algorithms in the matrix completion problem for image inpainting and recommender systems. It should be noted that the accelerated algorithm has a much faster convergence speed and a very close recovery precision when comparing with the proposed non-accelerated one.

Due to the powerful capability of the data representation, deep learning has achieved a remarkable performance in supervised hash function learning. However, most of the existing hashing methods focus on point-to-point matching that is too strict and unnecessary. In this article, we propose a novel deep supervised hashing method by relaxing the matching between each pair of instances to a point-to-angle way. Specifically, an inner product is introduced to asymmetrically measure the similarity and dissimilarity between the real-valued output and the binary code. Different from existing methods that strictly enforce each element in the real-valued output to be either +1 or -1, we only encourage the output to be close to its corresponding semantic-related binary code under the cross-angle. This asymmetric product not only projects both the real-valued output and the binary code into the same Hamming space but also relaxes the output with wider choices. To further exploit the semantic affinity, we propose a novel Hamming-distance-based triplet loss, efficiently making a ranking for the positive and negative pairs. An algorithm is then designed to alternatively achieve optimal deep features and binary codes. Experiments on four real-world data sets demonstrate the effectiveness and superiority of our approach to the state of the art.

The sparse representation-based classification (SRC) has been utilized in many applications and is an effective algorithm in machine learning. However, the performance of SRC highly depends on the data distribution. Some existing works proved that SRC could not obtain satisfactory results on uncontrolled data sets. Except the uncontrolled data sets, SRC cannot deal with imbalanced classification either. In this paper, we proposed a model named sparse supervised representation classifier (SSRC) to solve the above-mentioned issues. The SSRC involves the class label information during the test sample representation phase to deal with the uncontrolled data sets. In SSRC, each class has the opportunity to linearly represent the test sample in its subspace, which can decrease the influences of the uncontrolled data distribution. In order to classify imbalanced data sets, a class weight learning model is proposed and added to SSRC. Each class weight is learned from its corresponding training samples. The experimental results based on the AR face database (uncontrolled) and 15 KEEL data sets (imbalanced) with an imbalanced rate ranging from 1.48 to 61.18 prove SSRC can effectively classify uncontrolled and imbalanced data sets.

Palmprint recognition has been widely applied in security and, particularly, authentication. In the past decade, various palmprint recognition methods have been proposed and achieved promising recognition performance. However, most of these methods require rich a priori knowledge and cannot adapt well to different palmprint recognition scenarios, including contact-based, contactless, and multispectral palmprint recognition. This problem limits the application and popularization of palmprint recognition. In this article, motivated by the least square regression, we propose a salient and discriminative descriptor learning method (SDDLM) for general scenario palmprint recognition. Different from the conventional palmprint feature extraction methods, the SDDLM jointly learns noise and salient information from the pixels of palmprint images, simultaneously. The learned noise enforces the projection matrix to learn salient and discriminative features from each palmprint sample. Thus, the SDDLM can be adaptive to multiscenarios. Experiments were conducted on the IITD, CASIA, GPDS, PolyU near infrared (NIR), noisy IITD, and noisy GPDS palmprint databases, and palm vein and dorsal hand vein databases. It can be seen from the experimental results that the proposed SDDLM consistently outperformed the classical palmprint recognition methods and state-of-the-art methods for palmprint recognition.

To construct small mobile networks without performance loss and address the over-fitting issues caused by the less abundant training datasets, this paper proposes a novel super sparse convolutional (SSC) kernel, and its corresponding network is called SSC-Net. In a SSC kernel, every spatial kernel has only one non-zero parameter and these non-zero spatial positions are all different. The SSC kernel can effectively select the pixels from the feature maps according to its non-zero positions and perform on them. Therefore, SSC can preserve the general characteristics of the geometric and the channels’ differences, resulting in preserving the quality of the retrieved features and meeting the general accuracy requirements. Furthermore, SSC can be entirely implemented by the “shift” and “group point-wise” convolutional operations without any spatial kernels (e.g., “3×3”). Therefore, SSC is the first method to remove the parameters’ redundancy from the both spatial extent and the channel extent, leading to largely decreasing the parameters and Flops as well as further reducing the img2col and col2img operations implemented by the low leveled libraries. Meanwhile, SSC-Net can improve the sparsity and overcome the over-fitting more effectively than the other mobile networks. Comparative experiments were performed on the less abundant CIFAR and low resolution ImageNet datasets. The results showed that the SSC-Nets can significantly decrease the parameters and the computational Flops without any performance losses. Additionally, it can also improve the ability of addressing the over-fitting problem on the more challenging less abundant datasets.

Multi-view clustering aims to partition data collected from diverse sources based on the assumption that all views are complete. However, such prior assumption is hardly satisfied in many real-world applications, resulting in the incomplete multi-view learning problem. The existing attempts on this problem still have the following limitations: 1) the underlying semantic information of the missing views is commonly ignored; 2) The local structure of data is not well explored; 3) The importance of different views is not effectively evaluated. To address these issues, this paper proposes a Unified Embedding Alignment Framework (UEAF) for robust incomplete multi-view clustering. In particular, a locality-preserved reconstruction term is introduced to infer the missing views such that all views can be naturally aligned. A consensus graph is adaptively learned and embedded via the reverse graph regularization to guarantee the common local structure of multiple views and in turn can further align the incomplete views and inferred views. Moreover, an adaptive weighting strategy is designed to capture the importance of different views. Extensive experimental results show that the proposed method can significantly improve the clustering performance in comparison with some state-of-the-art methods.

Direction information has been intensively investigated for Finger-Knuckle-Print (FKP) recognition. However, most existing direction-based KFP recognition methods are handcrafted, which are heuristic and require too much prior knowledge to engineer them. In this paper, we propose a discriminative direction binary feature learning (DDBFL) method for FKP recognition. We first propose a direction convolution difference vector (DCDV) to better describe the direction information of FKP images. Then, we learn a feature projection to convert the DCDV into binary codes, which are compact for the intra-class samples and more separable for the inter-class samples. Finally, we concatenate the block-wise histograms of the DDBFL codes to form the final descriptor for FKP recognition. Experimental results on the baseline PolyU FKP database demonstrate the competitive performance of the proposed method.

Hyperspectral dorsal hand vein image analysis for biometrics is a relatively new technology with great potential. Compared to traditional dorsal hand biometrics that use only one spectral band to capture and analyse the veins, hyperspectral imaging allows additional information to be included. Given the difficulties of processing hyperspectral dorsal hand images, including uneven illuminations, a noisy background, translation, and deformation, this study proposes a robust and adaptive region of interest (ROI) extraction algorithm. First, a novel knuckle pinky invariant point is located. Next, based on this invariant point a key line on the dorsal hand representing one side of the squared ROI is found. The remaining three sides of the ROI can then be formed. To evaluate the proposed method, both identification and verification experiments were conducted on a large hyperspectral dorsal hand vein database. The experimental results showed that the proposed method outperformed other dorsal hand ROI extraction algorithms for each hyperspectral band. Having performed well in both experiments, the groundwork has been laid to further analyse the extracted ROI in terms of feature extraction.

Multiview learning methods often achieve improvement compared with single-view-based approaches in many applications. Due to the powerful nonlinear ability and probabilistic perspective of Gaussian process (GP), some GP-based multiview efforts were presented. However, most of these methods make a strong assumption on the kernel function (e.g., radial basis function), which limits the capacity of the real data modeling. In order to address this issue, in this paper, we propose a novel multiview approach by combining a multikernel and GP latent variable model. Instead of designing a deterministic kernel function, multiple kernel functions are established to automatically adapt various types of data. Considering a simple way of obtaining latent variables at the testing stage, a projection from the observed space to the latent space as a back constraint has also been simultaneously introduced into the proposed method. Additionally, different from some existing methods which apply the classifiers off-line, a hinge loss is embedded into the model to jointly learn the classification hyperplane, encouraging the latent variables belonging to the different classes to be separated. An efficient algorithm based on the gradient decent technique is constructed to optimize our method. Finally, we apply the proposed approach to three real-world datasets and the associated results demonstrate the effectiveness and superiority of our model compared with other state-of-the-art methods.

Palmprint directions have been proved to be one of the most effective features for palmprint recognition. However, most existing direction-based palmprint descriptors are hand-craft designed and require strong prior knowledge. In this paper, we propose a discriminant direction binary code (DDBC) learning method for palmprint recognition. Specifically, for each palmprint image, we first calculate the convolutions of the direction-based templates and palmprint and form the informative convolution difference vectors by computing the convolution difference between the neighboring directions. Then, we propose a simple yet effective model to learn feature mapping functions that can project these convolution difference vectors into DDBCs. For all training samples: (1) the variance of the learned binary codes is maximized; (2) the intra-class distance of the binary codes is minimized; and (3) the inter-class distance of the binary codes is maximized. Finally, we cluster the block-wise histograms of DDBC forming the discriminant direction binary palmprint descriptor for palmprint recognition. The experimental results on four challenging contactless palmprint databases clearly demonstrate the effectiveness of the proposed method.

Recently, there is a rapidly increasing attraction for the efficient recovery of low-rank matrix in computer vision and machine learning. The popular convex solution of rank minimization is nuclear norm-based minimization (NNM), which usually leads to a biased solution since NNM tends to overshrink the rank components and treats each rank component equally. To address this issue, some nonconvex nonsmooth rank (NNR) relaxations have been exploited widely. Different from these convex and nonconvex rank substitutes, this paper first introduces a general and flexible rank relaxation function named weighted NNR relaxation function, which is actually derived from the initial double NNR (DNNR) relaxations, i.e., DNNR relaxation function acts on the nonconvex singular values function (SVF). An iteratively reweighted SVF optimization algorithm with continuation technology through computing the supergradient values to define the weighting vector is devised to solve the DNNR minimization problem, and the closed-form solution of the subproblem can be efficiently obtained by a general proximal operator, in which each element of the desired weighting vector usually satisfies the nondecreasing order. We next prove that the objective function values decrease monotonically, and any limit point of the generated subsequence is a critical point. Combining the Kurdyka-Łojasiewicz property with some milder assumptions, we further give its global convergence guarantee. As an application in the matrix completion problem, experimental results on both synthetic data and real-world data can show that our methods are competitive with several state-of-the-art convex and nonconvex matrix completion methods.

Some recent works in classification show that the data obtained from various views with different sensors for an object contributes to achieving a remarkable performance. Actually, in many real-world applications, each view often contains multiple features, which means that this type of data has a hierarchical structure, while most of existing works do not take these features with multi-layer structure into consideration simultaneously. In this paper, a probabilistic hierarchical model is proposed to address this issue and applied for classification. In our model, a latent variable is first learned to fuse the multiple features obtained from a same view, sensor or modality. Particularly, mapping matrices corresponding to a certain view are estimated to project the latent variable from a shared space to the multiple observations. Since this method is designed for the supervised purpose, we assume that the latent variables associated with different views are influenced by their ground-truth label. In order to effectively solve the proposed method, the Expectation-Maximization (EM) algorithm is applied to estimate the parameters and latent variables. Experimental results on the extensive synthetic and two real-world datasets substantiate the effectiveness and superiority of our approach as compared with state-of-the-art.

Deep convolutional neural networks provide a powerful feature learning capability for image classification. The deep image features can be utilized to deal with many image understanding tasks like image classification and object recognition. However, the robustness obtained in one dataset can be hardly reproduced in the other domain, which leads to inefficient models far from state-of-the-art. We propose a deep collaborative weight-based classification (DeepCWC) method to resolve this problem, by providing a novel option to fully take advantage of deep features in classic machine learning. It firstly performs the $l_2$-norm based collaborative representation on the original images, as well as the deep features extracted by deep CNN models. Then, two distance vectors, obtained based on the pair of linear representations, are fused together via a novel collaborative weight. This collaborative weight enables deep and classic representations to weigh each other. We observed the complementarity between two representations in a series of experiments on 10 facial and object datasets. The proposed DeepCWC produces very promising classification results, and outperforms many other benchmark methods, especially the ones claimed for Fashion-MNIST. The code is going to be published in our public repository\footnote{https://github.com/zengsn/research}.

In order to address the overfitting problem caused by the small or simple training datasets and the large model’s size in Convolutional Neural Networks (CNNs), a novel Auto Adaptive Regularization (AAR) method is proposed in this paper. The relevant networks can be called AAR-CNNs. AAR is the first method using the “abstraction extent” (predicted by AE net) and a tiny learnable module (SE net) to auto adaptively predict more accurate and individualized regularization information. The AAR module can be directly inserted into every stage of any popular networks and trained end to end to improve the networks’ flexibility. This method can not only regularize the network at both the forward and the backward processes in the training phase, but also regularize the network on a more refined level (channel or pixel level) depending on the abstraction extent’s form. Comparative experiments are performed on low resolution ImageNet, CIFAR and SVHN datasets. Experimental results show that the AAR-CNNs can achieve state-of-the-art performances on these datasets.

Multi-view learning has shown its powerful potential in many applications and achieved outstanding performances compared with the single-view based methods. In this paper, we propose a novel multi-view learning model based on the Gaussian Process Latent Variable Model (GPLVM) to learn a shared latent variable in the manifold space with a linear and gaussian process prior based back projection. Different from existing GPLVM methods which only consider a mapping from the latent space to the observed space, the proposed method simultaneously takes a back projection from the observation to the latent variable into account. Concretely, due to the various dimensions of different views, a projection for each view is first learned to linearly map its observation to a subspace. The gaussian process prior is then imposed on another transformation to non-linearly and efficiently map the learned subspace to a shared manifold space. In order to apply the proposed approach to the classification, a discriminative regularization is also embedded to exploit the label information. Experimental results on three real-world databases substantiate the effectiveness and superiority of the proposed approach as compared with several state-of-the-art approaches.

Multiview learning reveals the latent correlation among different modalities and utilizes the complementary information to achieve a better performance in many applications. In this paper, we propose a novel multiview learning model based on the Gaussian process latent variable model (GPLVM) to learn a set of nonlinear and nonparametric mapping functions and obtain a shared latent variable in the manifold space. Different from the previous work on the GPLVM, the proposed shared autoencoder Gaussian process (SAGP) latent variable model assumes that there is an additional mapping from the observed data to the shared manifold space. Due to the introduction of the autoencoder framework, both nonlinear projections from and to the observation are considered simultaneously. Additionally, instead of fully connecting used in the conventional autoencoder, the SAGP achieves the mappings utilizing the GP, which remarkably reduces the number of estimated parameters and avoids the phenomenon of overfitting. To make the proposed method adaptive for classification, a discriminative regularization is embedded into the proposed method. In the optimization process, an efficient algorithm based on the alternating direction method and gradient decent techniques is designed to solve the encoder and decoder parts alternatively. Experimental results on three real-world data sets substantiate the effectiveness and superiority of the proposed approach as compared with the state of the art.

Sparse Bayesian learning has emerged as a powerful tool to tackle various image classification tasks. The existing sparse Bayesian models usually use independent Gaussian distribution as the prior knowledge for the noise. However, this assumption often contradicts to the practical observations in which the noise is long tail and pixels containing noise are spatially correlated. To handle the practical noise, this paper proposes to partition the noise image into several 2-D groups and adopt the long-tail distribution, i.e., the scale mixture of the matrix Gaussian distribution, to model each group to capture the intragroup correlation of the noise. Under the nonparametric Bayesian estimation, the low-rank-induced prior and the matrix Gamma distribution prior are imposed on the covariance matrix of each group, respectively, to induce two Bayesian correlated group regression (BCGR) methods. Moreover, the proposed methods are extended to the case with unknown group structure. Our BCGR method provides an effective way to automatically fit the noise distribution and integrates the long-tail attribute and structure information of the practical noise into model. Therefore, the estimated coefficients are better for reconstructing the desired data. We apply BCGR to address image classification task and utilize the learned covariance matrices to construct a grouped Mahalanobis distance to measure the reconstruction residual of each class in the design of a classifier. Experimental results demonstrate the effectiveness of our new BCGR model.

A robust semi-supervised concept factorization (RSSCF) method is proposed in this paper, which not only makes good use of the available label information, but also addresses noise and extracts meaningful information simultaneously. In the proposed method, a constraint matrix is embedded into the basic concept factorization model to guarantee data with the same label share the same new representation. We utilize L2,1-norm on both loss function and regularization, thus this new model is not sensitive to outliers and the L2,1-norm regularization helps select useful information with joint sparsity. An efficient and elegant iterative updating scheme is also introduced with convergence and correctness analysis. Simulations are given to illustrate the effectiveness of our proposed method.

Direction information serves as one of the most important features for palmprint recognition. In the past decade, many effective direction representation (DR)-based methods have been proposed and achieved promising recognition performance. However, due to an incomplete understanding for DR, these methods only extract DR in one direction level and one scale. Hence, they did not fully utilize all potentials of DR. In addition, most researchers only focused on the DR extraction in spatial coding domain, and rarely considered the methods in frequency domain. In this paper, we propose a general framework for DR-based method named complete DR (CDR), which reveals DR by a comprehensive and complete way. Different from traditional methods, CDR emphasizes the use of direction information with strategies of multi-scale, multi-direction level, multi-region, as well as feature selection or learning. This way, CDR subsumes previous methods as special cases. Moreover, thanks to its new insight, CDR can guide the design of new DR-based methods toward better performance. Motived this way, we propose a novel palmprint recognition algorithm in frequency domain. First, we extract CDR using multi-scale modified finite radon transformation. Then, an effective correlation filter, namely, band-limited phase-only correlation, is explored for pattern matching. To remove feature redundancy, the sequential forward selection method is used to select a small number of CDR images. Finally, the matching scores obtained from different selected features are integrated using score-level-fusion. Experiments demonstrate that our method can achieve better recognition accuracy than the other state-of-the-art methods. More importantly, it has fast matching speed, making it quite suitable for the large-scale identification applications.

Background modeling has played an important role in detecting the foreground for video analysis. In this paper, we presented a novel background modeling method for foreground segmentation. The innovations of the proposed method lie in the joint usage of the pixel-based adaptive segmentation method and the background updating strategy, which is performed in both pixel and object levels. Current pixel-based adaptive segmentation method only updates the background at the pixel level and does not take into account the physical changes of the object, which may result in a series of problems in foreground detection, e.g., a static or low-speed object is updated too fast or merely a partial foreground region is properly detected. To avoid these deficiencies, we used a counter to place the foreground pixels into two categories (illumination and object). The proposed method extracted a correct foreground object by controlling the updating time of the pixels belonging to an object or an illumination region respectively. Extensive experiments showed that our method is more competitive than the state-of-the-art foreground detection methods, particularly in the intermittent object motion scenario. Moreover, we also analyzed the efficiency of our method in different situations to show that the proposed method is available for real-time applications.

Orientation features are successfully used in coding-based palmprint recognition methods. In this paper, we propose a discriminative neighboring direction indicator to represent the orientation feature of the palmprint. The neighboring direction indicator feature not only represents the most dominant orientation feature of the palmprint, but also better describes the orientation feature of those points which have double dominant orientations. In addition, the neighboring direction indicator shows good robustness to noise and rotation. Using the neighboring direction indicator, we propose a novel palmprint recognition method. Extensive experiments conducted on three types of palmprint databases demonstrate that the proposed method gives better performance than the existing state-of-the-art orientation-based methods. By using the proposed method, the equal error rate is improved by about 10% for palmprint verification, and the average error rate is reduced by 2.7-14% for palmprint identification with a single training sample.

Structured sparsity, as an extension of standard sparsity, has shown the outstanding performance when dealing with some highly correlated variables in computer vision and pattern recognition. However, the traditional mixed (L1, L2) or (L1, L∞) group norm becomes weak in characterizing the internal structure of each group since they cannot alleviate the correla-tions between variables. Recently, nuclear norm has been vali-dated to be useful for depicting a spatially structured matrix variable. It considers the global structure of the matrix variable but overlooks the local structure. To combine the advantages of structured sparsity and nuclear norm, this paper presents a tree-structured nuclear norm approximation (TSNA) model as-suming that the representation residual with tree-structured prior is a random matrix variable and follows a dependent matrix dis-tribution. The Extended Alternating Direction Method of Multi-pliers (EADMM) is utilized to solve the proposed model. An effi-cient bound condition based on the extended restricted isometry constants is provided to show the exact recovery of the proposed model under the given noisy case. In addition, TSNA is connected with some newest methods such as sparse representation based classifier (SRC), nuclear-L1 norm joint regression (NL1R) and nuclear norm based matrix regression (NMR), which can be re-garded as the special cases of TSNA. Experiments with face re-construction and recognition demonstrate the benefits of TSNA over other approaches.

With the rapid development of wearable sensors and pervasive computing technologies including smartphones, acceleration-based human activity recognition is receiving increased attention for medical research applications. Motivated by the “weightlessness” feature, here we apply a bidirectional feature during the feature extraction phase of activity recognition; however, since the bidirectional feature has two components, they cannot simply be concatenated into a long vector, but can be naturally treated as a second-order tensor. Therefore, we propose a new tensor-based feature selection method termed tensor manifold discriminant projections (TMDP). TMDP simultaneously considers: 1) applying an optimization criterion that can directly process the tensor spectral analysis problem, thereby decreasing the computational cost compared to traditional tensor-based feature selection methods; 2) extracting local rank information by finding a tensor subspace that preserves the rank order information of the within-class input samples; and 3) extracting discriminant information by maximizing the sum of distances between every sample and their interclass sample mean. Experiments on the naturalistic mobile devices-based human activity 2.0 dataset are performed to demonstrate the effectiveness and robustness of TMDP.

In this paper, an in-depth analysis on the statistical distribution characteristics of human tongue color that aims to propose a mathematically described tongue color space for diagnostic feature extraction is presented. Three characteristics of tongue color space, i.e., tongue color gamut that defines the range of colors, color centers of 12 tongue color categories, and color distribution of typical image features in the tongue color gamut, are elaborately investigated in this paper. Based on a large database, which contains over 9000 tongue images collected by a specially designed noncontact colorimetric imaging system using a digital camera, the tongue color gamut is established in the CIE chromaticity diagram by an innovatively proposed color gamut boundary descriptor using one-class SVM algorithm. Thereafter, centers of 12 tongue color categories are defined accordingly. Furthermore, color distributions of several typical tongue features, such as red points and petechial points, are obtained to build a relationship between the tongue color space and color distributions of various tongue features. With the obtained tongue color space, a new color feature extraction method is proposed for diagnostic classification purposes, with experimental results validating its effectiveness.

Three-dimensional (3-D) palm print has proved to be a significant biometrics for personal authentication. Three-dimensional palm prints are harder to counterfeit than 2-D palm prints and more robust to variations in illumination and serious scrabbling on the palm surface. Previous work on 3-D palm-print recognition has concentrated on local features such as texture and lines. In this paper, we propose three novel global features of 3-D palm prints which describe shape information and can be used for coarse matching and indexing to improve the efficiency of palm-print recognition, particularly in very large databases. The three proposed shape features are maximum depth of palm center, horizontal cross-sectional area of different levels, and radial line length from the centroid to the boundary of 3-D palm-print horizontal cross section of different levels. We treat these features as a column vector and use orthogonal linear discriminant analysis to reduce their dimensionality. We then adopt two schemes: 1) coarse-level matching and 2) ranking support vector machine to improve the efficiency of palm-print recognition. We conducted a series of 3-D palm-print recognition experiments using an established 3-D palm-print database, and the results demonstrate that the proposed method can greatly reduce penetration rates.

Image alignment is an important step in various biometric authentication applications such as palmprint recognition. Most of the existing palmprint alignment methods make use of some key points between fingers or in palm boundary to establish the local coordinate system for region of interest (ROI) extraction. The ROI is consequently used for feature extraction and matching. Such alignment methods usually yield a coarse alignment of the palmprint images, while many missed and false matches are actually caused by inaccurate image alignments. To improve the palmprint verification accuracy, in this paper, we present an efficient palmprint alignment refinement method. After extracting the principal lines from the palmprint image, we apply the iterative closest point method to them to estimate the translation and rotation parameters between two images. The estimated parameters are then used to refine the alignment of palmprint feature maps for a more accurate palmprint matching. The experimental results show that the proposed method greatly improves the palmprint recognition accuracy and it works in real time.

Diabetic retinopathy (DR) is a common complication of diabetes that damages the retina and leads to sight loss if treated late. In its earliest stage, DR can be diagnosed by micro aneurysm (MA). Although some algorithms have been developed, the accurate detection of MA in color retinal images is still a challenging problem. In this paper we propose a new method to detect MA based on Sparse Representation Classifier (SRC). We first roughly locate MA candidates by using multi-scale Gaussian correlation filtering, and then classify these candidates with SRC. Particularly, two dictionaries, one for MA and one for non-MA, are learned from example MA and non-MA structures, and are used in the SRC process. Experimental results on the ROC database show that the proposed method can well distinguish MA from non-MA objects.

The detection of microaneurysms in digital color fundus photographs is a critical first step in automated screening for diabetic retinopathy (DR), a common complication of diabetes. To accomplish this detection numerous methods have been published in the past but none of these was compared with each other on the same data. In this work we present the results of the first international microaneurysm detection competition, organized in the context of the Retinopathy Online Challenge (ROC), a multiyear online competition for various aspects of DR detection. For this competition, we compare the results of five different methods, produced by five different teams of researchers on the same set of data. The evaluation was performed in a uniform manner using an algorithm presented in this work. The set of data used for the competition consisted of 50 training images with available reference standard and 50 test images where the reference standard was withheld by the organizers (M. Niemeijer, B. van Ginneken, and M. D. Abràmoff). The results obtained on the test data was submitted through a website after which standardized evaluation software was used to determine the performance of each of the methods. A human expert detected microaneurysms in the test set to allow comparison with the performance of the automatic methods. The overall results show that microaneurysm detection is a challenging task for both the automatic methods as well as the human expert. There is room for improvement as the best performing system does not reach the performance of the human expert. The data associated with the ROC microaneurysm detection competition will remain publicly available and the website will continue accepting submissions.