Change detection (CD) is a critical task in remote sensing (RS) image analysis. Recent deep learning networks for CD focus on identifying changes after mining the features of bi-temporal images separately. However, light differences in bi-temporal images lead to the networks extracting different features from the identical objects, which may cause pseudo-changes. From the Fourier transform perspective, an image can be decomposed into amplitude and phase, where the amplitude contains most of the light information and the phase is relevant to structure information. Therefore, amplitude-invariant features of the identical objects in different light conditions are roughly the same, which are pivotal to identify real and fake changes between bi-temporal images. In this article, we propose a capturing amplitude-invariant features network (CAIFNet), which reduces dependence on amplitude and captures diverse amplitude-invariant features. Firstly, we build an amplitude pre-processing module (APM) to provide diverse processed images by randomly mixing the amplitudes of the input images with the amplitudes of the reference images and keeping the phases of the input images constant. Secondly, a quadruple-stream encoder is proposed to capture amplitude-invariant features. Specifically, it is forced to learn and capture amplitude-invariant local details and amplitude-invariant contextual semantics based on the diverse processed images under CD task-oriented constraint, both reciprocate each other to become more accurate by local attention guide strategy (LGS). Moreover, a difference enhancement module (DEM) is designed in the quadruple-stream encoder to enhance the difference features. Thirdly, a bi-stream decoder decodes the captured amplitude-invariant features in main and boundary difference perspectives, enhancing main body and boundary details of the objects in the change maps, respectively. Finally, a spatial embedded module (SEM) allows the main and boundary difference features to be embedded into each other, obtaining more complete change maps. On three remote sensing change detection (RSCD) datasets, CAIFNet achieves better transferability and results compared to state-of-the-art methods. The source code is available at https://github.com/yihui1230/CAIFNet.

Remote sensing (RS) scene classification is a crucial component in the analysis of Earth observation data, aiding in a deeper understanding and monitoring of our dynamic planet. Its applications extend across various fields, including land management, urban analysis, and environmental monitoring. The complex semantic information in RS scene images and the relationships between different scene categories present significant challenges to improving scene classification tasks. Unlike previous methods that only focus on network structure or feature encoding, our approach emphasizes the association of scene categories, integrating feature learning and knowledge transfer together to enhance the analysis of scenes at a higher semantic level. To this end, we propose an RS scene classification scheme based on pseudo-scene category-relation reasoning and orthogonal feature (OF) learning modules, capturing the inherent semantic connections among diverse scene classes. Additionally, we introduce cascaded attention (CA) and selected separation modules to strategically optimize the network, targeting challenging classes with high feature similarities. Knowledge is then distilled across different branches, guiding to enhance the model’s robustness and prediction accuracy. Experiments are conducted on three challenging RS scene datasets of AID30, UCMerced21, and NWPU-RESISC45 to validate the effectiveness of the learned pseudo-category relationships. The results demonstrate that the proposed framework outperforms existing hierarchical approaches in leveraging the hierarchical structure of RS scene images.

Semantic change detection (SCD) consists of two separate but not independent parts, i.e., binary change detection (BCD) and semantic segmentation (SS), which aims to simultaneously locate changed areas and provide their semantic categories in bi-temporal remote sensing (RS) images. Although the existing SCD methods have achieved excellent performance, they still have two challenges: 1) Insufficient collaboration between BCD and SS: They can not efficiently leverage semantic information provided by SS to further improve performance of BCD and 2) Existing conflict between BCD and SS: They usually share input between SS and BCD branches to benefit from joint optimization. However, the shared input will be constrained by task-orientations of BCD and SS, which may produce conflict. Specifically, the task-orientation of BCD aims to align the distribution of bi-temporal domains, which inevitably results in the constraint of the shared input mining domain-specific features and further causes degradation of SS performance. Similarly, the task-orientation of SS also compromises domain alignment process of BCD as the shared input mines domain-specific features guided by the task-orientation of SS for improving its performance. Therefore, we propose enhancing collaboration-mitigating conflict network (EC-MCNet). Firstly, a semantic enhancement module (SEM) is designed to enhance inter-class variation and intra-class similarity of semantic features, which are beneficial for SS and BCD. Secondly, we no longer share input but share semantic contents between SS and BCD branches for mitigating the conflict. Specifically, we build a domain style removal module (DSRM), which ensures the input of BCD branch is removed from domain-specific styles and has the same semantic contents to the inputs of SS branches. In this way, the conflict between BCD and SS can be mitigated and the advantage of joint optimization can be preserved. Thirdly, we design a difference enhancement module (DEM) to enhance collaboration between SS and BCD, which leverages not only the attention of difference features but also the semantic similarity between bi-temporal features to enhance and identify changed areas of bi-temporal features. Extensive experimental results validate that our method outperforms state-of-the-art (SOTA) performances on two benchmark datasets for the SCD. The source code is available at https://github.com/yihui1230/ECMCNet.

Remote sensing change detection (RSCD) aims to accurately identify changed objects within the same region over time, facilitating the interpretation of Earth’s surface dynamics. However, RSCD faces two fundamental challenges: the variation in partial representations of identical objects over time and the similarity in certain properties of different objects, both of which significantly hinder performance. Drawing inspiration from the human difference analysis strategy, which detects changes by empirically selecting and integrating inter-object comparisons across various properties (such as color, boundary, and texture), a difference analyzer (DA) is introduced. The DA identifies changes based on property-based features rather than holistic spatial features, which enables adaptive suppression of inconsistent properties of the identical object over time to eliminate pseudo changes, as well as the suppression of similar properties of different objects to accurately identify the changes, thus effectively overcoming the aforementioned challenges. Specifically, the bi-temporal property descriptors are derived using a newly proposed sliding discrete cosine transform (SDCT), where distinct channels represent various object properties. Subsequently, a cross-descriptor attention (CDA) is developed that selectively activates discriminative properties (such as building boundaries) while suppressing interfering ones (such as illumination-induced color variations), enabling effective selected property integration and comparisons. Next, the DA constructs property difference descriptors that represent the inter-object comparisons. To embed DA within an end-to-end network, a learning from human insights for change detection network (LHICD-Net) is proposed, which replicates the human RSCD process through perception, analysis, refinement, and prediction stages. To match human perception in diverse bands, a high-frequency residual structure (HRS) that can preserve high-frequency information during down-sampling is incorporated in the encoder. A refinement module (RM) is also designed to enhance spatial details by extracting fine-grained features from diverse receptive fields, corresponding to the human refinement stage. Importantly, we reveal that the working mechanism of the DA aligns with the human difference analysis strategy and validate the superiority of our property descriptor generation method. Extensive experiments confirm that LHICD-Net outperforms state-of-the-art methods across three widely used RSCD datasets. The source code is available at https://github.com/yihui1230/LHICDNet.

With the development of deep learning, significant progress has been made in change detection (CD) methods for remote sensing (RS) images. However, many convolutional neural network (CNN)-based methods are constrained in capturing long-range dependencies due to the limitations of the receptive field. Transformers rely on self-attention mechanisms to effectively achieve global information modeling and are widely used in CD tasks. Nevertheless, transformer-based CD methods still suffer from issues such as pseudochanges and incomplete edges due to the lack of position and temporal correlations in bitemporal RS images. To deal with this issue, we propose a position-temporal awareness transformer (PT-Former), which models position and temporal relations in bitemporal images. Specifically, a Siamese network attached to a position-aware embedding module (PEM) serves as a feature encoder to extract the features of changed areas. Then, a temporal difference perception module (TDPM) is designed to capture the cross-temporal shift and enhance the difference perception ability during cross-temporal interaction. Meanwhile, the contextual information of the ground object is aggregated by the fusion block, and the spatial relation is reconstructed under the guidance of bitemporal features. The experimental results validate the superiority of PT-Former on three benchmark datasets, including the season-varying CD (SVCD) dataset, the learning vision and RS laboratory building CD (LEVIR-CD) dataset, and the WHU-CD dataset confirming the potential of PT-Former for CD tasks in RS images. The code will be available athttps://github.com/liuyk29/PT-Former.

Image reconstruction-based methods with autoencoder have been widely used for unsupervised anomaly detection. By training the reconstruction on normal samples, autoencoder is supposed to produce higher reconstruction error for anomalous samples, which is used as an indicator for detecting anomalies. However, since autoencoder adopts the bottleneck layer to reconstruct data, it is hard to control its generalization capability. When the generalization capability is high, anomalous features can be confused with normal features, resulting in accurate reconstruction of anomalous regions as well. In this article, we propose a dual-constraint autoencoder to alleviate the problem of feature confusion through the dual constraint of adversarial learning and global memory bank. Given a sample pair consisting of a normal sample and a synthetic anomaly sample generated from it as input, the proposed autoencoder first forces the encoding of the synthetic anomaly image to be close to the normal encoding by adversarial learning. Then, the synthetic anomaly encoding is rerepresented with items in memory bank, which records normal features that facilitate image restoration, and the new feature is fed to the decoder for image inpainting. Under such constraints, the generalization capability of the autoencoder is suppressed and will not reconstruct the anomaly region well. After that, we feed the synthetic anomaly image together with the repaired image into the proposed adaptive weighted similarity spatial attention-based U-Net and produce an accurate anomaly map with the help of our proposed adaptive weighted similarity spatial attention. Results on the a comprehensive real-world dataset for unsupervised anomaly detection (MVTec AD) dataset and the beanTech anomaly detection dataset dataset show that our framework achieves state-of-the-art performance.

Pan-sharpening aims to increase the spatial resolution of the low-resolution multispectral (LrMS) image with the guidance of the corresponding panchromatic (PAN) image. Although deep learning (DL)-based pan-sharpening methods have achieved promising performance, most of them have a two-fold deficiency. For one thing, the universally adopted black box principle limits the model interpretability. For another thing, existing DL-based methods fail to efficiently capture local and global dependencies at the same time, inevitably limiting the overall performance. To address these mentioned issues, we first formulate the degradation process of the high-resolution multispectral (HrMS) image as a unified variational optimization problem, and alternately solve its data and prior subproblems by the designed iterative proximal gradient descent (PGD) algorithm. Moreover, we customize a Local-Global Transformer (LGT) to simultaneously model local and global dependencies, and further formulate an LGT-based prior module for image denoising. Besides the prior module, we also design a lightweight data module. Finally, by serially integrating the data and prior modules in each iterative stage, we unfold the iterative algorithm into a stage-wise unfolding network, Local-Global Transformer Enhanced Unfolding Network (LGTEUN), for the interpretable MS pan-sharpening. Comprehensive experimental results on three satellite data sets demonstrate the effectiveness and efficiency of LGTEUN compared with state-of-the-art (SOTA) methods. The source code is available at https://github.com/lms-07/LGTEUN.

To mine the spectral-spatial information of target pixel in hyperspectral image classification (HSIC), convolutional neural network (CNN)-based models widely adopt patch-based input pattern, where a patch represents its central pixel and the neighbor pixels play auxiliary roles in the classification process. However, compared to the central pixel, its neighbor pixels often have different contributions for classification. Although many existing patch-based CNNs could adaptively emphasize the spatial neighbor information, most of them ignore the latent relationship between the center pixel and its neighbor pixels. Moreover, efficient spectral-spatial feature extraction has been a difficult yet vital topic for HSIC. To address the mentioned problems, a central vector oriented self-similarity network (CVSSN) is proposed for HSIC. Specifically, based on two similarity measures, we firstly design an adaptive weight addition based spectral vector self-similarity module (AWA-SVSS) in input space and a Euclidean distance based feature vector self-similarity module (ED-FVSS) in feature space to fully mine the central vector oriented spatial relationships. Besides, a spectral-spatial information fusion module (SSIF) is formulated as a new pattern to fuse the central 1D spectral vector and the corresponding 3D patch for efficient spectral-spatial feature learning of the subsequent modules. Moreover, we implement a channel spatial separation convolution module (CSS-Conv) and a scale information complementary convolution module (SIC-Conv) for efficient spectral-spatial feature learning. Extensive experimental results on four popular HSI data sets demonstrate the effectiveness and efficiency of the proposed method compared with other state-of-the-art methods. The source code is available athttps://github.com/lms-07/CVSSN

For the abundant spectral and spatial information recorded in hyperspectral images (HSIs), fully exploring spectral-spatial relationships has attracted widespread attention in hyperspectral image classification (HSIC) community. However, there are still some intractable obstructs. For one thing, in the patch based processing pattern, some spatial neighbor pixels are often inconsistent with the central pixel in land-cover class. For another thing, linear and nonlinear correlations between different spectral bands are vital yet tough for representing and excavating. To overcome these mentioned issues, an adaptive mask sampling and manifold to Euclidean subspace learning (AMS-M2ESL) framework is proposed for HSIC. Specifically, an adaptive mask based intra-patch sampling (AMIPS) module is firstly formulated for intra-patch sampling in an adaptive mask manner based on central spectral vector oriented spatial relationships. Then, based on distance covariance descriptor, a dual channel distance covariance representation (DC-DCR) module is proposed for modeling unified spectral-spatial feature representations and exploring spectral-spatial relationships, especially linear and nonlinear interdependence in spectral domain. Furthermore, considering that distance covariance matrix lies on the symmetric positive definite (SPD) manifold, we implement a manifold to Euclidean subspace learning (M2ESL) module respecting Riemannian geometry of SPD manifold for high-level spectral-spatial feature learning. Additionally, we introduce an approximate matrix square-root (ASQRT) layer for efficient Euclidean subspace projection. Extensive experimental results on three popular HSI data sets with limited training samples demonstrate the superior performance of the proposed method compared with other state-of-the-art methods. The source code is available at https://github.com/lms-07/AMS-M2ESL.

Semantic segmentation is an extremely challenging task in high-resolution remote sensing (HRRS) images as objects have complex spatial layouts and enormous variations in appearance. Convolutional neural networks (CNNs) have excellent ability to extract local features and have been widely applied as the feature extractor for various vision tasks. However, due to the inherent inductive bias of convolution operation, CNNs inevitably have limitations in modeling long-range dependencies. Transformer can capture global representations well, but unfortunately ignores the details of local features and has high computational and spatial complexity in processing high-resolution feature maps. In this paper, we propose a novel hybrid architecture for HRRS image segmentation, termed EMRT, to exploit the advantages of convolution operations and Transformer to enhance multi-scale representation learning. We incorporate the deformable self-attention mechanism in the Transformer to automatically adjust the receptive field, and design an encoder-decoder architecture accordingly to achieve efficient context modeling. Specifically, the CNN is constructed to extract feature representations. In the encoder, local features and global representations at different resolutions are extracted by the CNN and Transformer, respectively, and fused in an interactive manner. Moreover, a separate spatial branch is designed to extract multi-scale contextual information as queries, and global dependencies between features at different scales are efficiently established by the decoder. Extensive experiments on three public remote sensing datasets demonstrate the superiority of EMRT and indicate that the overall performance of our method outperforms state-of-the-art methods. Code is available at https://github.com/peach-xiao/EMRT.

Recently, finger vein sensors have been embedded in all kinds of electronic devices for personal identification, such as intelligent door locks and attendance machines. The embedded sensors are generally small, thus capturing only part of the finger vein. However, prior studies have focused on near-full finger vein recognition, without considering the partial finger vein image caused by the small imaging window of the finger vein sensor. This paper aims to study personal identification based on partial finger vein images, known as small-area finger vein recognition. The effect of the small-area finger vein on recognition performance is first analyzed by cutting out the local part from the near-full finger vein image to model a small-area finger vein image. Second, a small-area finger vein database is built using a commercial finger vein imaging device, in which the vein pattern from approximately one-third of one adult finger is captured. To explore more discriminative information from small-area finger vein images, we propose a locality-constrained consistent dictionary learning (LCDL) method to fuse multiple features for small-area finger vein recognition. Finally, the proposed method is evaluated on the self-built small-area finger vein database and four synthetic small-area finger vein databases. Experimental results show the promising recognition performance of the proposed method.

In recent years, Electrocardiogram (ECG) biometrics has gained extensive attention. However, most existing methods adopted offline batch learning, which means that they need to accumulate all data and retrain the model when new data comes. Therefore, it is inefficient and unpractical for them to handle the online scenario where new data may continually come. To overcome the above limitation, we propose a novel ECG biometrics framework, termed Online ECG Biomet-rics based on Hadamard Codes. Firstly, we leverage matrix factorization to learn discriminative representations for ECG signals from their base feature space. Considering to leverage the orthogonal property of the Hadamard matrix, we use it to construct Hadamard codes to represent individuals and further guide the learning of representations. Furthermore, we develop an online optimization algorithm, which is efficient and effective to investigate the incremental problem in the context of ECG biometrics. The experimental results on two benchmark datasets indicate the merits of the proposed framework over the state-of-the-art.

Electrocardiogram (ECG) biometrics has aroused extensive attention in the research field of biometric recognition. How-ever, most existing methods either only consider a single do-main (time domain or frequency domain) to extract features or extract multi-features while ignoring the specific proper-ties of each domain. In this paper, we propose a novel ECG biometrics framework termed Joint Dual-domain Matrix Factorization (JDMF). JDMF learns latent spaces for each do-main by exploring the cross-correlations between them and preserving domain-specific properties. To endow the latent spaces with more powerful representation capabilities, JDMF further makes full use of the supervised information and could automatically learn the weights of domains. The experimental results on two widely-used datasets indicate that the proposed framework can outperform state-of-the-arts.

Electrocardiogram (ECG) biometrics has been an emerging field, and off-the-person ECG biometrics capturing the ECG from fingertips is one of the new trends in this field. However, dynamic morphological variability in the same person and low signal-to-noise ratios pose great challenges for off-the-person ECG biometrics. To reduce the dynamic morphological variability, this paper introduces deep metric learning into ECG biometrics to learn intra-individual compact features. To enforce the robust of proposed method, dual-domain features extracted from both 1D signals and 2D spectrograms are integrated by low-rank fusion. Furthermore, this method dispenses with the need for noise removal and outliers discarding completely. Experiments on two off-the-person ECG benchmark databases demonstrate that the proposed method significantly outperforms the state-of-the-art methods. Additionally, ablation experiments show the effectiveness of every part of our framework.

Semantic segmentation for remote sensing is a crucial but challenging task. Many supervised semantic segmentation methods rely heavily on a large-scale pixel-wise annotated data set, but it is time-consuming and laborious to provide manual annotation. However, due to the common domain shift of remote sensing images, a direct transfer might not perform well. Therefore, many unsupervised domain adaptation methods have been proposed to solve the data distribution discrepancy in remote-sensing data sets, but these methods cannot completely utilize the features extracted in the training process. In addition, the correlations between feature map channels are crucial for the pixel-wise classification task. In this letter, a covariance-based channel attention module is proposed to capture correlations by covariance metric and weighting the feature map channels. To further improve the domain adaptation performance, we propose a three-stage unsupervised domain adaptation semantic segmentation method for remote-sensing images, we fine-tune the model which has been trained on the source domain on the target domain via self training and knowledge distillation. To test the effectiveness of the proposed method, experiments are conducted on the ISPRS 2-D Semantic Labeling data set and an urban drone data set. Our method shows a better performance advantage compared with other state-of-the-art methods.

ECG biometric recognition has received plenty of attention in biometrics area. In recent years, various classical sparse representation and dictionary learning methods have been utilized in ECG biometric recognition. However, to produce better classification results, lP-norm is used to regularize the representation coefficients, which undoubtedly brings time cost problem. To overcome this limitation, our method, namely label-guided dictionary pair learning, aims to learn a projective dictionary and reconstructed dictionary jointly, which achieves signal representation and reconstruction simultaneously. Introduction of label information with each dictionary item and Fisher-like regularization on projective dictionary enforce discriminability during the dictionary learning process. Alternating direction method of multipliers is then exploited to optimize the corresponding objective function. Extensive experiments on two databases demonstrate that our method can achieve better performance compared with state-of-the-art ECG biometric recognition methods.

Electrocardiogram (ECG) biometric recognition has recently attracted considerable attention and various promising approaches have been proposed. However, due to the real nonstationary ECG noise environment, it is still challenging to perform this technique robustly and precisely. In this paper, we propose a novel ECG biometrics framework named robuSt semanTic spacE leaRning with Local sImilarity preserviNG (STERLING) to learn a latent space where ECG signals can be robustly and discriminatively represented with semantic information and local structure being preserved. Specifically, in the proposed framework, a novel loss function is proposed to learn robust semantic representation by introducing l2,1-norm loss and making full use of the supervised information. In addition, a graph regularization is imposed to preserve the local structure information in each subject. Finally, in the learnt latent space, matching can be effectively done. The experimental results on three widely-used datasets indicate that the proposed framework can outperform the state-of-the-arts.

Devices such as smartphones and tablets have spurred interest in off-the-person electrocardiogram (ECG) biometric recognition. While the advantage of using multi-feature information for establishing identities has been widely recognized, computational sparse representation models for multi-feature biometric recognition have only recently received more attention. We propose a unified sparse representation framework which collaboratively exploits joint and specific patterns for ECG biometric recognition. In particular, unlike joint sparse representation, which only considers the consistency among sparsity patterns of multiple features, we combine the consistent and pairwise constraints, which not only learn latent discriminant representations for all features but capture the interactions between them. In addition, our framework is universal and easily adapts to other multi-feature sparse representation models by just tuning the regularization parameters. The optimization problem is solved by an efficient alternating direction method of multipliers (ADMM). Extensive experiments on two publicly available off-the-person datasets demonstrate that our method can achieve competitive or even superior performance compared to state-of-the-art ECG biometric recognition methods.

Vein pattern-based methods have powerfully promoted the performance of finger vein recognition. However, it is not easy to precisely extract vein patterns from images, especially from low-quality images, and the non-vein area have been proved to be helpful for recognition. This paper proposes to use low-rank representation to extract as much noiseless discriminative information as possible from finger vein images. However, image deformation and image quality variations weaken the correlation of genuine images, and therefore damage the low-rank linear representation. To further deal with this problem, the class labels of training images and the local geometric structure between testing images and training images, reflected by sparse reconstruction errors of testing images, are used as constraints of low-rank coefficients. In particular, vein backbone decomposition based sparse representation is proposed to fast compute the deformation-robust reconstruction errors of each testing image. The reconstruction errors on sub-backbones of one training image are summed and modified as the constraint of the low-rank coefficient on this training image. We evaluate the proposed method on three widely used finger vein databases, and experimental results show that the proposed method performs well in finger vein recognition.

In second paragraph of the footnote on the first page of[1], the institution information of Lu Yang and Xiaoming Xi is inaccurate. The correct institution name is “School of Computer Science and Technology, Shandong University of Finance and Economics.” So this paragraph should be corrected as:

This paper presents a novel deep learning framework for the electrocardiogram (ECG) heartbeat classification. Although there have been some studies with excellent overall accuracy, these studies have not been very accurate in the diagnosis of arrhythmia classes especially such as supraventricular ectopic beat (SVEB) and ventricular ectopic beat (VEB). In our work, we propose a Multi-Scale and Attention based Res Net for heartbeat classification in intra-patient and inter-patient paradigms respectively. Firstly, we extract shallow features from a convolutional layer. Secondly, the shallow features are sent into three branches with different convolution kernels in order to combine receptive fields of different sizes. Finally, fully connected layers are used to classify the heartbeat. Besides, we design a new attention mechanism based on the characteristics of heartbeat data. At last, extensive experiments on benchmark dataset demonstrate the effectiveness of our proposed model.

Vein pattern-based methods powerfully boost the recognition accuracy of finger veins, but real-time recognition cannot be guaranteed, especially in large-scale applications. Moreover, previous studies focused on either the matching task to enhance the accuracy or the indexing task to improve the efficiency. This paper proposes a finger vein code indexing method and combines it with a finger vein pattern matching method into an integration framework for improving both accuracy and efficiency. With the extracted vein patterns, the direction of each vein segment is detected and represented by the elliptical direction map as a feature for indexing, which will be encoded into a binary code by the angle K-means. The similarity between vein direction codes is measured by the grouped hamming distance in indexing, and further weighted by the overlap degree of the corresponding vein patterns to return the candidates for the probe. In addition, based on the above distance measurement, only vein segments with the same direction code are considered in following probe-to-candidate matching. Experimental results indicate that our indexing method outperforms the state-of-the-art methods and has competitive potential in performing the matching task. The results also indicate that the integration framework highly improves the identification efficiency with a slight improvement on the accuracy.

Retinal layer segmentation in optical coherence tomography (OCT) images is crucial for the diagnosis and study of retinal diseases. Graph-based methods are commonly used in layer segmentation. However, most of these methods require a lot of human efforts for determining an appropriate model to compute good edge weights. In this paper, we propose a novel automatic method for segmenting retinal layers in macular OCT images. Specially, we propose a new fully convolutional deep learning architecture with a side output layer to directly learn optimal graph-edge weights from raw pixels. The architecture can automatically learn multi-scale and multi-level features to generate accurate boundary probabilities as good edge weights without hand-crafted appropriate models. The boundaries are finalized by using graph segmentation method. The proposed method is evaluated on a dataset with 130 OCT B-scans. The experimental results show the mean absolute boundary positioning differences are 1.48±0.34 pixel.

ECG biometrics has achieved great success on high quality ECG signals. However, it is still a challenging problem to apply ECG biometrics on mobile devices due to the low quality signals. In this paper, we propose a robust two-stage model. In first stage, we utilize 1D CNN model to remove the invalid heartbeats from ECG recording. And then, we combine the raw signal with the hidden feature of 1D CNN as the feature representation of heartbeat. In second stage, we group a certain number of heartbeat representations as input sequence. Attention-based bidirectional LSTM is used to aggregate input sequence and generate discriminative identity features for recognition. We evaluate our method on two public datasets, and the results show that our two-stage model can achieve the state-of-the-art performance compared with other existing methods.

Finger vein recognition has received a lot of attention recently and is viewed as a promising biometric trait. In related methods, vein pattern-based methods explore intrinsic finger vein recognition, but their performance remains unsatisfactory owing to defective vein networks and weak matching. One important reason may be the neglect of deep analysis of the vein anatomy structure. By comprehensively exploring the anatomy structure and imaging characteristic of vein patterns, this paper proposes a novel finger vein recognition framework, including an anatomy structure analysis-based vein extraction algorithm and an integration matching strategy. Specifically, the vein pattern is extracted from the orientation map-guided curvature based on the valley- or half valley-shaped cross-sectional profile. In addition, the extracted vein pattern is further thinned and refined to obtain a reliable vein network. In addition to the vein network, the relatively clear vein branches in the image are mined from the vein pattern, referred to as the vein backbone. In matching, the vein backbone is used in vein network calibration to overcome finger displacements. The similarity of two calibrated vein networks is measured by the proposed elastic matching and further recomputed by integrating the overlap degree of corresponding vein backbones. Extensive experiments on two public finger vein databases verify the effectiveness of the proposed framework.

We propose a new multiscale rotation-invariant convolutional neural network (MRCNN) model for classifying various lung tissue types on high-resolution computed tomography. MRCNN employs Gabor-local binary pattern that introduces a good property in image analysis-invariance to image scales and rotations. In addition, we offer an approach to deal with the problems caused by imbalanced number of samples between different classes in most of the existing works, accomplished by changing the overlapping size between the adjacent patches. Experimental results on a public interstitial lung disease database show a superior performance of the proposed method to state of the art.

Image-set classification has recently generated great popularity due to its widespread applications in computer vision. The great challenges arise from effectively and efficiently measuring the similarity between image sets with high inter-class ambiguity and huge intra-class variability. In this paper, we propose deep match kernels (DMK) to directly measure the similarity between image sets in the match kernel framework. Specifically, we build deep local match kernels between images upon arc-cosine kernels, which can faithfully characterize the similarity between images by mimicking deep neural networks, we introduce anchors to aggregate those deep local match kernels into a global match kernel between image sets, which is learned in a supervised way by kernel alignment and therefore more discriminative. The DMK provides the first match kernel framework for image-set classification, which removes specific assumptions usually required in previous approaches and is computationally more efficient. We conduct extensive experiments on four datasets for three diverse image-set classification tasks. The DMK achieves high performance and consistently surpasses state-of-the-art methods, showing its great effectiveness for image-set classification.

Although some developments have been achieved in finger vein recognition recently, the image deformation problem has received relatively less attention and still intractable. In this paper, the reason and the harmfulness of this problem are analyzed firstly. And then, a deformable finger vein recognition framework is proposed to deal with this problem, consisting of the improved vein PCA-SIFT feature and bidirectional deformable spatial pyramid matching (BDSPM). Furthermore, we build a finger vein deformation database to imitate image deformation in real application. The experimental results, on the self-built deformation database and one public database, prove the effectiveness of the proposed framework for dealing with the image deformation problem.

Efficient identification of finger veins is still a challenging problem due to the increasing size of the finger vein database. Most leading finger vein image identification methods have high-dimensional real-valued features, which result in extremely high computation complexity. Hashing algorithms are extraordinary effective ways to facilitate finger vein image retrieval. Therefore, in this paper, we proposed a finger vein image retrieval scheme based on Affinity-Preserving K-means Hashing (APKMH) algorithm and bag of subspaces based image feature. At first, we represent finger vein image by Nonlinearly Sub-space Coding (NSC) method which can obtain the discriminative finger vein image features. Then the features space is partitioned into multiple subsegments. In each subsegment, we employ the APKMH algorithm, which can simultaneously construct the visual codebook by directly k-means clustering and encode the feature vector as the binary index of the codeword. Experimental results on a large fused finger vein dataset demonstrate that our hashing method outperforms the state-of-the-art finger vein retrieval methods.

Although some image retrieval methods were proposed to accelerate finger vein recognition, the insufficient feature (e.g., the number of vein point) and unfavorable encoding (e.g., predefined threshold based binarization) limited retrieval performance largely. In view of this problem, we develop a new retrieval framework, based on the integration of discriminative texture features and similarity-preserving binary codes. In detail, the vector and scalar features, measuring the gray level, gray difference, and gray gathering of image patch, are both used to represent finger vein image. And to improve the retrieval efficiency, the high-dimensional decimal features are further encoded into the compact binary patterns by principal component analysis (PCA) and similarity-preserving iterative quantization (ITQ). Experimental results on one large finger vein database prove that the proposed method can powerfully improve the retrieval accuracy and efficiency.

Finger vein image retrieval is one significant technique for performing fast identification especially in large-scale applications. However, most existing retrieval methods were based on fixed-scale feature of non-overlapped rectangular image block, in which the representation ability of feature and the local consistency of vein pattern were both overlooked. And the weak encoding (e.g., predefined threshold based binarization) was also limited the retrieval performance. Focusing on these problems, this paper proposes a novel finger vein image retrieval framework based on similarity-preserving encoding of scale-varied superpixel feature. In the framework, locally consistent pixels in one superpixel are used as a unit of feature representation, and the feature length is varied with the category of the superpixel classified by the variance of lowest dimensional feature. Additionally, the feature compaction and feature rotation based encoding can minimize the quantization loss and preserve the similarity between the scale-varied feature and the encoded binary codes. Experimental results on six public finger vein databases demonstrate that the superiority of the proposed coding scale-varied superpixel feature based retrieval approach over the state-of-the-arts.

Finger vein pattern, as a promising hand-based biometric technology, has been well studied in recent years. In this paper, a new superpixel-based finger vein recognition method is presented. In the proposed method, we develop two types of effective superpixels, i.e. stable superpixel and discriminative superpixel to represent finger vein image and these superpixels are expected to play different roles in matching stage. In detail, the stable and discriminative superpixels are firstly learned from the training images for each enrolled class. When verifying a testing image, we just compare the superpixels at the same location as the two types of superpixels in template. Then, the two types of superpixels are combined utilizing a reversible weight-based fusion method in score level. Additionally, to further improve the recognition performance, we explore the superpixel context feature (SPCF). For each superpixel the SPCF is obtained by comparing the current superpixel with its surrounding neighbors. In the final matching stage, we integrate the matching score of two types of superpixels and it of the SPCF using the weighted SUM fusion method. The experimental results on two open finger vein databases, i.e. PolyU and SDUMLA-FV, show that our method not only performs better than the existing superpixel-based method, but also has advantages in comparison with some traditional ones.

Finger vein pattern has become one of the most promising biometric identifiers. In this paper, a robust method based on Bag-of-Words (BoW) is developed for finger vein verification. Firstly, some robust and discriminative visual words are learned from local base features such as Local Binary Pattern (LBP), Mean Curvature and Webber Local Descriptor (WLD). We name these visual words as Finger Vein Textons (FVTs). Secondly, each image is mapped into a FVTs matrix. Finally, spatial pyramid matching (SPM) method is applied to maintain spatial layout information by representing each image as pyramid histogram which is performed for matching by histogram intersection function. Experimental results show that the proposed method achieves satisfactory performance both on our database and the open PolyU database. In addition, our method also has strong robustness and high accuracy on the self-built rotation and illumination databases.

Finger vein pattern has become one of the most promising biometric identifiers. In this paper, we propose a robust finger vein verification method based on a personalized best patches map (PBPM). Firstly, some robust and discriminative visual words of finger vein are learned from traditional base feature such as local binary pattern (LBP). These visual words are named as finger vein textons (FVTs), which can well represent the visual primitives of finger vein. Secondly, we represent the finger vein image as a finger vein textons map (FVTM) by mapping each patch of the image into the closest FVT. Thirdly, by rejecting inconsistent patches, the PBPM of a certain individual is learned from these FVTMs which are extracted from the training samples of the same finger. Finally, the matched best patch ratio is used to measure similarity between the extracted FVTM of the input finger and the PBPM of a certain individual. Experimental results show that our method achieves satisfactory performance on the open PolyU database. In addition, it also has strong robustness and high accuracy on the self-built rotation and translation databases.

Finger veins based biometrics, as a new approach to personal identification, has received much attention in recent years. The methods based on low level feature, for instance the gray, texture of finger vein, are the mainstream, but they are usually faced with many challenges, such as sensitivity to noise and low local consistency. In fact, finger vein recognition based on high level feature representation has been proved to be a promising way to effectively overcome the above limitations and improve the system performance. Thus, in this paper, we present a novel identification framework, which utilizes superpixel-based features (SPFs) of finger vein for high level feature representation. When comparing two finger veins, the features of each pixel are firstly extracted as base attributes by traditional way. Then, after superpixel over-segmentation, the SPF of each finger vein can be obtained based on its base attributes by some statistical techniques. Lastly, a weighted spatial pyramid matching (WSPM) scheme is utilized to implement matching. Our experiments have yielded some very good results evidenced by an EER of 0.0147 on the benchmark database PolyU.

How to improve the performance of an existing biometric system is always interesting and meaningful. In this paper, we present a novel method of score level fusion using multiple enrolled impressions to achieve higher verification accuracy of existing fingerprint systems. The main idea of the method is to build a representation of the biometric reference as a polyhedron by taking into account the matching results of multiple enrolled impressions. The verification step consists in measuring a distance between the centroid of the polyhedron and the acquired image. This novel method outperforms the traditional uni-matcher based scheme over a wide range of FAR and FRR values. The equal error rate of our method is observed to be 2.25%, while that of the uni-matcher is 5.75%.

The need for sensor interoperability has increased tremendously in many fingerprint large-scale application areas such as e-commerce, welfare-disbursement and e-education. However, the problem of feature selection for sensor interoperability has received limited attention in the literature. In this paper, the relationships among person, sensor and feature are discussed. Especially, a feature selection method for sensor interoperability is proposed. Some experimental results of feature selection for sensor interoperability in fingerprint segmentation are presented as a case study. Experiments show that the various features exhibit different sensor interoperability on different sensors.