LiDAR Semantic Scene Completion (SSC) in autonomous driving requires predicting both dense occupancy and semantic labels from sparse input point cloud. Existing methods typically adopt cascaded architecture for feature dilation and semantic abstraction, which blurs distinctive geometric patterns and reduces feature discriminability. Moreover, given an input, conventional processing of the ground truth labels overlooks voxel predictability in the target, resulting in ill-posed supervision and discards informative voxels. To address these limitations, we propose Sparse-Dense Net (SDNet), a dual-branch architecture that processes the input points through parallel sparse and dense encoders. The complementary features are aligned and fused using a Sparse Dense Feature Fusion (SDFF) module and further refined by a Feature Propagation (FP) module. Additionally, we introduce an input-aware label refinement strategy, including Sparse-Guided Filtering (SGF) to filter unpredictable targets and Ignored Voxel Recycling (IVR) to leverage informative ignored voxels for auxiliary supervision. These innovations enhance both feature learning and label quality. Extensive experiments on SemanticKITTI and nuScenes OpenOccupancy datasets validate the effectiveness of our approach, with SDNet achieving state-of-the-art performance on both datasets and ranking 1st on the official SemanticKITTI benchmark with 42.1 mIoU, outperforming the previous best by 4.2 (+11.1\%).

Vehicle-to-everything (V2X) collaborative perception has recently gained increasing attention in autonomous driving due to its ability to enhance scene understanding by integrating information from other collaborators, e.g. vehicles or infrastructure. Existing algorithms usually share deep features to achieve a trade-off between accuracy and bandwidth. However, most of these methods require joint training of all agents, which results in privacy leakage and is impractical and unacceptable in the real world. Sharing prediction results seems to be a direct solution, but its performance is suboptimal and sensitive to localization noise and communication delay. In this paper, we propose a privacy-preserving collaborative perception framework, where each agent is separately trained with its own dataset and the ego vehicle needs to integrate with completely unknown collaborators. Specifically, we propose MSD, a multi-scale feature fusion method combined with deformable attention, to better fuse features of different agents. We also propose a plug-in domain adapter to align the features from unknown collaborators to ego-domain. Extensive experiments on the challenging DAIR-V2X and V2V4Real demonstrate that: 1) MSD achieves remarkable performance, outperforming others by at least 2.8% and 6.7% in AP0.7 on DAIR-V2X and V2V4Real, respectively; 2) After domain adaptation, it significantly outperforms the No Fusion, Late Fusion scenarios and can approach or even surpass the performance of joint training. We truly achieves privacy-preserving collaboration, providing a new paradigm for the study of collaborative perception, which is crucial for practical applications.

3D object detection is one of the fundamental perception tasks for autonomous vehicles. Fulfilling such a task with a 4D millimeter-wave radar is very attractive since the sensor is able to acquire 3D point clouds similar to Lidar while maintaining robust measurements under adverse weather. However, due to the high sparsity and noise associated with the radar point clouds, the performance of the existing methods is still much lower than expected. In this paper, we propose a novel Semi-supervised Cross-modality Knowledge Distillation (SCKD) method for 4D radar-based 3D object detection. It characterizes the capability of learning the feature from a Lidar-radar-fused teacher network with semi-supervised distillation. We first propose an adaptive fusion module in the teacher network to boost its performance. Then, two feature distillation modules are designed to facilitate the cross-modality knowledge transfer. Finally, a semi-supervised output distillation is proposed to increase the effectiveness and flexibility of the distillation framework. With the same network structure, our radar-only student trained by SCKD boosts the mAP by 10.38% over the baseline and outperforms the state-of-the-art works on the VoD dataset. The experiment on ZJUODset also shows 5.12% mAP improvements on the moderate difficulty level over the baseline when extra unlabeled data are available.

Multivariate Time Series (MTS) forecasting plays a vital role in various real-world applications, such as traffic management and predictive maintenance. Existing approaches typically model MTS data in either Euclidean or Riemannian space, limiting their ability to capture the diverse geometric structures and complex Spatio-Temporal (ST) dependencies inherent in real-world data. To overcome this limitation, we propose the Hybird Symmetric Positive-Definite Manifold Graph Neural Network (HSMGNN), a novel graph neural network-based model that captures data geometry within a hybrid Euclidean–Riemannian framework. To the best of our knowledge, this is the first work to leverage hybrid geometric representations for MTS forecasting, enabling expressive and comprehensive modeling of geometric properties. Specifically, we introduce a Submanifold-Cross-Segment (SCS) embedding to project input MTS into both Euclidean and Riemannian spaces, thereby capturing ST variations across distinct geometric domains. To alleviate the high computational cost of Riemannian distance, we further design an Adaptive-Distance-Bank (ADB) layer with a trainable memory mechanism. Finally, a Fusion Graph Convolutional Network (FGCN) is devised to integrate features from the dual spaces via a learnable fusion operator for accurate prediction. Experiments on three benchmark datasets demonstrate that HSMGNN achieves up to 13.8% improvement over state-of-the-art baselines in forecasting accuracy.

4D radar has received significant attention in autonomous driving thanks to its robustness under adverse weathers. Due to the sparse points and noisy measurements of the 4D radar, most of the research finish the 3D object detection task by integrating images from camera and perform modality fusion in BEV space. However, the potential of the radar and the fusion mechanism is still largely unexplored, hindering the performance improvement. In this study, we propose a cross-view two-stage fusion network called CVFusion. In the first stage, we design a radar guided iterative (RGIter) BEV fusion module to generate high-recall 3D proposal boxes. In the second stage, we aggregate features from multiple heterogeneous views including points, image, and BEV for each proposal. These comprehensive instance level features greatly help refine the proposals and generate high-quality predictions. Extensive experiments on public datasets show that our method outperforms the previous state-of-the-art methods by a large margin, with 9.10% and 3.68% mAP improvements on View-of-Delft (VoD) and TJ4DRadSet, respectively. Our code will be made publicly available.

Collaborative visual perception methods have gained widespread attention in the autonomous driving community in recent years due to their ability to address sensor limitation problems. However, the absence of explicit depth information often makes it difficult for camera-based perception systems, e.g., 3D object detection, to generate accurate predictions. To alleviate the ambiguity in depth estimation, we propose RayFusion, a ray-based fusion method for collaborative visual perception. Using ray occupancy information from collaborators, RayFusion reduces redundancy and false positive predictions along camera rays, enhancing the detection performance of purely camera-based collaborative perception systems. Comprehensive experiments show that our method consistently outperforms existing state-of-the-art models, substantially advancing the performance of collaborative visual perception. Our code will be made publicly available.

One-shot object detection (OSOD) aims to detect all object instances towards the given category specified by a query image. Most existing studies in OSOD endeavor to establish effective cross-image correlation with limited query information, however, ignoring the problems of the model bias towards the base classes and the generalization degradation on the novel classes. Observing this, we propose a novel algorithm, namely Base-class Suppression with Prior Guidance (BSPG) network to achieve bias-free OSOD. Specifically, the objects of base categories can be detected by a base-class predictor and eliminated by a base-class suppression module (BcS). Moreover, a prior guidance module (PG) is designed to calculate the correlation of high-level features in a non-parametric manner, producing a class-agnostic prior map with unbiased semantic information to guide the subsequent detection process. Equipped with the proposed two modules, we endow the model with a strong discriminative ability to distinguish the target objects from distractors belonging to the base classes. Extensive experiments show that our method outperforms the previous techniques by a large margin and achieves new state-of-the-art performance under various evaluation settings.

The object of temporal action localization (TAL) is to predict the predefined action labels and the corresponding temporal boundary in a video. It can be found that TAL is a task of multi-modal modeling and highly dependent on the effect of temporal context representation. Inspired by this property, we propose an end-to-end local-to-global modeling architecture to learn the contextual consistency information in temporal sequence and cross-modal. Specifically, a local-to-global encoding Transformer is applied to model the video sequence to obtain video representation of different time scales. To achieve a reasonable balance between the specificity and correlation of different modalities, a cross semantic alignment (CSA) module is proposed to re-weight the encoded multi-model features by whether attending to the semantic correlations or specificity in different modalities. Further, to learn the trans-modal consistency from local to global and the uni-modal consistency belonging to the same category, the self-consistency learning (SCL) is designed to train the network. The experimental results demonstrate the significance of our method in major improvements upon prior works. Our model achieves 68.3% and 37.1% average mAPs on THUMOS14 and ActivityNet 1.3, outperforming state-of-the-art multi-stage and one-stage models.

With the widespread application of biometric, unprotected biometric data is still at risk of serious security and privacy breaches. When large amounts of unprotected biometric data leak, cancelable biometric become a powerfully remedial measure. In this paper, we propose a new method to generate stable and cancelable face templates based on multiple partial Walsh transformations (MPWT) and Simhash. Firstly, multiple partial Walsh matrices generated with random external parameters perform projection transformation on the original real-valued face features, ensuring the irreversibility and unlinkability of the system. Subsequently, the projected features are transformed into discrete binary codes (protected templates) using Simhash. And the random permutation seed ensures the revocability of generated protected template. Furtherly, the protected templates have small storage space and is more suitable for fast comparison but also yields improvements in recognition accuracy compared with several state-of-the-arts. Numerous experiments on CASIA-WebFace, LFW, FEI, and Color FERET databases show that the protected templates are nearly identical to the unprotected ones in the comparison performance. The scheme also meets the requirements of non-invertibility, revocability, unlinkability, as well as resistance for various types of attacks like attacks via record multiplicity, false accepts, brute force and pre-image. Therefore, the proposed methodology strikes a balance between recognition accuracy and security.

Multi-Agent Reinforcement Learning (MARL) has become a promising solution for constructing a multi-agent autonomous driving system (MADS) in complex and dense scenarios. But most methods consider agents acting selfishly, which leads to conflict behaviors. Some existing works incorporate the concept of social value orientation (SVO) to promote coordination, but they lack the knowledge of other agents' SVOs, resulting in conservative maneuvers. In this paper, we aim to tackle the mentioned problem by enabling the agents to understand other agents' SVOs. To accomplish this, we propose a two-stage system framework. Firstly, we train a policy by allowing the agents to share their ground truth SVOs to establish a coordinated traffic flow. Secondly, we develop a recognition network that estimates agents' SVOs and integrates it with the policy trained in the first stage. Experiments demonstrate that our developed method significantly improves the performance of the driving policy in MADS compared to two state-of-the-art MARL algorithms.

Accurate detection of the fake but photorealistic images is one of the most challenging tasks to address social, biometrics security and privacy related concerns in our community. Earlier research has underlined the existence of spectral domain artifacts in fake images generated by powerful generative adversarial network (GAN) based methods. Therefore, a number of highly accurate frequency domain methods to detect such GAN generated images have been proposed in the literature. Our study in this paper introduces a pipeline to mitigate the spectral artifacts. We show from our experiments that the artifacts in frequency spectrum of such fake images can be mitigated by proposed methods, which leads to the sharp decrease of performance of spectrum-based detectors. This paper also presents experimental results using a large database of images that are synthesized using BigGAN, CRN, CycleGAN, IMLE, Pro-GAN, StarGAN, StyleGAN and StyleGAN2 (including synthesized high resolution fingerprint images) to illustrate effectiveness of the proposed methods. Furthermore, we select a spatial-domain based fake image detector and observe a notable decrease in the detection performance when proposed method is incorporated. In summary, our insightful analysis and pipeline presented in this paper cautions the forensic community on the reliability of GAN-generated fake image detectors that are based on the analysis of frequency artifacts as these artifacts can be easily mitigated.

Matching contactless fingerprints or finger photos to contact-based fingerprint impressions has received increased attention in the wake of COVID-19 due to the superior hygiene of the contactless acquisition and the widespread availability of low cost mobile phones capable of capturing photos of fingerprints with sufficient resolution for verification purposes. This paper presents an end-to-end automated system, called C2CL, comprised of a mobile finger photo capture app, preprocessing, and matching algorithms to handle the challenges inhibiting previous cross-matching methods; namely i) low ridge-valley contrast of contactless fingerprints, ii) varying roll, pitch, yaw, and distance of the finger to the camera, iii) non-linear distortion of contact-based fingerprints, and vi) different image qualities of smartphone cameras. Our preprocessing algorithm segments, enhances, scales, and unwarps contactless fingerprints, while our matching algorithm extracts both minutiae and texture representations. A sequestered dataset of 9, 888 contactless 2D fingerprints and corresponding contact-based fingerprints from 206 subjects (2 thumbs and 2 index fingers for each subject) acquired using our mobile capture app is used to evaluate the cross-database performance of our proposed algorithm. Furthermore, additional experimental results on 3 publicly available datasets show substantial improvement in the state-of-the-art for contact to contactless fingerprint matching (TAR in the range of 96.67% to 98.30% at FAR=0.01%).

With the decreasing size of fingerprint scanners in portable devices, e.g., mobile phone and smart watch, partial fingerprint recognition has become a challenging and urgently needed technique due to the limited features contained in small area as well as the large rotation and translation between query and reference images. Deep learning as a powerful modeling method has advanced the research progress of fingerprint recognition, but it still suffers from the lack of labeled data in the scenario of partial fingerprint matching. In this paper, we propose a novel partial fingerprint verification network (PFVNet) based on spatial transformer network (STN) and the local self-attention mechanism. Our model can be trained end-to-end and learn multi-level fingerprint features automatically. To alleviate the data annotation work, the model is trained in a self-supervision and domain adaptation manner with data generated from large fingerprint image matching. The experimental results compared with other methods on FVC2006 DB1 dataset and in-house datasets (i.e., ZJUPartial database) show that our method achieves state-of-the-art performance, and also robust to different types of scanners.

Cancelable biometrics is a powerful remedy for information leakage caused by the extensive usage of unprotected biometric data. Current measures usually suffer from deteriorated accuracy, which is known as the security–performance tradeoff. Motivated by these concerns, in this article, a novel cancelable fingerprint approach, i.e., Indexing-Min–Max (IMM) hashing, is proposed to securely transform a fixed-length fingerprint feature vector to a discrete index hashed code. IMM hashing is essentially established upon the min–max hash and further strengthened by the integration of the partial Hadamard transform, which alleviates performance deterioration while maintaining a high security level. Extensive experiments on FVC2002 and FVC2004 fingerprint datasets coupled with comprehensive theoretical analyses demonstrate the favorable accuracy and strong anti-attack resilience of the proposed method. Besides, compared to the unprotected counterpart, the matching precision of the protected templates yields little accuracy loss or even improved performance, which means the security–performance tradeoff is well handled. Furthermore, IMM hashing also meets the unlinkability and revocability requisites of cancelable biometrics.

Representing fingerprint templates in binary form can provide outstanding merits compared to the conventional minutiae-based fingerprint recognition system. The existing fixed-length fingerprint feature extraction methods either suffer from redundant feature magnitude or lack of template security. In this letter, we present a compact (128 bytes) and cancelable fingerprint binary codes generation scheme which enables accurate and efficient comparison as well as high security. This binary representation is also available for advanced encryption schemes (e.g., fuzzy commitment). Specifically, a kernel learning-based real-valued fingerprint feature is converted into compact and cancelable binary code via one permutation hashing. A partial Haar transform is deployed to further strengthen the irreversibility of the whole system. Experimental results on six benchmark datasets FVC2002 and FVC2004 coupled with security analysis demonstrate the superiority of the proposed method compared with several state-of-the-arts.

Partial fingerprint verification is a challenging task because of the few features contained in small area as well as the large rotation angle and translation between query images and template images. In this paper, we propose a new framework of partial fingerprint verification based on spatial transformer networks (STN) model, where a transform model, i.e., AlignNet network, is proposed to estimate the alignment parameters, and the verification is modeled as a binary classification task. The experimental results on the simulated datasets created from FVC2004 and the real-world dataset FVC2006 DB1 show that our method is invariant to rotation, and also robust to different kinds of scanners, and dramatically outperforms the rank-1 entry of FVC2006 participants. The EER on FVC2006 DB1 of the proposed algorithm is 3.587% compared to that of 5.564%, the best of FVC2006 DB1 entries.

The minutiae set defined by the ISO/IEC 19794‐2 is one of the prevalent feature used in fingerprint recognition systems. Unfortunately, such characteristic of unordered and variable‐sized minutiae information causes a restriction on the operation in some advanced template protection methods (e.g. fuzzy commitment), which usually require an ordered and fixed‐length binary feature representation as the system input. In this study, in order to simultaneously extend the application of fingerprint recognition and provide satisfactory system performance, the authors propose a novel fixed‐length bit‐string conversion framework based on spectral clustering and the proposed newly designed discriminative fingerprint representation called minutia vicinity combined feature (MVCF). The proposed method consists of three stages: (i) the extraction of MVCF, (ii) bit conversion via the spectral clustering algorithm, and (iii) matching. Benefiting from feature invariance, fixed‐length and bit‐oriented coding, merits such as fast matching and decent accuracy are well guaranteed. The performance evaluation is conducted on six publicly available benchmark data sets: FVC2002 DB1, DB2, DB3 and FVC2004 DB1, DB2, DB3 confirms the superiority of the proposed method and suggests the promise of migrating to some other domains (e.g., template protection).

Infant recognition has received increasing attention in recent years in many applications, such as tracking child vaccination and identifying missing children. Due to the lack of efficient identification methods for infants and newborns, the current methods of infant recognition rely on identification of parents or certificates of identity. While biometric recognition technologies (e.g., face and fingerprint recognition) have been widely deployed in many applications for recognizing adults and teenagers, no such recognition systems yet exist for infants or newborns. One of the major problems is that the biometric traits of infants and newborns are either not permanent (e.g., face) or difficult to capture (e.g., fingerprint) due to lack of appropriate sensors. In this paper, we investigate the feasibility of infant recognition by their footprint using a 500 ppi commodity friction ridge sensor. We collected an infant footprint dataset in three sessions, consisting of 60 subjects, with age range from 1 to 9 months. We proposed a new minutia descriptor based on deep convolutional neural network for measuring minutiae similarity. The descriptor is compact and highly discriminative. We conducted verification experiments for both single enrolled template and fusion of multiple enrolled templates, and show the impact of age and time gap on matching performance. Comparison experiments with state of the art algorithm show the advantage of the proposed minutia descriptor.

Fingerprint features can be divided into three major categories based on the granularity at which they are extracted: level 1, level 2, and level 3 features. Orientation field, ridge frequency field, and minutiae set are three fundamental components of fingerprint, where the orientation field and ridge frequency field are regarded as level 1 features and minutiae set as level 2 features. It is generally believed that level 1 features, especially orientation field, can be reconstructed from level 2 features, i.e., minutiae. However, it is still a question that if minutiae can be extracted from level 1 features. In this paper, we analyze the relations between level 1 and level 2 features using the frequency modulation (FM) model and propose an approach to extract minutiae from level 1 features (i.e., orientation field and frequency field). The proposed algorithm is evaluated on NIST SD27 and FVC2002 DB1 databases. The true detection rate (TDR) and false detection rate (FDR) of minutiae detection on NIST SD27 and FVC2002 DB1 are about 45% and 30% compared with manually marked minutiae, respectively, with level 1 features extracted at a block size of 16 pixels. When pixelwise orientation and frequency fields are available, TDR and FDR can reach 70% and 25%, respectively. With a smaller block size, the minutiae recovering accuracy can be even higher. Our quantitative and experimental results show the deep relationship between level 1 and level 2 features of a fingerprint.

Latent fingerprint matching has played a critical role in identifying suspects and criminals. However, compared to rolled and plain fingerprint matching, latent identification accuracy is significantly lower due to complex background noise, poor ridge quality and overlapping structured noise in latent images. Accordingly, manual markup of various features (e.g., region of interest, singular points and minutiae) is typically necessary to extract reliable features from latents. To reduce this markup cost and to improve the consistency in feature markup, fully automatic and highly accurate ("lights-out" capability) latent matching algorithms are needed. In this paper, a dictionary-based approach is proposed for automatic latent segmentation and enhancement towards the goal of achieving "lights-out" latent identification systems. Given a latent fingerprint image, a total variation (TV) decomposition model with L1 fidelity regularization is used to remove piecewise-smooth background noise. The texture component image obtained from the decomposition of latent image is divided into overlapping patches. Ridge structure dictionary, which is learnt from a set of high quality ridge patches, is then used to restore ridge structure in these latent patches. The ridge quality of a patch, which is used for latent segmentation, is defined as the structural similarity between the patch and its reconstruction. Orientation and frequency fields, which are used for latent enhancement, are then extracted from the reconstructed patch. To balance robustness and accuracy, a coarse to fine strategy is proposed. Experimental results on two latent fingerprint databases (i.e., NIST SD27 and WVU DB) show that the proposed algorithm outperforms the state-of-the-art segmentation and enhancement algorithms and boosts the performance of a state-of-the-art commercial latent matcher.

Latent fingerprints serve as an important source of forensic evidence in a court of law. Automatic matching of latent fingerprints to rolled/plain (exemplar) fingerprints with high accuracy is quite vital for such applications. However, latent impressions are typically of poor quality with complex background noise which makes feature extraction and matching of latents a significantly challenging problem. We propose incorporating top-down information or feedback from an exemplar to refine the features extracted from a latent for improving latent matching accuracy. The refined latent features (e.g. ridge orientation and frequency), after feedback, are used to re-match the latent to the top K candidate exemplars returned by the baseline matcher and resort the candidate list. The contributions of this research include: (i) devising systemic ways to use information in exemplars for latent feature refinement, (ii) developing a feedback paradigm which can be wrapped around any latent matcher for improving its matching performance, and (iii) determining when feedback is actually necessary to improve latent matching accuracy. Experimental results show that integrating the proposed feedback paradigm with a state-of-the-art latent matcher improves its identification accuracy by 0.5-3.5 percent for NIST SD27 and WVU latent databases against a background database of 100k exemplars.

With the availability of live-scan palmprint technology, high resolution palmprint recognition has started to receive significant attention in forensics and law enforcement. In forensic applications, latent palmprints provide critical evidence as it is estimated that about 30 percent of the latents recovered at crime scenes are those of palms. Most of the available high-resolution palmprint matching algorithms essentially follow the minutiae-based fingerprint matching strategy. Considering the large number of minutiae (about 1,000 minutiae in a full palmprint compared to about 100 minutiae in a rolled fingerprint) and large area of foreground region in full palmprints, novel strategies need to be developed for efficient and robust latent palmprint matching. In this paper, a coarse to fine matching strategy based on minutiae clustering and minutiae match propagation is designed specifically for palmprint matching. To deal with the large number of minutiae, a local feature-based minutiae clustering algorithm is designed to cluster minutiae into several groups such that minutiae belonging to the same group have similar local characteristics. The coarse matching is then performed within each cluster to establish initial minutiae correspondences between two palmprints. Starting with each initial correspondence, a minutiae match propagation algorithm searches for mated minutiae in the full palmprint. The proposed palmprint matching algorithm has been evaluated on a latent-to-full palmprint database consisting of 446 latents and 12,489 background full prints. The matching results show a rank-1 identification accuracy of 79.4 percent, which is significantly higher than the 60.8 percent identification accuracy of a state-of-the-art latent palmprint matching algorithm on the same latent database. The average computation time of our algorithm for a single latent-to-full match is about 141 ms for genuine match and 50 ms for impostor match, on a Windows XP desktop system with 2.2-GHz CPU and 1.00-GB RAM. The computation time of our algorithm is an order of magnitude faster than a previously published state-of-the-art-algorithm.

Traditional minutiae matching algorithms assume that each minutia has the same discriminability. However, this assumption is challenged by at least two facts. One of them is that fingerprint minutiae tend to form clusters, and minutiae points that are spatially close tend to have similar directions with each other. When two different fingerprints have similar clusters, there may be many well matched minutiae. The other one is that false minutiae may be extracted due to low quality fingerprint images, which result in both high false acceptance rate and high false rejection rate. In this paper, we analyze the minutiae discriminability from the viewpoint of global spatial distribution and local quality. Firstly, we propose an effective approach to detect such cluster minutiae which of low discriminability, and reduce corresponding minutiae similarity. Secondly, we use minutiae and their neighbors to estimate minutia quality and incorporate it into minutiae similarity calculation. Experimental results over FVC2004 and FVC-onGoing demonstrate that the proposed approaches are effective to improve matching performance.

This letter develops a novel method for fingerprint singular point detection based on a new singularity representation of ridge-valley region called orientation entropy. The candidate singular point is obtained by the multiple-scale analysis of orientation entropy and some post processing steps are proposed to filter the spurious core and delta points. An iteration compensation scheme is proposed to search the precise location for core points against the offset further. Performance of the proposed method has been evaluated on the dataset of FVC2002 DB1. Experimental results show that the multiple-scale orientation entropy is correct and effective for singular detection and the location compensation scheme reduces the distance between the detection result and the truth singular point.