We aim to solve unsupervised anomaly detection in a practical challenging environment where the normal dataset is both contaminated with defective regions and its product class distribution is tailed but unknown. We observe that existing models suffer from tail-versus-noise trade-off where if a model is robust against pixel noise, then its performance deteriorates on tail class samples, and vice versa. To mitigate the issue, we handle the tail class and noise samples independently. To this end, we propose TailSampler, a novel class size predictor that estimates the class cardinality of samples based on a symmetric assumption on the class-wise distribution of embedding similarities. TailSampler can be utilized to sample the tail class samples exclusively, allowing to handle them separately. Based on these facets, we build a memory-based anomaly detection model TailedCore, whose memory both well captures tail class information and is noise-robust. We extensively validate the effectiveness of TailedCore on the unsupervised long-tail noisy anomaly detection setting, and show that TailedCore outperforms the state-of-the-art in most settings. Code is available in TailedCore

Very low-resolution face recognition (VLRFR) poses unique challenges, such as tiny regions of interest and poor resolution due to extreme standoff distance or wide viewing angle of the acquisition devices. In this paper, we study principled approaches to elevate the recognizability of a face in the embedding space instead of the visual quality. We first formulate a robust learning-based face recognizability measure, namely recognizability index (RI), based on two criteria: (i) proximity of each face embedding against the unrecognizable faces cluster center and (ii) closeness of each face embedding against its positive and negative class prototypes. We then devise an index diversion loss to push the hard-to-recognize face embedding with low RI away from unrecognizable faces cluster to boost the RI, which reflects better recognizability. Additionally, a perceptibility attention mechanism is introduced to attend to the most recognizable face regions, which offers better explanatory and discriminative traits for embedding learning. Our proposed model is trained end-to-end and simultaneously serves recognizability-aware embedding learning and face quality estimation. To address VLRFR, our extensive evaluations on three challenging low-resolution datasets and face quality assessment demonstrate the superiority of the proposed model over the state-of-the-art methods.

A neural network trained on a classification dataset often exhibits a higher vector norm of hidden layer features for in-distribution (ID) samples, while producing relatively lower norm values on unseen instances from out-of-distribution (OOD). Despite this intriguing phenomenon being utilized in many applications, the underlying cause has not been thoroughly investigated. In this study, we demystify this very phenomenon by scrutinizing the discriminative structures concealed in the intermediate layers of a neural network. Our analysis leads to the following discoveries: (1) The feature norm is a confidence value of a classifier hidden in the network layer, specifically its maximum logit. Hence, the feature norm distinguishes OOD from ID in the same manner that a classifier confidence does. (2) The feature norm is class-agnostic, thus it can detect OOD samples across diverse discriminative models. (3) The conventional feature norm fails to capture the deactivation tendency of hidden layer neurons, which may lead to misidentification of ID samples as OOD instances. To resolve this drawback, we propose a novel negative-aware norm (NAN) that can capture both the activation and deactivation tendencies of hidden layer neurons. We conduct extensive experiments on NAN, demonstrating its efficacy and compatibility with existing OOD detectors, as well as its capability in label-free environments.

Detecting out-of-distribution (OOD) samples are crucial for machine learning models deployed in open-world environments. Classifier-based scores are a standard approach for OOD detection due to their fine-grained detection capability. However, these scores often suffer from overconfidence issues, misclassifying OOD samples distant from the in-distribution region. To address this challenge, we propose a method called Nearest Neighbor Guidance (NNGuide) that guides the classifier-based score to respect the boundary geometry of the data manifold. NNGuide reduces the overconfidence of OOD samples while preserving the fine-grained capability of the classifier-based score. We conduct extensive experiments on ImageNet OOD detection benchmarks under diverse settings, including a scenario where the ID data undergoes natural distribution shift. Our results demonstrate that NNGuide provides a significant performance improvement on the base detection scores, achieving state-of-the-art results on both AUROC, FPR95, and AUPR metrics.

In this paper, we address the black-box face reconstruction attack with two crucial requirements: query efficiency and generalizability. A practical attack must be query efficient due to limited access to the target black-box model, and the reconstructed face must be generalizable so it can be used to attack other face recognition systems. To this end, we propose a novel face reconstruction attack that optimizes the latent vector of a pre-trained StyleGAN generator. Unlike existing methods, our method is query efficient as neither training nor simultaneous updating of multiple latent vectors is required. Furthermore, we propose a simple initialization scheme that greatly enhances the generalizability of the proposed method. We demonstrate the effectiveness of our method by a thorough evaluation on LFW and CFP-FP datasets across multiple state-of-the-art face recognition models. Project Code: github.com/1ho0jin1/Black-box-Face-Reconstruction.

Smartphones have become indispensable in our lives, even for security-critical tasks. Traditional security measures such as PINs provide only one-time authentication, while biometrics enable continuous authentication in mobile devices. This paper introduces a simple, lightweight, pretrained Transformer dubbed PIEformer for open-set authentication (OSA) of multimodal touchstrokes and gait biometrics. Compared to conventional mobile closed-set authentication, OSA enables more secure and practical authentication, with genuine and impostor users disjoint from the training set. PIEFormer incorporates a novel implicit ensembling mechanism for extracting discriminative embeddings within an open-set environment and enhancing generalization performance. This approach learns multiple diverse sub-embeddings, capturing complementary aspects of biometrics data with minimal computational overhead, allowing Transformers to exhibit robust capabilities in OSA. Our proposed methods demonstrate state-of-the-art results on HMOG and BBMAS datasets, particularly in open-set scenarios compared to closed-set literature, thus bringing mobile biometric authentication closer to real-world applications.

In this paper, we focus on addressing the open-set face identification problem on a few-shot gallery by finetuning. The problem assumes a realistic scenario for face identification, where only a small number of face images is given for enrollment and any unknown identity must be rejected during identification. We observe that face recognition models pretrained on a large dataset and naively fine-tuned models perform poorly for this task. Motivated by this issue, we propose an effective fine-tuning scheme with classifier weight imprinting and exclusive BatchNorm layer tuning. For further improvement of rejection accuracy on unknown identities, we propose a novel matcher called Neighborhood Aware Cosine (NAC) that computes similarity based on neighborhood information. We validate the effectiveness of the proposed schemes thoroughly on large-scale face benchmarks across different convolutional neural network architectures. The source code for this project is available at: https://github.com/1ho0jin1/OSFI-by-FineTuning

Open set recognition (OSR) models need not only discriminate between known classes but also detect unknown class samples unavailable during training. One promising approach is to learn discriminative representations over known classes with strong intra-class similarity and inter-class discrepancy. Then, the powerful class discrimination learned from the known classes can be extended to known and unknown classes. Without appropriate regularization, however, the model may learn representations trivially, collapsing unknown class representations to the known class ones. To resolve this problem, we propose Divergent Angular Representation (DivAR) based on two approaches. Firstly, DivAR maximizes its representational discrimination between known classes via a highly discriminative loss. Secondly, to ensure separation between known and unknown classes in the representation space, DivAR boosts the directional variation of representations over global samples. In addition, self-supervision is leveraged to improve the representation's robustness and extend DivAR to one-class classification. Moreover, unlike other OSR methods that require an extra machinery for inference, DivAR learns and infers in a single module. Extensive experiments on generic image datasets demonstrate the plausibility and effectiveness of DivAR for both OSR and One-Class Classification (OCC) problems.

Realistic low-resolution (LR) face images refer to those captured by the real-world surveillance cameras at extreme standoff distances, thereby LR and poor in quality essentially. Owing to severe scarcity of labeled data, a high-capacity deep convolution neural networks (CNN) is hardly trained to confront the realistic LR face recognition (LRFR) challenge. We introduce in this letter a dual-stream mutual information distillation network (MIND-Net), whereby the non-identity specific mutual information (MI) characterized by generic face features coexistent on realistic and synthetic LR face images are distilled to render a resolution-invariant embedding space for LRFR. For a thorough analysis, we quantify the degree of MI distillation in terms normalized MI index. Our experimental results on the realistic LR face datasets substantiate that the MIND-Net instances assembled from the pre-learned CNNs stand out from the baselines and other state of the arts by a notable margin.

Semi-supervised Learning (SSL) is a classical problem where a model needs to solve classification as it is trained on a partially labeled train data. After the introduction of generative adversarial network (GAN) and its success, the model has been modified to be applicable to SSL. ImprovedGAN as a representative model for GAN-based SSL, it showed promising performance on the SSL problem. However, the inner mechanism of this model has been only partially revealed. In this work, we revisit ImprovedGAN with a fresh look on it based on metric learning. In particular, we interpret ImprovedGAN by general pair weighting, a recent framework in metric learning. Based on this interpretation, we derive two theoretical properties of ImprovedGAN: (i) its discriminator learns to make confident predictions over real samples, (ii) the adversarial interaction in ImprovedGAN constrains the discriminator to decrease the angles between the features of real samples and class weight vectors. The two properties suggest that the adversarial interaction induces the class-wise cluster separation of the features as experimentally verified. Motivated by the findings, we propose a variant of ImprovedGAN, called Intensified ImprovedGAN, where its cluster separation characteristic is improved by two proposed techniques: (a) the unsupervised discriminator loss is scaled up and (b) the generated batch size is enlarged. As a result, I2GAN produces better class-wise cluster separation and, hence, generalization. Extensive experiments on the widely known benchmark data sets verify the effectiveness of our proposed method, showing that its performance is better than or comparable to other GAN based SSL models.

In one-class novelty detection, a model learns solely on the in-class data to single out out-class instances. Autoencoder (AE) variants aim to compactly model the in-class data to reconstruct it exclusively, thus differentiating the in-class from out-class by the reconstruction error. However, compact modeling in an improper way might collapse the latent representations of the in-class data and thus their reconstruction, which would lead to performance deterioration. Moreover, to properly measure the reconstruction error of high-dimensional data, a metric is required that captures high-level semantics of the data. To this end, we propose Discriminative Compact AE (DCAE) that learns both compact and collapse-free latent representations of the in-class data, thereby reconstructing them both finely and exclusively. In DCAE, (a) we force a compact latent space to bijectively represent the in-class data by reconstructing them through internal discriminative layers of generative adversarial nets. (b) Based on the deep encoder's vulnerability to open set risk, out-class instances are encoded into the same compact latent space and reconstructed poorly without sacrificing the quality of in-class data reconstruction. (c) In inference, the reconstruction error is measured by a novel metric that computes the dissimilarity between a query and its reconstruction based on the class semantics captured by the internal discriminator. Extensive experiments on public image datasets validate the effectiveness of our proposed model on both novelty and adversarial example detection, delivering state-of-the-art performance.

Periocular biometric covering the immediate vicinity of human eye is a synergistic alternative to face particularly when the face is masked or occluded. Most present work for periocular recognition in the wild are mainly convolutional neural networks learned based on cross-entropy loss. However, periocular images only capture the least salient face features, and thus suffering from severe intra-class compactness and inter-class dispersion issues for discriminative deep feature learning. Recently, label smoothing regularization (LSR) is discerned capable of diminishing the intra-class variation by minimizing the Kullback-Liebler divergence of a uniform distribution and a network prediction distribution. In this letter, we extend LSR to that of Generalized LSR (GLSR) by learning a pre-task network prediction, in place of the predefined uniform distribution. Extensive experiments on four periocular in the wild datasets disclose that the GSLR-trained networks prevail over the LSR-based counterpart and other most recent the state of the arts. This is supported by our empirical analyses that the embedding periocular features rendered by GLSR results in better class-wise cluster separation than the conventional LSR.

Stacking-based deep neural network (S-DNN) is aggregated with pluralities of basic learning modules, one after another, to synthesize a deep neural network (DNN) alternative for pattern classification. Contrary to the DNNs trained from end to end by backpropagation (BP), each S-DNN layer, that is, a self-learnable module, is to be trained decisively and independently without BP intervention. In this paper, a ridge regression-based S-DNN, dubbed deep analytic network (DAN), along with its kernelization (K-DAN), are devised for multilayer feature relearning from the pre-extracted baseline features and the structured features. Our theoretical formulation demonstrates that DAN/K-DAN relearn by perturbing the intra/interclass variations, apart from diminishing the prediction errors. We scrutinize the DAN/K-DAN performance for pattern classification on datasets of varying domains-faces, handwritten digits, generic objects, to name a few. Unlike the typical BP-optimized DNNs to be trained from gigantic datasets by GPU, we reveal that DAN/K-DAN are trainable using only CPU even for small-scale training sets. Our experimental results show that DAN/K-DAN outperform the present S-DNNs and also the BP-trained DNNs, including multiplayer perceptron, deep belief network, etc., without data augmentation applied.