Abstract While most continual learning methods focus on mitigating forgetting and improving accuracy, they often overlook the critical aspect of network calibration, despite its importance. Neural collapse, a phenomenon where last-layer features collapse to their class means, has demonstrated advantages in continual learning by reducing feature-classifier misalignment. Few works aim to improve the calibration of continual models for more reliable predictions. Our work goes a step further by proposing a novel method that not only enhances calibration but also improves performance by reducing overconfidence, mitigating forgetting, and increasing accuracy. We introduce Sphere-Adaptive Mixup (SAMix), an adaptive mixup strategy tailored for neural collapse-based methods. SAMix adapts the mixing process to the geometric properties of feature spaces under neural collapse, ensuring more robust regularization and alignment. Experiments show that SAMix significantly boosts performance, surpassing SOTA methods in continual learning while also improving model calibration. SAMix enhances both across-task accuracy and the broader reliability of predictions, making it a promising advancement for robust continual learning systems.

Deep anomaly detection aims to provide robust and efficient classifiers for zero-shot (unsupervised, UNS) and few-shot (imbalanced supervised, IMS) settings. However, current models still struggle on edge-case normal samples and are often unable to keep high performance over different scales of anomalies. Additionally, there is a lack of a unified framework that efficiently addresses both UNS and IMS settings. To address these limitations, we present a novel two-stage method which leverages multi-scale normal prototypes during training to compute an anomaly deviation score. First, we employ a novel memory-augmented contrastive learning to jointly learn representations and memory modules across multiple scales. This allows us to effectively capture subtle features of normal data while adapting to varying levels of anomaly complexity. Then, we train an efficient anomaly distance-based detector that computes spatial deviation maps between the learned prototypes and incoming observations. Our model outperforms the SoTA on a wide range of anomalies, including object, style, and local anomalies, as well as industrial inspection and face anti-spoofing, while being on par with SoTa out-of-distribution detectors. Notably, it stands as the first model capable of maintaining exceptional performance across both settings.

Contrastive learning has significantly improved representation quality, enhancing knowledge transfer across tasks in continual learning (CL). However, catastrophic forgetting remains a key challenge, as contrastive based methods primarily focus on “soft relationships” or “softness” between samples, which shift with changing data distributions and lead to representation overlap across tasks. Recently, the newly identified Neural Collapse phenomenon has shown promise in CL by focusing on ““““hard relationships” or “hardness” between samples and fixed proto-types. However, this approach overlooks “softness”, crucial for capturing intra-class variability, and this rigid focus can also pull old class representations toward current ones, increasing forgetting. Building on these insights, we propose Focal Neural Collapse Contrastive$(FNC^{2})$, a novel representation learning loss that effectively balances both soft and hard relationships. Additionally, we introduce the Hardness-Softness Distillation (HSD) loss to progressively preserve the knowledge gained from these relationships across tasks. Our method outperforms state-of-the-art approaches, particularly in minimizing memory reliance. Remarkably, even without the use of memory, our approach rivals rehearsal-based methods, offering a compelling solution for data privacy concerns.

Recent advancements in DETR for object detection have led to innovative techniques to improve efficiency. The challenge is effectively using object queries, which capture content and positional information. Several method increase the number of queries from 300 to 1800, but redundancy is introduced, resulting in only one group being used during inference. Interestingly, we have observed that detection’s with lower confidence levels often result in more accurate bounding box predictions than their higher-confidence counterparts. To exploit this untapped potential, we introduce SG-DETR (Sub-Group Detection Transformer), a plug-and-play method that maximizes the utilisation of object queries. SG-DETR partitions object queries into subgroups with an equal number per subgroup, reducing the impact of negative queries. Additionally, we suppress redundant boxes by clustering those of the same size and position. Our method not only mitigates the issue of negative queries but also enhances various DETR-like models, including Deformable, Conditional and DAB DETR architectures, through experiment on the large-scale COCO2017 dataset.

Falls represent the main risk of injury for elderly people. One-third of adults aged over 65 and half of people over 80 will have at least one fall a year. People at risk should visit a clinical service to detect gait difficulties. Solutions for detecting daily activities are being studied more and more, aiming to develop a complementary method to early detect this type of health risk as effectively as possible. Non-intrusiveness in the person's life for this type of problem is an important criterion, which is why current research is focusing on solutions involving non-conventional imagery such as radar systems. This paper presents an embedded system for classifying daily activities based on the processing of micro-Doppler images. The implementation of the pre-processing chain with a filter enables the acquisition of detailed spectrograms, which proves to be effective in detecting walking. Additionally, by porting it onto the Jetson Orin, it could be possible to accelerate the inference phase of the classification model. We used the ResNet-18 classification method to classify six human activities: Walking, Sitting, Standing, Picking up objects, Drinking water, and Fall events. The results showed that the model is capable of recognising most of the activities on real data.

Modern deepfake detectors have achieved encouraging results, when training and test images are drawn from the same data collection. However, when these detectors are applied to images produced with unknown deepfake-generation techniques, considerable performance degradations are commonly observed. In this paper, we propose a novel deepfake detector, called SeeABLE, that formalizes the detection problem as a (one-class) out-of-distribution detection task and generalizes better to unseen deepfakes. Specifically, SeeABLE first generates local image perturbations (referred to as soft-discrepancies) and then pushes the perturbed faces towards predefined prototypes using a novel regression-based bounded contrastive loss. To strengthen the generalization performance of SeeABLE to unknown deepfake types, we generate a rich set of soft discrepancies and train the detector: (i) to localize, which part of the face was modified, and (ii) to identify the alteration type. To demonstrate the capabilities of SeeABLE, we perform rigorous experiments on several widely-used deepfake datasets and show that our model convincingly outperforms competing state-of-the-art detectors, while exhibiting highly encouraging generalization capabilities. The source code for SeeABLE is available from: https://github.com/anonymous-author-sub/seeable.

This paper addresses the issue of redundancy in object queries in the fully end-to-end transformer-based detector (DETR). We demonstrate that the redundancy arises from the positional dependence of object queries, the multiple interactions between object queries and feature maps through self- and cross-attention, and the unstable Hungarian matching algorithm. To maintain the expressiveness of the object queries, we propose a novel loss that reduces the pairwise correlation of learned object query features across different decoder layers. This novel plug-and-play approach, called DOQ-DETR, can be applied to various DETR variants. Our experiments on the large-scale COCO2017 dataset demonstrate that the proposed training scheme improves various state-of-the-art DETR-like models, including Deformable-, Conditional-, and DAB-DETRs.

Anomaly detection is important in many real-life applications. Recently, self-supervised learning has greatly helped deep anomaly detection by recognizing several geometric transformations. However these methods lack finer features, usually highly depend on the anomaly type, and do not perform well on fine-grained problems. To address these issues, we first introduce in this work three novel and efficient discriminative and generative tasks which have complementary strength: (i) a piece-wise jigsaw puzzle task focuses on structure cues; (ii) a tint rotation recognition is used within each piece, taking into account the colorimetry information; (iii) and a partial re-colorization task considers the image texture. In order to make the re-colorization task more object-oriented than background-oriented, we propose to include the contextual color information of the image border via an attention mechanism. We then present a new out-of-distribution detection function and highlight its better stability compared to existing methods. Along with it, we also experiment different score fusion functions. Finally, we evaluate our method on an extensive protocol composed of various anomaly types, from object anomalies, style anomalies with fine-grained classification to local anomalies with face anti-spoofing datasets. Our model significantly outperforms state-of-the-art with up to 36% relative error improvement on object anomalies and 40% on face anti-spoofing problems.

Deep anomaly detection has become over the years an appealing solution in many fields, and has seen many recent developments. One of the most promising avenues is the use of pretext tasks, which have greatly improved one-class anomaly detection. However this approach is limited by the lack of anomalous samples and carries an important inductive bias. Indeed we could further improve the discrimination power of pretext tasks by incorporating a small set of anomalies, which in practice is often available.To this end, we introduce the concept of learnable pretext tasks, where a pretext task itself is learned to succeed on normal samples while failing on anomalies. To our knowledge it is the first work to explore this direction. By applying the learnable task on a thin plate transform recognition task, our method helps discriminating harder edge-case anomalies and greatly improves anomaly detection. It outperforms state-of-the-art with up to 49% relative error reduction measured with AUROC on various anomaly detection problems including one-vs-all and face presentation attack detection.

Deep anomaly detection has recently seen significant developments to provide robust and efficient classifiers using only a few anomalous samples. Many of those models consist in a first isolated step of representation learning. However, in its current form the learned representation does not encode the semantics of normal sample and anomalies. Indeed during the first step these models will not utilize the available normal/anomaly labels, harming the downstream anomaly detection classifier performances.In the light of this limitation, we introduce a new deep anomaly detector enforcing an anomaly distance constraint on the norm of the representations while using contrastive learning on the direction of the features. This allows it to learn representations well-suited to anomaly detection while avoiding any representation collapse. Moreover, we introduce two strategies of anomaly enriching to improve the robustness of any distance-based anomaly detector. Our model highly improves the state-of-the-art performances on a wide array of anomaly types with up to 74% error relative improvement on object anomalies.

Detecting anomalies using deep learning has become a major challenge over the last years, and is becoming increasingly promising in several fields. The introduction of self-supervised learning has greatly helped many methods including anomaly detection where simple geometric transformation recognition tasks are used. However these methods do not perform well on fine-grained problems since they lack finer features. By combining both high-scale shape features and low-scale fine features in a multi-task framework, our method greatly improves fine-grained anomaly detection. It outperforms state-of-the-art with up to 31% relative error reduction measured with AUROC on various anomaly detection problems including one-vs-all, out-of-distribution detection and face presentation attack detection.

Liveness Detection (LivDet)-Face is an international competition series open to academia and industry. The competition’s objective is to assess and report state-of-the-art in liveness / Presentation Attack Detection (PAD) for face recognition. Impersonation and presentation of false samples to the sensors can be classified as presentation attacks and the ability for the sensors to detect such attempts is known as PAD. LivDet-Face 2021 * will be the first edition of the face liveness competition. This competition serves as an important benchmark in face presentation attack detection, offering (a) an independent assessment of the current state of the art in face PAD, and (b) a common evaluation protocol, availability of Presentation Attack Instruments (PAI) and live face image dataset through the Biometric Evaluation and Testing (BEAT) platform. The competition can be easily followed by researchers after it is closed, in a platform in which participants can compare their solutions against the LivDet-Face winners.

Convolutional neural networks (CNNs) have become the power method for many computer vision applications, including image classification and action recognition. However, they are almost computationally and memory intensive, thus are challenging to use and to deploy on systems with limited resources, except for a few recent networks which were specifically designed for mobile and embedded vision applications such as MobileNet, NASNet-Mobile. In this paper, we present a novel efficient algorithm to compress CNN models to decrease the computational cost and the run-time memory footprint. We propose a strategy to measure the redundancy of parameters based on their relationship using the covariance and correlation criteria, and then prune the less important ones. Our method directly applies to CNNs, both on convolutional and fully connected layers, and requires no specialized software/hardware accelerators. The proposed method significantly reduces the model sizes (up to 70%) and thus computing costs without performance loss on different CNN models (AlexNet, ResNet, and LeNet) for image classification on different datasets (MNIST, CIFAR10, and ImageNet) as well as for human action recognition (on dataset like the UCF101).

Here, the authors introduce a novel system which incorporates the discriminative motion of oriented magnitude patterns (MOMP) descriptor into simple yet efficient techniques. The authors’ descriptor both investigates the relations of the local gradient distributions in neighbours among consecutive image sequences and characterises information changing across different orientations. The proposed system has two main contributions: (i) the authors adopt feature post‐processing principal component analysis followed by vector of locally aggregated descriptors encoding to de‐correlate MOMP descriptor and reduce the dimension in order to speed up the algorithm; (ii) then the authors include the feature selection (i.e. statistical dependency, mutual information, and minimal redundancy maximal relevance) to find out the best feature subset to improve the performance and decrease the computational expense in classification through support vector machine techniques. Experiment results on four data sets, Weizmann (98.4%), KTH (96.3%), UCF Sport (82.0%), and HMDB51 (31.5%), prove the efficiency of the authors’ algorithm.

This letter proposes an effective method to improve object location in Chamfer template matching (CTM) based object detection using image segmentation. In our method, object bounding boxes are iteratively adjusted to fit with the object images obtained from image segmentation in a probabilistic model. The proposed method was tested with state-of-the-art CTM-based object detectors. Experimental results have shown the proposed method improved the location accuracy of the object detectors and reduce the false alarms rate.

Regarding texture features, Local-based methods such as Local Binary Pattern (LBP) and its variants are computationally efficient high-performing but sensitive to noise, and suffering global structure information loss. By contrast, filter-based counterparts, the Scattering Transform for instance, are tolerant to noise and translation but often lack of small local structure information. In this paper we propose an integration of those to take full advantages of both local and global features. In this way, LBP is used for extracting local features while the Scattering Transform feature plays the role of a global descriptor. In addition to the combination of these two state-of-the-art features, we further integrate a new preprocessing technique called biologically-inspired filtering (BF) as well as an efficient PCA classifier. Intensive experiments conducted on many texture benchmarks such as CUReT, UIUC, KTH-TIPS2b, and OUTEX show that our combined method not only outweighs each one which stands alone but also competes with state-of-the-art on the experimented datasets.

This paper investigates the classification of photographic paper textures using visual descriptors. Such classification is called fine grain due to the very low inter-class variability. We propose a novel image representation for photographic paper texture categorization, relying on the incorporation of a powerful local descriptor into an efficient higher-order model deviation where texture is represented by computing statistics on the occurrences of specific local visual patterns. We perform an evaluation on two different challenging datasets of photographic paper textures and show such advanced methods indeed outperforms existing descriptors.

The intent of 3D-model classification is to find categories of similar objects according to their shapes. This task is a challenging and important problem in 3D-mining and shape processing. In this paper, we present a novel method to categorize 3D-objects based on view-based descriptors. The proposed method goes into two stages. The first stage corresponds to the training in which 3D-objects in the same category are processed and a set of representative 2D views is selected, The second stage corresponds to the labelling in which unknown objects are classified using a belief based classifier. The experimental results obtained on the Shrec07 datasets show that the system efficiently performs in categorizing 3D-models.

A new action model is proposed, by revisiting local binary patterns (LBP) for dynamic texture models, applied on trajectory beams calculated on the video. The use of semi-dense trajectory field allows to dramatically reduce the computation support to essential motion information, while maintaining a large amount of data to ensure robustness of statistical bag of features action models. A new binary pattern, called Spatial Motion Pattern (SMP) is proposed, which captures self-similarity of velocity around each tracked point (particle), along its trajectory. This operator highlights the geometric shape of rigid parts of moving objects in a video sequence. SMPs are combined with basic velocity information to form the local action primitives. Then, a global representation of a space × time video block is provided by using hierarchical blockwise histograms, which allows to efficiently represent the action as a whole, while preserving a certain level of spatiotemporal relation between the action primitives. Inheriting from the efficiency and the invariance properties of both the semi-dense tracker Video extruder and the LBP-based representations, the method is designed for the fast computation of action descriptors in unconstrained videos. For improving both robustness and computation time in the case of high definition video, we also present an enhanced version of the semi-dense tracker based on the so-called super particles, which reduces the number of trajectories while improving their length, reliability and spatial distribution.

A novel direction for efficiently describing face images is proposed by exploring the relationships between both gradient orientations and magnitudes of different local image structures. Presented in this paper are not only a novel feature set called patterns of orientation difference (POD) but also several improvements to our previous algorithm called patterns of oriented edge magnitudes (POEM). The whitened principal component analysis (PCA) dimensionality reduction technique is applied upon both the POEM- and POD-based representations to get more compact and discriminative face descriptors. We then show that the two methods have complementary strength and that by combining the two algorithms, one obtains stronger results than either of them considered separately. By experiments carried out on several common benchmarks, including the FERET database with both frontal and nonfrontal images as well as the very challenging LFW data set, we prove that our approach is more efficient than contemporary ones in terms of both higher performance and lower complexity.

Describing efficiently faces is a task of mounting importance. Most of existing algorithms do not address all the three criteria, i.e., robustness, distinctiveness and low computational cost. Inspired by recent features so-called POEM (Patterns of Oriented Edge Magnitudes) which is argued balancing well the three concerns, we first provide an improvement to it and then propose novel features so-called self-POEM considering the relations between edge distributions along different directions of one region itself. Self-POEM provides the complementary strength which is not present in POEM. Combining them together, a more robust algorithm is obtained and by experiments we prove that our approach is more efficient than contemporary ones.

A good feature descriptor is desired to be discriminative, robust, and computationally inexpensive in both terms of time and storage requirement. In the domain of face recognition, these properties allow the system to quickly deliver high recognition results to the end user. Motivated by the recent feature descriptor called Patterns of Oriented Edge Magnitudes (POEM), which balances the three concerns, this paper aims at enhancing its performance with respect to all these criteria. To this end, we first optimize the parameters of POEM and then apply the whitened principal-component-analysis dimensionality reduction technique to get a more compact, robust, and discriminative descriptor. For face recognition, the efficiency of our algorithm is proved by strong results obtained on both constrained (Face Recognition Technology, FERET) and unconstrained (Labeled Faces in the Wild, LFW) data sets in addition with the low complexity. Impressively, our algorithm is about 30 times faster than those based on Gabor filters. Furthermore, by proposing an additional technique that makes our descriptor robust to rotation, we validate its efficiency for the task of image matching.

Good face recognition system is one which quickly de- livers high accurate results to the end user. For this purpose, face representation must be robust, discriminative and also of low computational cost in both terms of time and space. Inspired by recently proposed feature set so-called POEM (Patterns of Oriented Edge Magnitudes) which considers the relationships between edge distributions of different image patches and is argued balancing well the three concerns, this work proposes to further exploit patterns of both orientations and magnitudes for building more efficient algorithm. We first present novel features called Patterns of Dominant Orientations (PDO) which consider the relationships between "dominant" orientations of local image regions at different scales. We also propose to apply the whitened PCA technique upon both the POEM and PDO based representations to get more compact and discriminative face descriptors. We then show that the two methods have complementary strength and that by combining the two descriptors, one obtains stronger results than either of them considered separately. By experiments carried out on several common benchmarks, including both frontal and non- frontal FERET as well as the AR datasets, we prove that our approach is more efficient than contemporary ones.

In this paper we present a new architecture for face recognition with a single reference image, which completely separates the training process from the recognition process. In the training stage, by using a database containing various individuals, the spatial relations between face components are represented by two Hidden Markov Models (HMMs), one modeling within-subject similarities, and the other modeling inter-subject differences. This allows us during the recognition stage to take a pair of face images, neither of which has been seen before, and to determine whether or not they come from the same individual. Whilst other face-recognition HMMs use Maximum Likelihood criterion, we test our approach using both Maximum Likelihood and Maximum a Posteriori (MAP) criterion, and find that MAP provides better results. Importantly, the training database can be entirely separated from the gallery and test images: this means that adding new individuals to the system can be done without re-training. We present results based upon models trained on the FERET training dataset, and demonstrate that these give satisfactory recognition rates on both the FERET database itself and more impressively the unseen AR database. When compared to other HMM based face recognition techniques, our algorithm is of much lower complexity due to the small size of our observation sequence.

Illumination variations that might occur on face images degrade the performance of face recognition systems. In this paper, we propose a novel method of illumination normalization based on retina modeling by combining two adaptive nonlinear functions and a Difference of Gaussians filter. The proposed algorithm is evaluated on the Yale B database and the Feret illumination database using two face recognition methods: PCA based and Local Binary Pattern based (LBP). Experimental results show that the proposed method achieves very high recognition rates even for the most challenging illumination conditions. Our algorithm has also a low computational complexity.