2D-to-3D lifting is a fundamental approach in 3D human pose estimation (3DHPE). This task is crucial in applications, including motion analysis and virtual reality. While Graph Convolutional Networks (GCNs) have demonstrated effectiveness in capturing spatial relationships in human skeletons, they suffer from over-smoothing and limited receptive fields. Transformer-based models provide global context but struggle with local feature extraction and computational efficiency. To address these challenges, we propose ADGT, a novel parallel GCN-transformer architecture combining the strengths of both approaches. Our method introduces three key innovations: Hop-Wise Scalable Adaptive GCN to refine local feature extraction, Attention-Based Local Feature Extractor to enhance the integration of local and global representations, and Register-Based Transformer Enhancement to improve feature separation. Extensive experiments on Human3.6M and MPI-INF-3DHP datasets demonstrate ADGT achieves state-of-the-art performance among frame-based methods while maintaining computational efficiency. These results highlight the potential of ADGT for real-time applications requiring accurate and efficient 3DHPE. The code is available at https://github.com/sYANGunique1111/ADGT .

Deep anomaly detection aims to provide robust and efficient classifiers for zero-shot (unsupervised, UNS) and few-shot (imbalanced supervised, IMS) settings. However, current models still struggle on edge-case normal samples and are often unable to keep high performance over different scales of anomalies. Additionally, there is a lack of a unified framework that efficiently addresses both UNS and IMS settings. To address these limitations, we present a novel two-stage method which leverages multi-scale normal prototypes during training to compute an anomaly deviation score. First, we employ a novel memory-augmented contrastive learning to jointly learn representations and memory modules across multiple scales. This allows us to effectively capture subtle features of normal data while adapting to varying levels of anomaly complexity. Then, we train an efficient anomaly distance-based detector that computes spatial deviation maps between the learned prototypes and incoming observations. Our model outperforms the SoTA on a wide range of anomalies, including object, style, and local anomalies, as well as industrial inspection and face anti-spoofing, while being on par with SoTa out-of-distribution detectors. Notably, it stands as the first model capable of maintaining exceptional performance across both settings.

Recent works have demonstrated promising performances of neural networks on hyperbolic spaces and symmetric positive definite (SPD) manifolds. These spaces belong to a family of Riemannian manifolds referred to as symmetric spaces of noncompact type. In this paper, we propose a novel approach for developing neural networks on such spaces. Our approach relies on a unified formulation of the distance from a point to a hyperplane on the considered spaces. We show that some existing formulations of the point-to-hyperplane distance can be recovered by our approach under specific settings. Furthermore, we derive a closed-form expression for the point-to-hyperplane distance in higher-rank symmetric spaces of noncompact type equipped with G-invariant Riemannian metrics. The derived distance then serves as a tool to design fully-connected (FC) layers and an attention mechanism for neural networks on the considered spaces. Our approach is validated on challenging benchmarks for image classification, electroencephalogram (EEG) signal classification, image generation, and natural language inference.

Riemannian symmetric spaces (RSS) such as hyperbolic spaces and symmetric positive definite (SPD) manifolds have become popular spaces for representation learning. In this paper, we propose a novel approach for building discriminative neural networks on Siegel spaces, a family of RSS that is largely unexplored in machine learning tasks. For classification applications, one focus of recent works is the construction of multiclass logistic regression (MLR) and fully-connected (FC) layers for hyperbolic and SPD neural networks. Here we show how to build such layers for Siegel neural networks. Our approach relies on the quotient structure of those spaces and the notation of vector-valued distance on RSS. We demonstrate the relevance of our approach on two applications, i.e., radar signal classification and node classification. Our results successfully demonstrate state-of-the-art performance across all datasets.

Deep neural networks (DNNs) on Riemannian manifolds have garnered increasing interest in various applied areas. For instance, DNNs on spherical and hyperbolic manifolds have been designed to solve a wide range of computer vision and nature language processing tasks. One of the key factors that contribute to the success of these networks is that spherical and hyperbolic manifolds have the rich algebraic structures of gyrogroups and gyrovector spaces. This enables principled and effective generalizations of the most successful DNNs to these manifolds. Recently, some works have shown that many concepts in the theory of gyrogroups and gyrovector spaces can also be generalized to matrix manifolds such as Symmetric Positive Definite (SPD) and Grassmann manifolds. As a result, some building blocks for SPD and Grassmann neural networks, e.g., isometric models and multinomial logistic regression (MLR) can be derived in a way that is fully analogous to their spherical and hyperbolic counterparts. Building upon these works, in this paper, we design fully-connected (FC) and convolutional layers for SPD neural networks. We also develop MLR on Symmetric Positive Semi-definite (SPSD) manifolds, and propose a method for performing backpropagation with the Grassmann logarithmic map in the projector perspective. We demonstrate the effectiveness of the proposed approach in the human action recognition and node classification tasks.

Anomaly detection is important in many real-life applications. Recently, self-supervised learning has greatly helped deep anomaly detection by recognizing several geometric transformations. However these methods lack finer features, usually highly depend on the anomaly type, and do not perform well on fine-grained problems. To address these issues, we first introduce in this work three novel and efficient discriminative and generative tasks which have complementary strength: (i) a piece-wise jigsaw puzzle task focuses on structure cues; (ii) a tint rotation recognition is used within each piece, taking into account the colorimetry information; (iii) and a partial re-colorization task considers the image texture. In order to make the re-colorization task more object-oriented than background-oriented, we propose to include the contextual color information of the image border via an attention mechanism. We then present a new out-of-distribution detection function and highlight its better stability compared to existing methods. Along with it, we also experiment different score fusion functions. Finally, we evaluate our method on an extensive protocol composed of various anomaly types, from object anomalies, style anomalies with fine-grained classification to local anomalies with face anti-spoofing datasets. Our model significantly outperforms state-of-the-art with up to 36% relative error improvement on object anomalies and 40% on face anti-spoofing problems.

In deep metric learning, the training procedure relies on sampling informative tuples. However, as the training procedure progresses, it becomes nearly impossible to sample relevant hard negative examples without proper mining strategies or generation-based methods. Recent work on hard negative generation have shown great promises to solve the mining problem. However, this generation process is difficult to tune and often leads to incorrectly labeled examples. To tackle this issue, we introduce MIRAGE, a generation-based method that relies on virtual classes entirely composed of generated examples that act as buffer areas between the training classes. We empirically show that virtual classes significantly improve the results on popular datasets (Cub-200-2011 and Cars-196) compared to other generation methods.

Deep anomaly detection has become over the years an appealing solution in many fields, and has seen many recent developments. One of the most promising avenues is the use of pretext tasks, which have greatly improved one-class anomaly detection. However this approach is limited by the lack of anomalous samples and carries an important inductive bias. Indeed we could further improve the discrimination power of pretext tasks by incorporating a small set of anomalies, which in practice is often available.To this end, we introduce the concept of learnable pretext tasks, where a pretext task itself is learned to succeed on normal samples while failing on anomalies. To our knowledge it is the first work to explore this direction. By applying the learnable task on a thin plate transform recognition task, our method helps discriminating harder edge-case anomalies and greatly improves anomaly detection. It outperforms state-of-the-art with up to 49% relative error reduction measured with AUROC on various anomaly detection problems including one-vs-all and face presentation attack detection.

Deep anomaly detection has recently seen significant developments to provide robust and efficient classifiers using only a few anomalous samples. Many of those models consist in a first isolated step of representation learning. However, in its current form the learned representation does not encode the semantics of normal sample and anomalies. Indeed during the first step these models will not utilize the available normal/anomaly labels, harming the downstream anomaly detection classifier performances.In the light of this limitation, we introduce a new deep anomaly detector enforcing an anomaly distance constraint on the norm of the representations while using contrastive learning on the direction of the features. This allows it to learn representations well-suited to anomaly detection while avoiding any representation collapse. Moreover, we introduce two strategies of anomaly enriching to improve the robustness of any distance-based anomaly detector. Our model highly improves the state-of-the-art performances on a wide array of anomaly types with up to 74% error relative improvement on object anomalies.

Detecting anomalies using deep learning has become a major challenge over the last years, and is becoming increasingly promising in several fields. The introduction of self-supervised learning has greatly helped many methods including anomaly detection where simple geometric transformation recognition tasks are used. However these methods do not perform well on fine-grained problems since they lack finer features. By combining both high-scale shape features and low-scale fine features in a multi-task framework, our method greatly improves fine-grained anomaly detection. It outperforms state-of-the-art with up to 31% relative error reduction measured with AUROC on various anomaly detection problems including one-vs-all, out-of-distribution detection and face presentation attack detection.

Learning an effective similarity measure between image representations is key to the success of recent advances in visual search tasks (e.g. verification or zero-shot learning). Although the metric learning part is well addressed, this metric is usually computed over the average of the extracted deep features. This representation is then trained to be discriminative. However, these deep features tend to be scattered across the feature space. Consequently, the representations are not robust to outliers, object occlusions, background variations, etc. In this paper, we tackle this scattering problem with a distribution-aware regularization named HORDE. This regularizer enforces visually-close images to have deep features with the same distribution which are well localized in the feature space. We provide a theoretical analysis supporting this regularization effect. We also show the effectiveness of our approach by obtaining state-of-the-art results on 4 well-known datasets (Cub-200-2011, Cars-196, Stanford Online Products and Inshop Clothes Retrieval).

Learning rich and compact representations is an open topic in many fields such as object recognition or image retrieval. Deep neural networks have made a major breakthrough during the last few years for these tasks but their representations are not necessary as rich as needed nor as compact as expected. To build richer representations, high order statistics have been exploited and have shown excellent performances, but they produce higher dimensional features. While this drawback has been partially addressed with factorization schemes, the original compactness of first order models has never been retrieved, or at the cost of a strong performance decrease. Our method, by jointly integrating codebook strategy to factorization scheme, is able to produce compact representations while keeping the second order performances with few additional parameters. This formulation leads to state-of-the-art results on three image retrieval datasets.

Pulse oximetry is one of the most commonly employed monitoring modalities in critical care setting. Conventional pulse oximeters use two leds at different wavelengths and a photodiode to estimate blood oxygen saturation noninvasively, based in the difference of absorption coefficients between hemoglobin and deoxyhemoglobin. Nevertheless, factors such as low oxygen saturations, skin melanin content, nail polish presence or led wavelength displacement, modify differently the expected light absorbance for both LEDs. To address these issues, a novel approach combining a single led and a Buried Quad Junction photodetector is proposed. With this fundamental modification of the pulse oximetry principle, errors associated with the aforementioned modifying effects are expected to be reduced. The preliminary results show that there is a correlation of 0.84, -0.93, 0.91 and -0.88 for channels 1, 2, 3 and 4 respectively between our proposed method and the response from the measuring reference.

Fibrosis represents an open issue for medium to long-term active implants given that this biological medium surrounds the stimulation electrodes and can impact or modify the performances of the system. For this reason, Embedded Impedance Spectroscopy techniques has been investigated these last years to sense the fibrosis. The following article introduces a new paradigm for Electrochemical Impedance Spectroscopy (EIS) derived from multi-carrier digital communication methods. Due to its properties of flat spectrum and fast generation the Orthogonal-Frequency Division Multiplexing (OFDM) technique for EIS seems to be a real alternative to traditional ones. This article focuses on this approach and defines its performances on gold electrodes used for in-vitro experiments. An embedded implementation is also presented. This designed prototype allows to measure a medium with an error between 2% and 3%, when stimulating with a 16 or 32 tones multitone signal and a sampling frequency of 12KHz.

The following article introduces a wireless and portable system for Electrochemical Impedance Spectroscopy (EIS) sensing based on the Orthogonal Frequency-Division Modulation (OFDM). This technique is derived from multi-carrier digital communication methods and due to its properties of flat spectrum, fast generation and low-foot print memory, the OFDM technique for EIS seems to be a real alternative to traditional ones. The manufacture of the OFDM-EIS system is under the framework of the electrical sensing of fibrosis induced by medium to long-term active implants, given that the detection of fibrous tissues surrounding the electrode, that usually affect the operation of the implant, is an open research problem. Because of this, this article defines its performance on living cells under in-vitro experimentation where the proliferation of cells are matched with the results. Furthermore, traditional EIS techniques, such as frequency sweep and multi-sine are compared to OFDM-EIS.

Most image retrieval methods use global features that aggregate local distinctive patterns into a single representation. However, the aggregation process destroys the relative spatial information by considering orderless sets of local descriptors. We propose to integrate relative spatial information into the aggregation process by taking into account co-occurrences of local patterns in a tensor framework. The resulting signature called Improved Spatial Tensor Aggregation (ISTA) is able to reach state of the art performances on well known datasets such as Holidays, Oxford5k and Paris6k.

Real-time monitoring of hematophagous diptera (such as mosquitoes) populations in the field is a crucial challenge to foresee vaccination campaigns and to restrain potential diseases spreading. However, current methods heavily rely on costly DNA extraction which is destructive, costly, time consuming and requires experts. The contributions of this work are: 1) the usage of a new type of imaging, named Wing Interference Patterns (WIPs), which is non-destructive and easier to produce during in the field experiments; 2) a deep learning architecture which is optimized for very low computation cost, memory usage and a short inference time; 3) the use of a dataset of more than 50 medically important species of hematophagous diptera with more than 3000 images of WIPs. With these contributions, we demonstrate that WIPs are an excellent medium to automatically recognize a large amount of hematophagous diptera species with very high accuracy and low computational cost convolutional neural network.

In this article, our concern is the early diagnosis of colorectal cancer from a computeraided detection point of view in order to help physicians in their diagnosis during the gold standard examination: optical video colonoscopy. Since many years, some methods and materials have been developed to reduce the polyp miss rate and to improve detection capabilities. Nevertheless, the real challenge lies in the real-time use of these methods. In this context, more precisely, we focus our attention on the hardware implementation of a previous method we recently introduced in the literature for real-time detection of colorectal polyps, lesions that may degenerate into cancer. This implementation is subject to three performance criteria: real-time processing capabilities, detection rate and necessary computational resources. Six different platforms were tested and compared. If we noticed that only workstation computers are able to perform the detection with a good tradeoff between the three aforementioned criteria, possibilities of architecture optimizations are also identified and discussed in order to achieve real-time performance on platforms with low available computational resources like Raspberry Pi for instance. This latter issue is of major importance for possible integration of the detection algorithm inside smallconnected object like videocapsule, a promising alternative to standard colonoscopy.

A wireless and portable system, consisting in an electronic board and a computer software graphical interface, is presented in this article as a feasible way to do in-vitro electric bioimpedance spectroscopy of cells. It is designed to work inside a culture chamber performing impedance measurement in the frequency range of 64Hz to 200KHz. The board is equipped with a Bluetooth Low Energy (BLE) module allowing it to be wirelessly controlled. The software interface (also called ISMI) is coded with all required functionalities to manage the parameters of the board such as start frequency and sweep, measuring intervals, data acquisition and visualization and also a function to perform automatic measuring between desired time intervals during whole day for many days. In addition, the electrodes used for the measurements of cells are characterized giving an impedance magnitude between 103 to 105 ohms in a frequency range of 300Hz to 100KHz and a maximum voltage without considerable electrode deterioration of 120mVpeak. This information is used in the calibration of the ISMI system giving a measurement accuracy above 98% when compared with simulation results and with a reference instrument. The proposed system is an introductory step in the study of cells related to fibrous tissue induced by implants showing to be a viable and reproducibility-improving approach for such analysis. This first prototype provides information regarding the requirements for the design of an integrated version for embedded applications.

Colonoscopy is the gold standard for colon cancer screening though some polyps are still missed, thus preventing early disease detection and treatment. Several computational systems have been proposed to assist polyp detection during colonoscopy but so far without consistent evaluation. The lack of publicly available annotated databases has made it difficult to compare methods and to assess if they achieve performance levels acceptable for clinical use. The Automatic Polyp Detection sub-challenge, conducted as part of the Endoscopic Vision Challenge (http://endovis.grand-challenge.org) at the international conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2015, was an effort to address this need. In this paper, we report the results of this comparative evaluation of polyp detection methods, as well as describe additional experiments to further explore differences between methods. We define performance metrics and provide evaluation databases that allow comparison of multiple methodologies. Results show that convolutional neural networks are the state of the art. Nevertheless, it is also demonstrated that combining different methodologies can lead to an improved overall performance.

This article deals with statistical region-based active contour segmentation using the alpha-divergence family as similarity measure between the density probability functions of the background and the object regions of interest. Following previous publications on that topic, main originality of this contribution is in the proposed joint optimization of the energy steering the evolution of the active curve and the parameter alpha related to the metric of the divergence and closely related to the statistical luminance distribution of the data. Experiments are shown on both synthetic noisy and textured data as well as on real images (natural and medical ones). We show that the joint optimization process leads to satisfying results for every targeted tasks: above all, it is shown that the proposed approach overcome classic statistical-based region active contour approach using Kullback-Leibler divergence as similarity measure, that can stuck in local extrema during the usual optimization process.

Wireless capsule endoscopy (WCE) is commonly used for noninvasive gastrointestinal tract evaluation, including the identification of polyps. In this paper, a new multimodal embeddable method for polyp detection and classification in wireless capsule endoscopic images was developed and tested. The multimodal wireless capsule used both 2D and 3D data to identify possible polyps and to deliver cancerous information of the polyps based on 3D geometric features. Possible polyps within the image (2D) were extracted using simple geometric shape features and, in a second step, the candidate regions of interest (ROI) were evaluated with a boosting-based method using textural features. Once the 2D identification of polyps has been performed, the two-class (“malignant” or “begnin”) classification of the polyps is achieved using the 3D parameters computed from the preselected ROI using an active stereo vision system. At this stage, a Support Vector Machine (SVM) classifier is used to proceed to the final classification and to make possible a pre diagnosis. The new proposed multimodal approach based on 2D-3D feature extraction improves WCE capabilities to identify and classify polyps: The boosting-based polyp classification demonstrated a sensitivity of 91%, a specificity of 95% and a false detection rate of 4.8% on a database composed of 300 hundred positive examples and 1200 negative ones; Considering the 3D performance, a large scale demonstrator was evaluated and tested to perform in vitro experiments on an ad hoc polyp database. The performance of the 3D approach achieved a correct classification rate (malignant or benin) of approximately 95%.

In this article, a complete original framework for unsupervised statistical region-based active contour segmentation is proposed. More precisely, the method is based on the maximization of alpha-divergences between non-paramterically estimated probability density functions (PDFs) of the inner and outer regions defined by the evolving curve. We define the variational context associated to distance maximization in the particular case of alpha-divergences and provide the complete derivation of the partial differential equation leading the segmentation. Results on synthetic data, corrupted with a high level of Gaussian and Poisson noises, but also on clinical X-ray images show that the proposed unsupervised method improves standard approaches of that kind.

The paper reports on a novel method for reconstruction of cellular features including cell nuclei and cellular boundaries from actin tagged fluorescence confocal microscopy images. Such reconstruction can provide spatial context for subsequent quantitative analysis of changes to actin organisation and cell morphology in both controlled and stressed cell cultures. The proposed method is fully automatic and is formulated within active contour multiphase level set framework. The derived level set evolution PDEs combine previously proposed curvature and advection flows with propagation flow defined by specially designed set of geodesic distance maps. Additionally the proposed PDEs include additional components to impose known inclusion/exclusion topological constraints between cellular structures. The paper gives an overview of the proposed methodology as well as reports on initial results obtained for monolayer of human prostate cells (PNT2) culture visualised using acting tagged fluorescence confocal microscopy.

This paper describes a novel method for active contour segmentation based on foreground/background alpha-divergence histogram distance measure. In recent years a number of variational segmentation techniques have been proposed for a region based active contour segmentation utilising different distance measures between probability density functions (PDFs) describing foreground and background regions. The most common techniques use χ2, Hellinger/Bhattacharya distances or Kullback-Leibler divergence. In this paper, it is proposed to generalize these methods by using the alpha-divergences distance function. This distance function depending on the selected value of its parameter encompasses mentioned above classical distances. The paper defines a partial differential equation, associated with alpha-divergence variational criterion, that governs the iterative deformations of the active contour. The experimental results on a synthetic data demonstrate that the proposed method outperforms previously proposed histogram based methods in terms of segmentation accuracy and robustness with respect to type and level of noise. The potential of the proposed technique for segmentation of cellular structures in fluorescence confocal microscopy data is also illustrated.

The noninvasive evaluation of the cardiac function presents a great interest for the diagnosis of cardiovascular diseases. Cardiac tagged MRI allows the measurement of anatomical and functional myocardial parameters. This protocol generates a dark grid which is deformed with the myocardium. Tracking the grid allows the displacement estimation in the myocardium. The work described in this paper aims to automate the myocardial contours detection and the following of the grids of tags on short-axis and long-axis time sequences, in order to firstly optimize the 3D+T study of the parietal contractions and secondly make possible its clinical use. The method we have developed for endocardial and epicardial contours detection is based on the use of texture analysis and active contours models. Texture analysis allows us to define energy maps more efficient than those usually used in active contours methods where attractor is often based on gradient and which were useless in our case of study. The follow-up of the grid of tags that we have implemented is based on a grid of active contours (B-snakes) which part of the energy is calculated in the Fourier's domain. The results obtained with our method is fully automatic and correct on short-axis as well as on long-axis sequences, when previous works on cardiac tagged MR images analysis always used manual contours detection.