Recently, significant efforts were made towards Variational Autoencoder (VAE) -based prototypical Self Explainable Models (SEM) for image classification. The principle is to learn class-specific prototypes that can be projected back into the image space thanks to the decoding branch of a VAE. However, existing VAE-based SEM fail to represent properly the distribution of training samples in the embedding space requiring to define specific additional constraints as diversity or orthogonality. In this work, we propose to define the prototypes as the components of a Gaussian Mixture of VAE (GMVAE) that is an approximation of the distribution of training samples. We show that this definition allows to produce relevant and diverse prototypes providing a probabilistic explanation of the model without assigning prototypes to a specific class. We support our definition with extensive experimentation and comparison with previous self-explainable approaches.

In this paper, we introduce Slope-Track. Slope-Track is a novel multiple object tracking (MOT) dataset designed to reflect the complexities of real ski slope environments. The dataset has over 96,000 frames collected from 10 different ski resorts under various weather and visibility conditions. Slope-Track addresses significant challenges in slope monitoring, including small object sizes, occlusions, fast and irregular motion, and low appearance consistency. It is densely annotated with bounding boxes and object identities, facilitating the evaluation of detection and tracking algorithms. We analyze the dataset’s characteristics comparing it to the existing MOT datasets. The results demonstrate that Slope-Track encapsulates a combination of challenges found in other datasets. Additionally, we benchmark a range of existing tracking algorithms and propose a new module that improves motion-based association by dealing with the specific shape of trajectories along ski slopes. Our results demonstrate that incorporating appearance features can have a mixed impact, depending on how they are used within each tracking algorithm. In contrast, motion-based methods and spatial association strategies show more reliable performance. Overall, we provide a challenging benchmark for evaluating and improving multi-object tracking systems in real-world outdoor environments. The dataset and code can be found at https://slopetrack.github.io/ .

Deep learning methods are now achieving strong results for segmentation tasks, and the standard metric for evaluating methods is the Intersection over Union (IOU). However, we show in this paper that IOU is not efficient in evaluating the quality of segmentation for electron tomography (ET) images of zeolites. We perform a physics-oriented evaluation to ensure that the segmentation results yield coherent physical measures. We also formalize Mixed Supervised / Self-Supervised Contrastive Learning Segmentation (M3S-CLS), a semi-supervised approach using a contrastive learning approach that uses expert annotations to train the neural network model. A detailed comparison of this method with a standard cross-entropy-based model is provided. In addition, we publish a database of five fully segmented ET volumes along with corresponding baseline results. The code and the database is available at https://gitlab.univ-st-etienne.fr/labhc-iscv/M3S-CLS .

Deep learning for automated cell imaging analysis has become a tool of choice to process large amounts of data. But many of these methods lack explainability, slowing down their deployment for tasks such as diagnosis. We present a prototype-based framework to analyze structural changes which addresses the specific challenges of explainability in the context of cell imaging. Our method relies on classification between two distinct cell populations in a weakly supervised context where no label for individual cells is available. Our model extracts typical features from each population, representing intra-cellular structure, and provides an explanation on its classification decision by creating visualization of the local textures corresponding to the structures of interest. We show a real application where it effectively highlights a change in the organization of the actin content of the cells.

Abstract Infant pose estimation is crucial in different clinical applications, including preterm automatic general movements assessment. Recent infant pose estimation methods are limited by a lack of real clinical data and are mainly focused on 2D detection. We introduce a stereoscopic system for infants’ 3D pose estimation, based on fine-tuning state-of-the-art 2D human pose estimation networks on a large, real, and manually annotated dataset of infants’ images. Our dataset contains over 88k images, collected from 175 videos from 53 premature infants born <33 weeks of gestational age (GA), acquired within the Neonatology department of the Centre Hospitalier Universitaire de Saint Etienne, France, between 32 and 41 weeks of GA. This framework significantly reduced the pose estimation error compared to existing 2D infant pose estimation networks. It achieved a mean error of 1.72 cm on 18000 stereoscopic images in the 3D pose estimation task. This framework is the first 3D pose estimation tool dedicated to preterm infants hospitalized in the Neonatal Unit that does not depend on any visual markers or infrared cameras.

Most recent methods of image augmentation and prediction are building upon the deep learning paradigm. A careful preparation of the image dataset and the choice of a suitable network architecture are crucial steps to assess the desired image features and, thence, achieve accurate predictions. We first propose to help the learning process by adding structural information with specific distance transform to the input image data . To handle cases with limited number of training samples, we propose a patch-based procedure with a stratified sampling method at inference. We validate our approaches on two image datasets, corresponding to two different tasks. The ability of our method to segment and predict images is investigated through the ISBI 2012 segmentation challenge dataset and generated electric field masks, respectively. The obtained results are evaluated using appropriate metrics: VRand for image segmentation and SSIM, UIQ and PSNR for image prediction. The proposed techniques demonstrate that the established framework is a reliable estimation method that could be used for a wide range of applications.

Multi-object tracking is a difficult problem underlying many computer vision applications. In this work, we focus on sediment transport experiments in a flow were sediments are represented by spherical calibrated beads. The aim is to track all beads over long time sequences to obtain sediment velocities and concentration. Classical algorithms used in fluid mechanics fail to track the beads over long sequences with a high precision because they incorrectly handle both miss-detections and detector imprecision. Our contribution is to propose a particle filter-based algorithm including an adapted multiple motion model. Additionally, this algorithm integrates several improvements to account for the lack of precision of the detector. The evaluation was made using a test sequence with a dedicated ground-truth. The results show that the method outperforms state-of-the-art concurrent algorithms.

Color description is a challenging task because of large variations in RGB values which occur due to scene accidental events, such as shadows, shading, specularities, illuminant color changes, and changes in viewing geometry. Traditionally, this challenge has been addressed by capturing the variations in physics-based models, and deriving invariants for the undesired variations. The drawback of this approach is that sets of distinguishable colors in the original color space are mapped to the same value in the photometric invariant space. This results in a drop of discriminative power of the color description. In this paper we take an information theoretic approach to color description. We cluster color values together based on their discriminative power in a classification problem. The clustering has the explicit objective to minimize the drop of mutual information of the final representation. We show that such a color description automatically learns a certain degree of photometric invariance. We also show that a universal color representation, which is based on other data sets than the one at hand, can obtain competing performance. Experiments show that the proposed descriptor outperforms existing photometric invariants. Furthermore, we show that combined with shape description these color descriptors obtain excellent results on four challenging datasets, namely, PASCAL VOC 2007, Flowers-102, Stanford dogs-120 and Birds-200.

International audience

In this paper, we introduce two new morphological transforms for pattern matching in gray scale images. They rely on a profiling approach and are defined in the context of mathematical morphology. The first transform allows to detect all occurrences of a single pattern in an image, which justifies the name SOMP (single object matching using probing). It is shown to have the properties of a metric and therefore returns a measure of similarity between the search image and the reference pattern. Other properties relative to noise and computation time are highlighted. The second transform MOMP (multiple objects matching using probing) offers the ability to locate multiple patterns simultaneously. It is particularly suited to the detection of objects varying in size and with noisy distortion. Some results are presented for both transforms.

We present a scheme for feature points detection on a grey level image. The use of this algorithm does not imply the segmentation of the objects but only the detection of their edges. This task is achieve through the study of the behavior of wavelet coefficients across scales. Once the edges are detected, the high curvature points along them are localized. These points are extracted as the transition points of a gradient phase signal, with a wavelet based algorithm. Finally, our algorithm is able to select between the feature points a set of the most representative ones through the determination of the point type and the measurement of the local curvature. We prove the efficiency of our algorithm on three examples and we discuss the robustness of our algorithm versus classical ones.

A new algorithm to determine the motion of unnrigid objects like smoke clouds is presented. It consists of two parts. The first one is the localisation of edges and feature points along them. The wavelet transform is used for each of these localisations. The determination of feature points is done through the characterisation of a quantity related to Lipschitz exponents. The second part is the tracking of the set of feature points obtained over an image sequence. We introduce a method which uses the gradient phase as characteristic to be tracked. Experimental results are presented to demonstrate the efficiency of the algorithm.