International audience

International audience

Fruit ripeness prediction (FRP) is a classification-based agricultural computer vision task that has attracted much attention, thanks to its wide-ranging advantages in agriculture field for both pre-harvest and post-harvest management. Accurate and timely FRP can be achieved using machine/deep learning-based hyperspectral image classification techniques. However, challenges including the limited availability of labeled data and the lack of robust methods generalizable to various hyperspectral cameras and fruit types can compromise the effectiveness of hyperspectral image-based FRP. Addressing these challenges, this paper introduces Fruit-HSNet, a machine learning architecture specifically designed for hyperspectral classification of fruit ripeness. Fruit-HSNet incorporates a spatio-spectral feature extraction module based on Fourier Transform and central pixel spectral signature followed by learnable feature fusion and a classifier optimized for ripeness classification. The proposed architecture was evaluated using the DeepHS Fruit dataset, the largest publicly available labeled real-world hyperspectral dataset for predicting fruit ripeness, which includes five different types of fruits-avocado, kiwi, mango, kaki, and papaya-captured with three distinct hyperspectral cameras at various stages of ripeness. Experimental results highlight that Fruit-HSNet substantially outperforms existing deep learning methods, from baseline to state-of-the-art models, with improvements of 12%, achieving a new state-of-the-art overall accuracy of 70.73%.

International audience

Virtual Try-On (VTO) technology, powered by Augmented Reality (AR) and Artificial Intelligence (AI), is reshaping online retail with immersive, real-time user experiences. However, the demand for high-quality, real-time VTO interactions introduces computational and latency challenges. Offloading computations to edge devices, enabled by 5 G networks, offers a promising solution to reduce end-to-end (E2E) latency and support multi-user environments. This paper proposes an optimized VTO platform tailored for edge deployment, featuring a novel knowledge distillation scheme for efficient 3D hand reconstruction. Our knowledge distillation approach reduces the model size by up to 70 % compared to the teacher model, achieving close visual accuracy in simpler tasks, such as flat hand reconstruction, ensuring it meets the quality standards of VTO applications. Experimental results demonstrate that our edge-optimized VTO architecture effectively balances quality and latency, supporting scalable, real-time interactions and advancing VTO technology for next-generation e-commerce applications.

Virtual Try-On (VTO) technologies are becoming a cornerstone of immersive retail experiences in the Metaverse. However, delivering real-time, high-fidelity interactions at scale remains a significant challenge, especially for Artificial Intelligence (AI) based applications that demand substantial computational resources. While edge servers effectively reduce latency and provide greater computational power than constrained User Equipment (UE), they face scalability issues under heavy multiuser loads.In this paper, we propose a novel scheme called Hybrid-Edge-VR4Fit. It is a hybrid offloading framework designed for scalable, low-latency VTO in 5G-edge environments. The system enhances responsiveness by leveraging edge inference to refine subsequent locally processed frames, reducing offloading frequency without compromising quality of service. We then introduce a dynamic offloading strategy modeled as a finite congestion game. Each UE autonomously adjusts its behavior based on real-time edge server and 5G network conditions to balance latency and quality trade-offs. Extensive experimental evaluations demonstrate that our proposal significantly reduces server congestion and queuing delays while maintaining visual fidelity, enabling responsive VTO services across diverse user densities.

3D point cloud classification, a task in computer vision, has recently gained substantial interest due to its extensive applications across various domains such as augmented and virtual reality, robotics, and autonomous driving. However, classifying 3D point clouds becomes challenging in real-world scenarios where the density levels vary. These variations are due to limitations inherent to current sensing technologies and variable environmental conditions, which often prevent the capture of detailed and comprehensive geometric representations. Existing classification methods, which have demonstrated impressive results on dense point clouds, may be affected and could experience reduced performance when dealing with low-density point clouds. Addressing this challenge, this paper introduces LD-PointNet++, an architecture optimized for the classification of low-density point clouds. To pave the way for LD-PointNet++, we conduct the first benchmarking study that systematically evaluates state-of-the-art point cloud classification methods across varying densities, revealing significant performance declines in low-density environments. Experimental results on the ModelNet benchmark dataset demonstrate that LD-PointNet++ not only outperforms these methods in low-density scenarios but also demonstrates consistent accuracy above 90 % across different density levels.

The Transformer architecture has gained significant popularity in computer vision tasks due to its capacity to generalize and capture long-range dependencies. This characteristic makes it well-suited for generating spatiotemporal tokens from videos. On the other hand, convolutions serve as the fundamental backbone for processing images and videos, as they efficiently aggregate information within small local neighborhoods to create spatial tokens that describe the spatial dimension of a video. While both CNN-based architectures and pure transformer architectures are extensively studied and utilized by researchers, the effective combination of these two backbones has not received comparable attention in the field of activity recognition. In this research, we propose a novel approach that leverages the strengths of both CNNs and Transformers in an hybrid architecture for performing activity recognition using RGB videos. Specifically, we suggest employing a CNN network to enhance the video representation by generating a 128-channel video that effectively separates the human performing the activity from the background. Subsequently, the output of the CNN module is fed into a transformer to extract spatiotemporal tokens, which are then used for classification purposes. Our architecture has achieved new SOTA results with 90.05 %, 99.6%, and 95.09% on HMDB51. UCF101. and ETRI-Activity3D respectively.

Deep learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this article, we propose a geometric deep learning approach using rigid and non-rigid transformations, named KShapenet, for 2D and 3D landmark-based human motion analysis. Landmark configuration sequences are first modeled as trajectories on Kendall's shape space and then mapped to a linear tangent space. The resulting structured data are then input to a deep learning architecture, which includes a layer that optimizes over rigid and non-rigid transformations of landmark configurations, followed by a CNN-LSTM network. We apply KShapenet to 3D human landmark sequences for action and gait recognition, and 2D facial landmark sequences for expression recognition, and demonstrate the competitiveness of the proposed approach with respect to state-of-the-art.

Deep Learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this paper, we propose a geometry aware deep learning approach using rigid and non rigid transformation optimization for skeleton-based action recognition. Skeleton sequences are first modeled as trajectories on Kendall’s shape space and then mapped to the linear tangent space. The resulting structured data are then fed to a deep learning architecture, which includes a layer that optimizes over rigid and non rigid transformations of the 3D skeletons, followed by a CNN-LSTM network. The assessment on two large scale skeleton datasets, namely NTU-RGB+D and NTU-RGB+D 120, has proven that the proposed approach outperforms existing geometric deep learning methods and exceeds recently published approaches with respect to the majority of configurations.

In this paper we propose a novel gait recognition approach from animated 3D skeletal data. Our approach is based on two disparate ideas from Shape Analysis and Functional Data Analysis (FDA) for a joint geometric-functional analysis. That is, skeletal sequences are viewed as time-parametrized trajectories on the Kendall's shape space when scaling, translation and rotation variations are filtered out from fixed-time 3D skeletons. A Riemannian Functional Principal Component Analysis (RFPCA) is carried out on our manifold-valued trajectories in order to build a new basis of principal functions, termed EigenTrajectories. Thus, each trajectory, could be projected into the eigenbasis which give rise to a compact signature, or EigenScores. The latter is fed to pre-trained `One-vs-All' SVM classifiers for identity recognition and authentication. Based on the geometry of the underlying shape space, tools for re-sampling and synchronizing trajectories are naturally derived to apply the proposed variant of FPCA. We have conducted experiments on a subset of the CMU dataset. Our approach shows promising results compared to the state-of-the-art when a compact and robust signature is considered.

Completely embedded in the 3D era, depth maps coding becomes a must in order to favour 3D admission to different fields of application, ranging from video games to medical imaging. This study presents a novel depth coding approach that, after a decimation step favouring the foreground, decomposes depth maps onto a set of sparse coefficients and redundant mixed discrete cosine and B‐splines atoms highly correlated to depth maps piece‐wise linear nature. Depth decomposition searches the best rate/distortion tradeoff through minimisation of an adaptive cost function, where its weight parameter is manipulated according to depth homogeneity. The bigger the parameter is, the more the sparsity is favoured at the expense of synthesis quality. Furthermore, handled distortion measure of the cost function quantifies the effect of depth maps coding on rendered views quality. The experiments show the relevance of the proposed method, able to obtain considerable tradeoffs between bitrate and synthesised views distortion.

Completely embedded in the 3D era, depth maps coding becomes a must in order to favor 3D admission to different fields of application, ranging from video games to medical imaging. This paper presents a novel depth coding approach that decomposes a decimated version of the original depth image on a sparse set of coefficients and mixed discrete cosine and B-splines atoms. The upstream decimation step reduces encoding bitrate without significant loss of virtual views quality. Depth decomposition is performed through minimization of an adaptive Rate/Distortion cost function, where we manipulate its weight parameter according to depth discontinuities. We then refine the choice of distortion metric in order to quantify the effect of depth maps coding on rendered views quality. Experiments show the relevance of the proposed method, able to obtain considerable tradeoffs between bitrate and synthesized views distortion.

Multiview video plus depth is the most popular 3D video representation that would support novel applications including free viewpoint television. These applications highly depend on high quality rendering of interpolated views which, as well, highly depends on the quality of decoded depth images. Therefore, a depth map coding that preserves perceptual quality, particularly on high frequency regions, is primary. In this paper, we propose a coding depth maps method that deals with emerging sparse signal decomposition technique. Depth images are approximated by a linear combination of few nonzero coefficients and dictionary atoms. Selected atoms are elementary signals based on a mixture of discrete cosine and B-splines of first degree. Sparse depth maps coding is tuned using a couple quality criterion such that depth discontinuities are preserved. The results investigated by objective evaluations over several depth maps imply that the proposed depth maps coding achieves better Rate-Distortion than JPEG and JPEG 2000. Subjective evaluation is also presented to stress the visual quality of interpolated views.

This paper introduces a novel scalable 3D mesh compression technique based on a shape approximation prediction strategy. The proposed approach, so-called Shape Approximation Compression (SAC), directly compresses the levels of detail (LoDs) defined by the content creators, while exploiting their inter-correlations. Here, the geometry of each LoD is used in order to compute a smooth approximation of the next layer. A progressive mesh hierarchy is then built on the top of the approximated version making it possible to efficiently predict and progressively transmit the geometry approximation errors. The SAC codec was evaluated within the framework of the MPEG core experiments on Multi-Resolution 3D Mesh Coding (MR3DMC) and was preliminarily accepted for future standardization.

This paper introduces an original multi-resolution 3D mesh compression technique, called Shape Approximation-based Progressive Mesh (SAPM). The proposed approach losslessly compresses the mesh connectivity and exploits it in order to build a smooth approximation of the original mesh. The obtained mesh approximation is then decimated yielding a progressive mesh hierarchy. This hierarchy is used to efficiently predict and progressively transmit the geometry approximation errors. The proposed codec supports both spatial and quality scalabilities and offers high rate-distortion performances. Experimental evaluation shows that the SAPM codec is on average 38-57% more efficient than the state-of-the-art connectivity preserving 3D mesh compression techniques.