Abstract Time‐varying meshes (TVMs), that is mesh sequences with varying connectivity, are a greatly versatile representation of shapes evolving in time, as they allow a surface topology to change or details to appear or disappear at any time during the sequence. This, however, comes at the cost of large storage size. Since 2003, there have been attempts to compress such data efficiently. While the problem may seem trivial at first sight, considering the strong temporal coherence of shapes represented by the individual frames, it turns out that the varying connectivity and the absence of implicit correspondence information that stems from it makes it rather difficult to exploit the redundancies present in the data. Therefore, efficient and general TVM compression is still considered an open problem. We describe and categorize existing approaches while pointing out the current challenges in the field and hint at some related techniques that might be helpful in addressing them. We also provide an overview of the reported performance of the discussed methods and a list of datasets that are publicly available for experiments. Finally, we also discuss potential future trends in the field.

Point Cloud Registration is a key step in tasks like Simultaneous Localization and Mapping and 3D Reconstruction. Methods that manage to use geometric information efficiently, show remarkable registration results in practice. Although these methods, in general, introduce intricate cost functions that prove challenging for general-purpose optimization algorithms. In this work, we introduce an efficient algorithm for solving the Generalized Least Squares problem, involved in the Generalized Iterative Closest Point, the most representative method of this class. Our method successfully combines the speed of the fastest algorithms with the accuracy of the most precise.

Gesture recognition is a tool to enable novel interactions with different techniques and applications, like Mixed Reality and Virtual Reality environments. With all the recent advancements in gesture recognition from skeletal data, it is still unclear how well state-of-the-art techniques perform in a scenario using precise motions with two hands. This paper presents the results of the SHREC 2024 contest organized to evaluate methods for their recognition of highly similar hand motions using the skeletal spatial coordinate data of both hands. The task is the recognition of 7 motion classes given their spatial coordinates in a frame-by-frame motion. The skeletal data has been captured using a Vicon system and pre-processed into a coordinate system using Blender and Vicon Shogun Post. We created a small, novel dataset with a high variety of durations in frames. This paper shows the results of the contest, showing the techniques created by the 5 research groups on this challenging task and comparing them to our baseline method.

Autonomous vehicles are expected to operate safely in real-life road conditions in the next years. Nevertheless, unanticipated events such as the existence of unexpected objects in the range of the road, can put safety at risk. The advancement of sensing and communication technologies and Internet of Things may facilitate the recognition of hazardous situations and information exchange in a cooperative driving scheme, providing new opportunities for the increase of collaborative situational awareness. Safe and unobtrusive visualization of the obtained information may nowadays be enabled through the adoption of novel Augmented Reality (AR) interfaces in the form of windshields. Motivated by these technological opportunities, we propose a saliency-based distributed, cooperative and rendering scheme for increasing the driver’s situational awareness through (i) automated negative obstacle (potholes) detection, (ii) AR visualization and (iii) information sharing (upcoming potential dangers) with other connected vehicles or road infrastructure. An extensive evaluation study using a variety of real datasets for pothole detection showed that the proposed method provides favorable results and features compared to other recent and relevant approaches.

Nowadays, three-dimensional (3D) meshes are widely used in various applications in different areas (e.g., industry, education, entertainment and safety). The 3D models are captured with multiple RGB-D sensors, and the sampled geometric manifolds are processed, compressed, simplified, stored, and transmitted to be reconstructed in a virtual space. These low-level processing applications require the accurate representation of the 3D models that can be achieved through saliency estimation mechanisms that identify specific areas of the 3D model representing surface patches of importance. Therefore, saliency maps guide the selection of feature locations facilitating the prioritization of 3D manifold segments and attributing to vertices more bits during compression or lower decimation probability during simplification, since compression and simplification are counterparts of the same process. In this work, we present a novel deep saliency mapping approach applied to 3D meshes, emphasizing decreasing the execution time of the saliency map estimation, especially when compared with the corresponding time by other relevant approaches. Our method utilizes baseline 3D importance maps to train convolutional neural networks. Furthermore, we present applications that utilize the extracted saliency, namely feature-aware multiscale compression and simplification frameworks.

The new generation 3D scanner devices have revolutionized the way information from 3D objects is acquired, making the process of scene capturing and digitization straightforward. However, the effectiveness and robustness of conventional algorithms for real scene analysis are usually deteriorated due to challenging conditions, such as noise, low resolution, and bad perceptual quality. In this work, we present a methodology for identifying and registering partially-scanned and noisy 3D objects, lying in arbitrary positions in a 3D scene, with corresponding high-quality models. The methodology is assessed on point cloud scenes with multiple objects with large missing parts. The proposed approach does not require connectivity information and is thus generic and computationally efficient, thereby facilitating computationally demanding applications, like augmented reality. The main contributions of this work are the introduction of a layered joint registration and indexing scheme of cluttered partial point clouds using a novel multi-scale saliency extraction technique to identify distinctive regions, and an enhanced similarity criterion for object-to-model matching. The processing time of the process is also accelerated through 3D scene segmentation. Comparisons of the proposed methodology with other state-of-the-art approaches highlight its superiority under challenging conditions.

3D representations of highly deformable 3D models, such as dynamic 3D meshes, have recently become very popular due to their wide applicability in various domains. This trend inevitably leads to a demand for storage and transmission of voluminous data sets, making the need for the design of a robust and reliable compression scheme a necessity. In this work, we present an approach for dynamic 3D mesh compression, that effectively exploits the spatio-temporal coherence of animated sequences, achieving low compression ratios without noticeably affecting the visual quality of the animation. We show that, on contrary to mainstream approaches that either exploit spatial (e.g., spectral coding) or temporal redundancies (e.g., PCA-based method), the proposed scheme, achieves increased efficiency, by projecting the differential coordinates sequence to the subspace of the covariance of the point trajectories. An extensive evaluation study, using, different dynamic 3D models, highlights the benefits of the proposed approach in terms of both execution time and reconstruction quality, providing extremely low bit-per-vertex- per-frame (bpvf) rates.

New generation 3-D scanning technologies are expected to create a revolution at the Industry 4.0, facilitating a large number of virtual manufacturing tools and systems. Such applications require the accurate representation of physical objects and/or systems achieved through saliency estimation mechanisms that identify certain areas of the 3-D model, leading to a meaningful and easier to analyze representation of a 3-D object. 3-D saliency mapping is, therefore, guiding the selection of feature locations and is adopted in a large number of low-level 3-D processing applications including denoising, compression, simplification, and registration. In this article, we propose a robust and fast method for creating 3-D saliency maps that accurately identifies sharp and small-scale geometric features in various industrial 3-D models. An extensive experimental study using a large number of 3-D scanned and CAD models verifies the effectiveness of the proposed method as compared to other recent and relevant approaches despite the constraints posed by complex geometry patterns or the presence of noise.

Recent advances in 3-D scanning technology have enabled the deployment of 3-D models in various industrial applications such as digital twins, remote inspection, and reverse engineering. Despite their evolving performance, 3-D scanners still introduce noise and artifacts in the acquired dense models. In this article, we propose a fast and robust denoising method for the dense 3-D scanned industrial models. The proposed approach employs conditional variational autoencoders to effectively filter face normals. Training and inference are performed in a sliding patch setup reducing the size of the required training data and execution times. We conducted extensive evaluation studies using 3-D scanned and CAD models. The results verify plausible denoising outcomes, demonstrating similar or higher reconstruction accuracy, compared to other state-of-the-art approaches. Specifically, for 3-D models with more than 1e4 faces, the presented pipeline is twice as fast as methods with equivalent reconstruction error.

Throughout the years, several works have been proposed for 3D mesh denoising. Nevertheless, despite their reconstruction quality, there are still challenges related to the preservation of the fine surface features. Motivated by the impressive results of image denoising by 3D transform-domain collaborative filtering (CF), we extend it to 3D mesh denoising. CF is also capable of revealing the finest details shared by grouped blocks while preserving at the same time the unique features of each block. A new promising approach suggests unrolling the computational pipeline of CF into a convolutional neural network (CNN) structure increasing significantly the efficiency of this solution. In this paper, we successfully extend and apply this method to 3D meshes making a transition from face normals to pixels. Extensive evaluation studies carried out using a variety of 3D meshes verify that the proposed approach achieves plausible reconstruction outputs and provides very promising results.

Mesh saliency has been widely considered as the measure of visual importance of certain parts of 3D geometries, distinguishable from their surroundings, with respect to human visual perception. This work is based on the use of convolutional neural networks to extract saliency maps for large and dense 3D scanned models. The network is trained with saliency maps extracted by fusing local and global spectral characteristics. Extensive evaluation studies carried out using various 3D models, include visual perception evaluation in simplification and compression use cases. As a result, they verify the superiority of our approach as compared to other state-of-the-art approaches. Furthermore, these studies indicate that CNN-based saliency extraction method is much faster in large and dense geometries, allowing the application of saliency aware compression and simplification schemes in low-latency and energy-efficient systems.

Throughout the years, several works have been proposed for performing feature-preserving mesh denoising. Despite their reconstruction benefits, there are still challenges that need to be addressed. Meanwhile, several researchers, system engineers, and software developers have shown increasing interest in the application of deep learning approaches for performing several low-level information processing tasks, such as denoising, compression, etc., in image and video applications. In this paper, we present a method for reconstructing accurately static and dynamic noisy meshes by applying autoencoders on guided normals. To increase the effectiveness of the proposed method, we use different models for different set of faces that are classified as features and non-features. Extensive evaluation studies carried out using a variety of static and dynamic meshes, verify that the proposed approach achieves plausible reconstruction output despite the constraints posed by complex noise and geometry patterns.

Many mixed reality applications are based on the real-time compression and streaming of three-dimensional (3D) models. Thus, they demand very high-bandwidth and ultra-low latency from network specifications. The next-generation wireless networks will employ promising technologies to significantly improve the communication data rates. However, due to implementation complexity and thus increased energy consumption of these technologies, a trade-off between the quality-of-user-experience (QoE) and the hardware specifications is necessary. To overcome these limitations low-resolution quantizers have been of interest, which provide a trade-off between quality and complexity. In this paper, we propose a complexity-aware perceptual coding scheme that minimizes the reconstruction losses of the 3D models. Extensive simulations assuming different 3D models show that the proposed scheme achieves plausible reconstruction output offering significantly higher energy efficiency gains, as compared to a context unaware coding approaches.

The latest advancements in 3D scanning technologies have facilitated the generation and adoption of 3D models in several industrial applications ranging from manufacturing inspection and repair to digital twins and medical industry. All these applications require an accurate representation of physical objects through saliency mechanisms identifying certain areas of the 3D model that are considered as important information by humans. Hence, 3D saliency mapping is an essential mechanism in a number of 3D processing applications including denoising, compression, simplification, registration, viewpoint selection, etc. In this work, we propose a robust 3D saliency mapping method ideally suited for industrial 3D models with sharp and small scale geometric features. An extensive simulation study using a variety of 3D scanned and CAD models, verify the effectiveness of the proposed method as compared to other relevant approaches.

Through the years, several works have demonstrated high-quality 3D mesh denoising. Despite the high reconstruction quality, there are still challenges that need to be addressed ranging from variations in configuration parameters to high computational complexity. These drawbacks are crucial especially if the reconstructed models have to be used for quality check, inspection or repair in manufacturing environments where we have to deal with large objects resulting in very dense 3D meshes. Recently, deep learning techniques have shown that are able to automatically learn and find more accurate and reliable results, without the need for setting manually parameters. In this work, motivated by the aforementioned requirements, we propose a fast and reliable denoising method that can be effectively applied for reconstructing very dense noisy 3D models. The proposed method applies conditional variational autoencoders on face normals. Extensive evaluation studies carried out using a variety of 3D models verify that the proposed approach achieves plausible reconstruction outputs, very relative or even better of those proposed by the literature, in considerably faster execution times.

The increasing interest for reliable generation of large scale scenes and objects has facilitated several real-time applications. Although the resolution of the new generation geometry scanners are constantly improving, the output models, are inevitably noisy, requiring sophisticated approaches that remove noise while preserving sharp features. Moreover, we no longer deal exclusively with individual shapes, but with entire scenes resulting in a sequence of 3D surfaces that are affected by noise with different characteristics due to variable environmental factors (e.g., lighting conditions, orientation of the scanning device). In this work, we introduce a novel coarse-to-fine graph spectral processing approach that exploits the fact that the sharp features reside in a low dimensional structure hidden in the noisy 3D dataset. In the coarse step, the mesh is processed in parts, using a model based Bayesian learning method that identifies the noise level in each part and the subspace where the features lie. In the feature-aware fine step, we iteratively smooth face normals and vertices, while preserving geometric features. Extensive evaluation studies carried out under a broad set of complex noise patterns verify the superiority of our approach as compared to the state-of-the-art schemes, in terms of reconstruction quality and computational complexity.

Recently, there is a tremendous interest in the processing of unorganized point clouds, generated using a variety of 3D scanning technologies such as structured light and LIDAR systems. Without a doubt, the most compelling problem in this domain is the removal of outliers. To effectively address the aforementioned issue, we present a novel method, that detects accurately and efficiently the outliers by exploiting the spatial coherence in the object geometry and the sparsity of the outliers in the spatial domain. This is achieved by solving a convenient convex method called Robust PCA (RPCA). To demonstrate the effectiveness of the proposed technique, we evaluate it by using real scanned point clouds which are extremely dense consisting of millions of points.

Recently, there has been increasing interest in the processing of dynamic scenes as captured by 3D scanners, ideally suited for challenging applications such as immersive tele-presence systems and gaming. Despite the fact that the resolution and accuracy of the modern 3D scanners are constantly improving, the captured 3D point clouds are usually noisy with a perceptive percentage of outliers, stressing the need of an approach with low computational requirements which will be able to automatically remove the outliers and create a consolidated point cloud. In this paper, we introduce a novel method which first recognizes and removes outliers from a dynamic point cloud sequence (DPCS) using a very fast Robust PCA (RPCA) approach and then we use a novel weighted Laplacian interpolation approach to achieve a fast and effective consolidation of a DPCS. Extensive evaluation studies, carried out using a collection of different DPCS, verify that the proposed technique achieves plausible reconstruction output despite the constraints posed by arbitrarily complex motion scenarios.

In this paper, we present a progressive compression scheme that enables aggressive compression ratios, by successfully identifying and encoding sharp and small scale geometric features. The accurate identification of the features is achieved by exploiting the low rank property of the captured geometry and the sparsity of the features in the Laplacian domain, permeating benefits from robust principal component analysis. Due to the visual importance of the identified geometric features, the geometry coding process, is optimized for preserving the geometric features at extremely low bit rates. Extensive evaluation studies, carried out using a collection of scanned and synthetic 3D models, show that the proposed feature aware high pass quantization method achieves extremely high compression ratios, offering at the same time meaningful approximations of the given surfaces. Finally, a short discussion regarding the applicability of the proposed feature identification scheme to smooth completion and feature preserving surface denoising is also offered.