Solutions to visual computing problems such as shape reconstruction, analysis, generation, and retrieval rely on fast and task-appropriate 3D shape representations for efficiency. This work explores representing shapes with neural ray fields, which are inherently more efficient to render than 3D coordinate-based neural representations requiring only a single network evaluation per ray, but with high data requirements and reduced fidelity thus far. We propose an improved ray field representation with improved gradient flow and a more tractable parameter domain, along with a more stable objective function and data augmentation improving stability and fidelity. We study the strengths and failure modes of the prior art and our method, and how performance scales with data and compute.

Describing and differentiating 3D objects using shape descriptors is crucial in many fields, yet objects’ color information is often overlooked when these methods are designed. This paper presents two main contributions in that regard. First, it introduces ColorQUICCI, an extension of the QUICCI descriptor that integrates color information, as well as a distance function that balances color and shape information when comparing pairs of such descriptors. Results demonstrate that ColorQUICCI is advantageous when the overall accuracy and efficiency are considered. Second, it presents ColorShapeBench, an extension of ShapeBench for the large-scale evaluation of descriptors that include color information. An illumination change filter is also introduced in the benchmark, which provides a more robust platform for evaluating descriptors in scenarios with varying lighting conditions.

Abstract This paper presents the application of two key stages in Cultural Heritage (CH) analysis: cross-time registration and change detection, aimed at automatically identifying subtle geometric variations in CH objects. The proposed method addresses the challenge of manually aligning and detecting differences among large collections of 3D-digitized objects, which is both time-consuming and error-prone. The method combines CrossTimeReg deep learning technique for automatic registration and the Change-Based-Segmentation method for identifying differences and segmented changes. We applied this method to two ceramic sculptures, titled Zephyr and Flora, from the Museum of King Jan III’s Palace at Wilanów, Poland, and validated the results with the CH scientific experts of the museum. The results demonstrated that our method effectively detected geometric variations resulting from potential alterations or restorations, which are crucial for verifying the authenticity of artifacts. Additionally, the method facilitated comparative analysis, enabling researchers to examine similar objects and establish connections, origins, and historical significance. The technique also proved useful in analyzing changes in the geometry of the same object over time due to destructive factors.

Deep learning methods are revolutionizing the solutions to visual computing problems, such as shape retrieval and generative shape modeling, but require novel shape representations that are both fast and differentiable. Neural ray fields and their improved rendering performance are promising in this regard, but struggle with a reduced fidelity and multi-view consistency when compared to the more studied coordinate-based methods which, however, are slower in training and evaluation. We propose PMARF, an improved ray field which explicitly models the skeleton of the target shape as a set of (0-thickness) parametric medial surfaces. This formulation reduces by construction the degrees-of-freedom available in the reconstruction domain, improving multi-view consistency even from sparse training views. This in turn improves fidelity while facilitating a reduction in the network size.

We propose Medial Atom Ray Fields (MARFs), a novel neural object representation that enables accurate differentiable surface rendering with a single network evaluation per camera ray. Existing neural ray fields struggle with multi-view consistency and representing surface discontinuities. MARFs address both using a medial shape representation, a dual representation of solid geometry that yields cheap geometrically grounded surface normals, in turn enabling computing analytical curvature despite the network having no second derivative. MARFs map a camera ray to multiple medial intersection candidates, subject to ray-sphere intersection testing. We illustrate how the learned medial shape quantities applies to sub-surface scattering, part segmentation, and aid representing a space of articulated shapes. Able to learn a space of shape priors, MARFs may prove useful for tasks like shape retrieval and shape completion, among others. Code and data can be found at https://github.com/ANONYMOUS/marf.

Multimodal registration is a challenging problem in visual computing, commonly faced during medical image-guided interventions, data fusion and 3D object retrieval. The main challenge of multimodal registration is finding accurate correspondence between modalities, since different modalities do not exhibit the same characteristics. This paper explores how the coherence of different modalities can be utilized for the challenging task of 3D multimodal registration. A novel deep learning multimodal registration framework is proposed by introducing a siamese deep learning architecture, especially designed for aligning and fusing modalities of different structural and physical principles. The cross-modal attention blocks lead the network to establish correspondences between features of different modalities. The proposed framework focuses on the alignment of 3D point clouds and the micro-CT 3D volumes of the same object. A multimodal dataset consisting of real micro-CT scans and their synthetically generated 3D models (point clouds) is presented and utilized for evaluating our methodology.

This work presents a novel multimodal biometric dataset with emerging biometric traits including 3D face, thermal face, iris on-the-move, iris mobile, somatotype and smartphone sensors. This dataset was created to resemble on-the-move characteristics in applications such as border control. The five types of biometric traits were selected as they can be captured while on-the-move, are contactless, and show potential for use in a multimodal fusion verification system in a border control scenario. Innovative sensor hardware was used in the data capture. The data featuring these biometric traits will be a valuable contribution to advancing biometric fusion research in general. Baseline evaluation was performed on each unimodal dataset. Multimodal fusion was evaluated based on various scenarios for comparison. Real-time performance is presented based on an Automated Border Control (ABC) scenario.

A complete pipeline is presented for accurate and efficient partial 3D object retrieval based on Quick Intersection Count Change Image (QUICCI) binary local descriptors and a novel indexing tree. It is shown how a modification to the QUICCI query descriptor makes it ideal for partial retrieval. An indexing structure called Dissimilarity Tree is proposed which can significantly accelerate searching the large space of local descriptors; this is applicable to QUICCI and other binary descriptors. The index exploits the distribution of bits within descriptors for efficient retrieval. The retrieval pipeline is tested on the artificial part of SHREC’16 dataset with near-ideal retrieval results.

Registration is a ubiquitous operation in visual computing and constitutes an important pre-processing step for operations such as 3D object reconstruction, retrieval and recognition. Particularly in cultural heritage (CH) applications, registration techniques are essential for the digitization and restoration pipelines. Cross-time registration is a special case where the objects to be registered are instances of the same object after undergoing processes such as erosion or restoration. Traditional registration techniques are inadequate to address this problem with the required high accuracy for detecting minute changes; some are extremely slow. A deep learning registration framework for cross-time registration is proposed which uses the DeepGMR network in combination with a novel down-sampling scheme for cross-time registration. A dataset especially designed for cross-time registration is presented (called ECHO) and an extensive evaluation of state-of-the-art methods is conducted for the challenging case of cross-time registration.

A novel shape descriptor for cluttered scenes is presented, the Radial Intersection Count Image (RICI), and is shown to significantly outperform the classic Spin Image (SI) and 3D Shape Context (3DSC) in both uncluttered and, more significantly, cluttered scenes. It is also faster to compute and compare. The clutter resistance of the RICI is mainly due to the design of a novel distance function, capable of disregarding clutter to a great extent. As opposed to the SI and 3DSC, which both count point samples, the RICI uses intersection counts with the mesh surface, and is therefore noise-free. For efficient RICI construction, novel algorithms of general interest were developed. These include an efficient circle-triangle intersection algorithm and an algorithm for projecting a point into SI-like (α, β) coordinates. The ’clutterbox experiment’ is also introduced as a better way of evaluating descriptors’ response to clutter. The SI, 3DSC, and RICI are evaluated in this framework and the advantage of the RICI is clearly demonstrated.

A binary descriptor indexing scheme based on Hamming distance called the Hamming tree for local shape queries is presented. A new binary clutter resistant descriptor named Quick Intersection Count Change Image (QUICCI) is also introduced. This local shape descriptor is extremely small and fast to compare. Additionally, a novel distance function called Weighted Hamming applicable to QUICCI images is proposed for retrieval applications. The effectiveness of the indexing scheme and QUICCI is demonstrated on 828 million QUICCI images derived from the SHREC2017 dataset, while the clutter resistance of QUICCI is shown using the clutterbox experiment.

Registration is a ubiquitous operation in Visual Computing, with applications in 3D object retrieval among others. Registration is the process of overlaying two or more datasets taken from different viewpoints, at different times or by different sensors into a common reference frame. Multimodal registration is a special case where the data to be matched do not belong to the same modality and is challenging due to the diverse nature of the modalities involved which makes the creation of a distance function harder. Due to the large number of possible modality combinations and application fields, a considerable number of multimodal registration techniques have been proposed in diverse fields, including medicine and archaeology. This survey aims to unify 3D multimodal registration techniques (i.e. where at least one of the modalities is in 3D) across application domains, with the hope of providing an application-independent view and the potential for cross-fertilization. The problem of 3D multimodal registration is explicitly defined and the various methods are systematically categorized and described in terms of a number of important properties. Methods with publicly available source code have been compared on common datasets. A discussion on trends, observations and challenges for further research concludes the review.

• High-dimensional descriptors. • Machine-learned descriptors. • Texture retrieval.

A fully automatic mesh segmentation scheme using heterogeneous graphs is presented. We introduce a spectral framework where local geometry affinities are coupled with surface patch affinities. A heterogeneous graph is constructed combining two distinct graphs: a weighted graph based on adjacency of patches of an initial over-segmentation, and the weighted dual mesh graph. The partitioning relies on processing each eigenvector of the heterogeneous graph Laplacian individually, taking into account the nodal set and nodal domain theory. Experiments on standard datasets show that the proposed unsupervised approach outperforms the state-of-the-art unsupervised methodologies and is comparable to the best supervised approaches.

A 3D landmark detection method for 3D facial scans is presented and thoroughly evaluated. The main contribution of the presented method is the automatic and pose-invariant detection of landmarks on 3D facial scans under large yaw variations (that often result in missing facial data), and its robustness against large facial expressions. Three-dimensional information is exploited by using 3D local shape descriptors to extract candidate landmark points. The shape descriptors include the shape index, a continuous map of principal curvature values of a 3D object's surface, and spin images, local descriptors of the object's 3D point distribution. The candidate landmarks are identified and labeled by matching them with a Facial Landmark Model (FLM) of facial anatomical landmarks. The presented method is extensively evaluated against a variety of 3D facial databases and achieves state-of-the-art accuracy (4.5-6.3 mm mean landmark localization error), considerably outperforming previous methods, even when tested with the most challenging data.

We present a novel approach for computing a compact and highly discriminant biometric signature for 3D face recognition using linear dimensionality reduction techniques. Initially, a geometry-image representation is used to effectively resample the raw 3D data. Subsequently, a wavelet transform is applied and a biometric signature composed of 7,200 wavelet coefficients is extracted. Finally, we apply a second linear dimensionality reduction step to the wavelet coefficients using Linear Discriminant Analysis and compute a compact biometric signature. Although this biometric signature consists of just 57 coefficients, it is highly discriminant. Our approach, UR3D-C, is experimentally validated using four publicly available databases (FRGC vl, FRGC v2, Bosphorus and BU-3DFE). State-of-the-art performance is reported in all of the above databases.

The uncontrolled conditions of real-world biometric applications pose a great challenge to any face recognition approach. The unconstrained acquisition of data from uncooperative subjects may result in facial scans with significant pose variations along the yaw axis. Such pose variations can cause extensive occlusions, resulting in missing data. In this paper, a novel 3D face recognition method is proposed that uses facial symmetry to handle pose variations. It employs an automatic landmark detector that estimates pose and detects occluded areas for each facial scan. Subsequently, an Annotated Face Model is registered and fitted to the scan. During fitting, facial symmetry is used to overcome the challenges of missing data. The result is a pose invariant geometry image. Unlike existing methods that require frontal scans, the proposed method performs comparisons among interpose scans using a wavelet-based biometric signature. It is suitable for real-world applications as it only requires half of the face to be visible to the sensor. The proposed method was evaluated using databases from the University of Notre Dame and the University of Houston that, to the best of our knowledge, include the most challenging pose variations publicly available. The average rank-one recognition rate of the proposed method in these databases was 83.7 percent.

In this paper, we present a new method for bidirectional relighting for 3D-aided 2D face recognition under large pose and illumination changes. During subject enrollment, we build subject-specific 3D annotated models by using the subjects' raw 3D data and 2D texture. During authentication, the probe 2D images are projected onto a normalized image space using the subject-specific 3D model in the gallery. Then, a bidirectional relighting algorithm and two similarity metrics (a view-dependent complex wavelet structural similarity and a global similarity) are employed to compare the gallery and probe. We tested our algorithms on the UHDB11 and UHDB12 databases that contain 3D data with probe images under large lighting and pose variations. The experimental results show the robustness of our approach in recognizing faces in difficult situations.

In this paper, we introduce a new system for profile-based face recognition. The specific scenario involves a driver entering a gated area and using his/her side-view image (the driver remains seated in the vehicle) as identification. The system has two modes: enrollment and identification. In the enrollment mode, 3D face models of subjects are acquired and profiles extracted under different poses and stored to form a gallery database. In the identification mode, 2D images are acquired and the corresponding planar profiles are extracted and used as probes. Then, probes are matched to the gallery profiles to determine identity. The matching is accomplished using implicit shape registration via the vector distance functions. In our experiments, the approach using implicit registration exhibited higher accuracy than the iterative closest point methodology due to the use of more general transformations. The performance of our system is illustrated using a variety of databases.

In this paper, we present an overview of the 3D object retrieval method that we employed in our participation to the generic models track of SHREC 2008 organized by the AIM@SHAPE network of excellence. The proposed methodology is detailed in [2]. Our method is based on a hybrid scheme where 2D features as well as 3D features are extracted from a 3D model which has been previously normalized for rotation using two alternative alignment techniques. The alignment methods that are used are CPCA and NPCA. The 2D features are Fourier coefficients extracted from a set of depth buffers and the 3D features are spherical harmonic coefficients extracted from a spherical function-based representation of a 3D model.

Three-dimensional data are increasingly being used for biometric purposes as they offer resilience to problems common mon in two-dimensional data. They have been successfully applied to face recognition and more recently to ear recognition. However, real-life biometric applications require algorithms that are both robust and efficient so that they scale well with the size of the databases. A novel ear recognition method is presented that uses a generic annotated ear model to register and fit each ear dataset. Then a compact biometric signature is extracted that retains 3D information. The proposed method is evaluated using the largest publicly available 3D ear database appended with our own database, resulting in a database containing data from multiple 3D sensor types. Using this database it is shown that the proposed method is not only robust, accurate and sensor invariant but also extremely efficient, thus making it suitable for real-life biometric applications.

Methods for representing curves in \mathbb{R}^2 and \mathbb{R}^3 using fractal interpolation techniques are presented. We show that such representations are both effective and convenient for irregular or complicated data. Experiments in various datasets, including geographical and medical data, verify the practical usefulness of these methods.

In this paper, we present the computational tools and a hardware prototype for 3D face recognition. Full automation is provided through the use of advanced multistage alignment algorithms, resilience to facial expressions by employing a deformable model framework, and invariance to 3D capture devices through suitable preprocessing steps. In addition, scalability in both time and space is achieved by converting 3D facial scans into compact metadata. We present our results on the largest known, and now publicly available, Face Recognition Grand Challenge 3D facial database consisting of several thousand scans. To the best of our knowledge, this is the highest performance reported on the FRGC v2 database for the 3D modality.

As the size of the available collections of 3D objects grows, database transactions become essential for their management with the key operation being retrieval (query). Large collections are also precategorized into classes so that a single class contains objects of the same type (e.g., human faces, cars, four-legged animals). It is shown that general object retrieval methods are inadequate for intraclass retrieval tasks. We advocate that such intraclass problems require a specialized method that can exploit the basic class characteristics in order to achieve higher accuracy. A novel 3D object retrieval method is presented which uses a parameterized annotated model of the shape of the class objects, incorporating its main characteristics. The annotated subdivision-based model is fitted onto objects of the class using a deformable model framework, converted to a geometry image and transformed into the wavelet domain. Object retrieval takes place in the wavelet domain. The method does not require user interaction, achieves high accuracy, is efficient for use with large databases, and is suitable for nonrigid object classes. We apply our method to the face recognition domain, one of the most challenging intraclass retrieval tasks. We used the Face Recognition Grand Challenge v2 database, yielding an average verification rate of 95.2 percent at 10-3 false accept rate. The latest results of our work can be found at http://www.cbl.uh.edu/UR8D/.

In this paper, we present a new 3D face recognition approach. Full automation is provided through the use of advanced multi-stage alignment algorithms, resilience to facial expressions by employing a deformable model framework, and invariance to 3D capture devices through suitable preprocessing steps. In addition, scalability in both time and space is achieved by converting 3D facial scans into compact wavelet metadata. We present results on the largest known, and now publicly-available, Face Recognition Grand Challenge 3D facial database consisting of several thousand scans. To the best of our knowledge, our approach has achieved the highest accuracy on this dataset. 1

It is becoming increasingly important to be able to credential and identify authorized personnel at key points of entry. Such identity management systems commonly employ biometric identifiers. In this paper, we present a novel multimodal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors. Data from multiple cameras is used to construct a three-dimensional mesh representing the face and a facial thermal texture map. An annotated face model with explicit two-dimensional parameterization (UV) is then fitted to this data to construct: 1) a three-channel UV deformation image encoding geometry, and 2) a one-channel UV vasculature image encoding facial vasculature. Recognition is accomplished by comparing: 1) the parametric deformation images, 2) the parametric vasculature images, and 3) the visible spectrum texture maps. The novelty of our work lies in the use of deformation images and physiological information as means for comparison. We have performed extensive tests on the Face Recognition Grand Challenge v1.0 dataset and on our own multimodal database with very encouraging results.

Biometrics-based technologies in the area of identity management are gaining increasing importance, as a means of establishing non-falsifiable credentials for end users. However, in the three-way tug-of-war between convenient, unobtrusive data collection (required for user acceptance), accuracy in results (required for justifying deployment), and speed (required for widespread use in practice), no single biometric to date has managed to hold the middle ground that would allow for its ready adoption. The overall goal of our project is to develop the theoretical framework and computational tools that will lead to the development of a practical, unobtrusive, and accurate face recognition system for convenient and effective access control. This framework encompasses 8D characteristics of the face (3D geometry+2D visible texture+2D infrared texture, over time). In this paper, we present a novel multi-modal facial recognition approach that employs data from both visible spectrum and thermal infrared sensors. From the fitted parametric model we extract two images corresponding to the subject's face and process these images to extract biometric signatures. Specifically, the deformation image is compressed using a wavelet transform and the vasculature graph is extracted from the parametric thermal image.

This paper presentes a novel algorithm for the voxelization of surface models of arbitrary topology. Our algorithm uses the depth and stencil buffers, available in most commercial graphics hardware, to achieve high performance. It is suitable for both polygonal meshes and parametric surfaces. Experiments highlight the advantages and limitations of our approach.

Vector fields produced by experiments or simulations are usually extremely dense, which makes their manipulation and visualization cumbersome. Often, such fields can be simplified without much loss of information. A simplification method for 3D vector fields defined over tetrahedral meshes is presented. The underlying tetrahedral mesh is progressively simplified by successive half-edge collapses. The order of collapses is determined by a compound metric which takes into account the field and domain error incurred as well as the quality of the resulting mesh. Special attention is given to the preservation of the mesh boundary and of critical points on the vector field. A tool has been developed for the measurement of the difference between two vector fields over tetrahedral meshes, and it is used to quantify the simplification error

Abstract Progressive meshes are an established tool for triangle mesh simplification. By suitably adapting the simplification process, progressive hulls can be generated which enclose the original mesh in gradually simpler, nested meshes. We couple progressive hulls with a selective refinement framework and use them in applications involving intersection queries on the mesh. We demonstrate that selectively refinable progressive hulls considerably speed up intersection queries by efficiently locating intersection points on the mesh. Concerning the progressive hull construction, we propose a new formula for assigning edge collapse priorities that significantly accelerates the simplification process, and enhance the existing algorithm with several conditions aimed at producing higher quality hulls. Using progressive hulls has the added advantage that they can be used instead of the enclosed object when a lower resolution of display can be tolerated, thus speeding up the rendering process. ACM CSS: I.3.3 Computer Graphics—Picture/Image Generation, I.3.5 Computer Graphics—Computational Geometry and Object Modeling, I.3.7 Computer Graphics—Three‐Dimensional Graphics and Realism

The problem of re-assembling an object from its parts or fragments has never been addressed with a unified computational approach, which depends on the pure geometric form of the parts and not on application-specific features. We propose a method for the automatic reconstruction of a model based on the geometry of its parts, which may be computer-generated models or range-scanned models. The matching process can benefit from any other external constraint imposed by the specific application.

The main purpose of this paper is to present a functional view of the fundamentals of the computer graphics process based on the classic polygonal model. There are several advantages for adopting such an approach. Firstly, the functional view is a natural abstraction of the problem. Secondly, many well known computer graphics optimization techniques can be directly obtained from the original specification by applying general and well understood transformational programming algebraic laws on functional expressions. Thirdly, a number of highly parallel implementations suited for various parallel architectures can be derived from the initial specification by a systematic application of general transformation strategies for parallelizing functional programs.

Polygonal models are the most common representation of structured 3D data in computer graphics, pattern recognition and machine vision. The method presented here automatically identifies and labels all compact surface regions of a polygonal mesh, visible or not, and extracts valuable invariant features regarding their geometric attributes. A method that is independent of the mesh topology is also presented for the surface bumpiness estimation and the identification of coarse surface regions.

The selection of an appropriate classification model and algorithm is crucial for effective knowledge discovery on a dataset. For large databases, common in data mining, such a selection is necessary, because the cost of invoking all alternative classifiers is prohibitive. This selection task is impeded by two factors. First, there are many performance criteria, and the behaviour of a classifier varies considerably with them. Second, a classifier's performance is strongly affected by the characteristics of the dataset. Classifier selection implies mastering a lot of background information on the dataset, the models and the algorithms in question. An intelligent assistant can reduce this effort by inducing helpful suggestions from background information. In this study, we present such an assistant, NOEMON. For each registered classifier, NOEMON measures its performance for a collection of datasets. Rules are induced from those measurements and accommodated in a knowledge base. The suggestion on the most appropriate classifier(s) for a dataset is then based on those rules. Results on the performance of an initial prototype are also given.

Abstract We demonstrate how both area coherence and parallelism can be exploited in order to speed up rendering operations on a SIMD square array of processors. Our algorithms take advantage of the method of differences, in order to incrementally compute the values of a linear polynomial function at discrete intervals and thus implement area rendering operations efficiently. We discuss how filling of convex polygons, hidden surface elimination and smooth shading can be implemented on an N × N processor array that supports planar arithmetic, that is, arithmetic operations performed on N × N matrices in parallel for all matrix elements. A major attraction of the method we present is that it is based on a SIMD processor array; such machines are now recognised as highly general purpose given the wide range of applications successfully implemented on them.

We describe an algorithm for rendering a restricted class of trapeziums, and hence arbitrary polygons, in parallel on an NxN SIMD array processor. The algorithm achieves good performance by precomputing “surface patches” (finite portions of half -planes), thus trading storage for increased speed. The number of surface patches that are precomputed grows only linearly with the size of the array processor but as the square of the subpixel accuracy desired. NxN texture patterns can be added at very little extra cost additional to filling a trapezium with a particular colour.