We introduce the Pose and Illumination agnostic Anomaly Detection (PIAD) problem, a generalization of pose-agnostic anomaly detection (PAD). Being illumination agnostic is critical, as it relaxes the assumption that training data for an object has to be acquired in the same light configuration of the query images that we want to test. Moreover, even if the object is placed within the same capture environment, being illumination agnostic implies that we can relax the assumption that the relative pose between environment light and query object has to match the one in the training data. We introduce a new dataset to study this problem, containing both synthetic and real-world examples, propose a new baseline for PIAD, and demonstrate how our baseline provides state-of-the-art results in both PAD and PIAD, not only in the new proposed dataset, but also in existing datasets that were designed for the simpler PAD problem. Project page: https://kaichen-yang.github.io/piad/.

Although supervised deep normal estimators have recently shown impressive results on synthetic benchmarks, their performance deteriorates significantly in real-world scenarios due to the domain gap between synthetic and real data. Building high-quality real training data to boost those supervised methods is not trivial because point-wise annotation of normals for varying-scale real-world 3D scenes is a tedious and expensive task. This paper introduces PointNorm-Net, the first self-supervised deep learning framework to tackle this challenge. The key novelty of PointNorm-Net is a three-stage multi-modal normal distribution estimation paradigm that can be integrated into either deep or traditional optimization-based normal estimation frameworks. Extensive experiments show that our method achieves superior generalization and outperforms state-of-the-art conventional and deep learning approaches across three real-world datasets that exhibit distinct characteristics compared to the synthetic training data.

Most of the existing 3D talking face synthesis methods suffer from the lack of detailed facial expressions and realistic head poses, resulting in unsatisfactory experiences for users. In this article, we propose a novel pose-aware 3D talking face synthesis method with a novel geometry-guided audio-vertices attention. To capture more detailed expression, such as the subtle nuances of mouth shape and eye movement, we propose to build hierarchical audio features including a global attribute feature and a series of vertex-wise local latent movement features. Then, in order to fully exploit the topology of facial models, we further propose a novel geometry-guided audio-vertices attention module to predict the displacement of each vertex by using vertex connectivity relations to take full advantage of the corresponding hierarchical audio features. Finally, to accomplish pose-aware animation, we expand the existing database with an additional pose attribute, and a novel pose estimation module is proposed by paying attention to the whole head model. Numerical experiments demonstrate the effectiveness of the proposed method on realistic expression and head movements against state-of-the-art methods.

The normals of feature points, i.e., the intersection points of multiple smooth surfaces, are ambiguous and undefined. This paper presents a unified definition for point cloud normals of feature and non-feature points, which allows feature points to possess multiple normals. This definition facilitates several succeeding operations, such as feature points extraction and point cloud filtering. We also develop a feature preserving normal estimation method which outputs multiple normals per feature point. The core of the method is a pair consistency voting scheme. All neighbor point pairs vote for the local tangent plane. Each vote takes the fitting residuals of the pair of points and their preliminary normal consistency into consideration. Thus the pairs from the same subspace and relatively far off features dominate the voting. An adaptive strategy is designed to overcome sampling anisotropy. In addition, we introduce an error measure compatible with traditional normal estimators, and present the first benchmark for normal estimation, composed of 152 synthesized data with various features and sampling densities, and 288 real scans with different noise levels. Comprehensive and quantitative experiments show that our method generates faithful feature preserving normals and outperforms previous cutting edge normal estimation methods, including the latest deep learning based method.

Benefiting from global rank constraints, the low-rank representation (LRR) method has been shown to be an effective solution to subspace learning. However, the global mechanism also means that the LRR model is not suitable for handling large-scale data or dynamic data. For large-scale data, the LRR method suffers from high time complexity, and for dynamic data, it has to recompute a complex rank minimization for the entire data set whenever new samples are dynamically added, making it prohibitively expensive. Existing attempts to online LRR either take a stochastic approach or build the representation purely based on a small sample set and treat new input as out-of-sample data. The former often requires multiple runs for good performance and thus takes longer time to run, and the latter formulates online LRR as an out-of-sample classification problem and is less robust to noise. In this paper, a novel online LRR subspace learning method is proposed for both large-scale and dynamic data. The proposed algorithm is composed of two stages: static learning and dynamic updating. In the first stage, the subspace structure is learned from a small number of data samples. In the second stage, the intrinsic principal components of the entire data set are computed incrementally by utilizing the learned subspace structure, and the LRR matrix can also be incrementally solved by an efficient online singular value decomposition algorithm. The time complexity is reduced dramatically for large-scale data, and repeated computation is avoided for dynamic problems. We further perform theoretical analysis comparing the proposed online algorithm with the batch LRR method. Finally, experimental results on typical tasks of subspace recovery and subspace clustering show that the proposed algorithm performs comparably or better than batch methods, including the batch LRR, and significantly outperforms state-of-the-art online methods.

Partial differential equations (PDEs) have been used to formulate image processing for several decades. Generally, a PDE system consists of two components: the governing equation and the boundary condition. In most previous work, both of them are generally designed by people using mathematical skills. However, in real world visual analysis tasks, such predefined and fixed-form PDEs may not be able to describe the complex structure of the visual data. More importantly, it is hard to incorporate the labeling information and the discriminative distribution priors into these PDEs. To address above issues, we propose a new PDE framework, named learning to diffuse (LTD), to adaptively design the governing equation and the boundary condition of a diffusion PDE system for various vision tasks on different types of visual data. To our best knowledge, the problems considered in this paper (i.e., saliency detection and object tracking) have never been addressed by PDE models before. Experimental results on various challenging benchmark databases show the superiority of LTD against existing state-of-the-art methods for all the tested visual analysis tasks.

A calligram is an arrangement of words or letters that creates a visual image, and a compact calligram fits one word into a 2D shape. We introduce a fully automatic method for the generation of legible compact calligrams which provides a balance between conveying the input shape, legibility, and aesthetics. Our method has three key elements: a path generation step which computes a global layout path suitable for embedding the input word; an alignment step to place the letters so as to achieve feature alignment between letter and shape protrusions while maintaining word legibility; and a final deformation step which deforms the letters to fit the shape while balancing fit against letter legibility. As letter legibility is critical to the quality of compact calligrams, we conduct a large-scale crowd-sourced study on the impact of different letter deformations on legibility and use the results to train a letter legibility measure which guides the letter deformation. We show automatically generated calligrams on an extensive set of word-image combinations. The legibility and overall quality of the calligrams are evaluated and compared, via user studies, to those produced by human creators, including a professional artist, and existing works.

In this paper, we propose least-squares images (LS-images) as a basis for a novel edge-preserving image smoothing method. The LS-image requires the value of each pixel to be a convex linear combination of its neighbors, i.e., to have zero Laplacian, and to approximate the original image in a least-squares sense. The edge-preserving property inherits from the edge-aware weights for constructing the linear combination. Experimental results demonstrate that the proposed method achieves high quality results compared to previous state-of-the-art works. We also show diverse applications of LS-images, such as detail manipulation, edge enhancement, and clip-art JPEG artifact removal.

We present a deformation-driven approach to topology-varying 3D shape correspondence. In this paradigm, the best correspondence between two shapes is the one that results in a minimal-energy, possibly topology-varying, deformation that transforms one shape to conform to the other while respecting the correspondence. Our deformation model, called GeoTopo transform , allows both geometric and topological operations such as part split, duplication, and merging, leading to fine-grained and piecewise continuous correspondence results. The key ingredient of our correspondence scheme is a deformation energy that penalizes geometric distortion, encourages structure preservation, and simultaneously allows topology changes. This is accomplished by connecting shape parts using structural rods , which behave similarly to virtual springs but simultaneously allow the encoding of energies arising from geometric, structural, and topological shape variations. Driven by the combined deformation energy, an optimal shape correspondence is obtained via a pruned beam search. We demonstrate our deformation-driven correspondence scheme on extensive sets of man-made models with rich geometric and topological variation and compare the results to state-of-the-art approaches.

Partial Differential Equations (PDEs) have been successful in solving many low-level vision tasks. However, it is a challenging task to directly utilize PDEs for visual saliency detection due to the difficulty in incorporating human perception and high-level priors to a PDE system. Instead of designing PDEs with fixed formulation and boundary condition, this paper proposes a novel framework for adaptively learning a PDE system from an image for visual saliency detection. We assume that the saliency of image elements can be carried out from the relevances to the saliency seeds (i.e., the most representative salient elements). In this view, a general Linear Elliptic System with Dirichlet boundary (LESD) is introduced to model the diffusion from seeds to other relevant points. For a given image, we first learn a guidance map to fuse human prior knowledge to the diffusion system. Then by optimizing a discrete submodular function constrained with this LESD and a uniform matroid, the saliency seeds (i.e., boundary conditions) can be learnt for this image, thus achieving an optimal PDE system to model the evolution of visual saliency. Experimental results on various challenging image sets show the superiority of our proposed learning-based PDEs for visual saliency detection.

We introduce an algorithm for generating novel 3D models via topology-varying shape blending. Given a source and a target shape, our method blends them topologically and geometrically, producing continuous series of in-betweens as new shape creations. The blending operations are defined on a spatio-structural graph composed of medial curves and sheets. Such a shape abstraction is structure-oriented, part-aware, and facilitates topology manipulations. Fundamental topological operations including split and merge are realized by allowing one-to-many correspondences between the source and the target. Multiple blending paths are sampled and presented in an interactive, exploratory tool for creative 3D modeling. We show a variety of topology-varying 3D shapes generated via continuous structural blending between man-made shapes exhibiting complex topological differences, in real time.

A shape signature is defined as any 1-D function on a shape, which is a compact and concise representation for some essence of the shape. Although a variety of shape signatures are proposed and utilized in shape retrieval and recognition tasks, the existing signatures cannot yet provide entirely satisfactory solutions to describe the shape variations well, especially when significant noise or articulation occurs. Motivated by the fact that electrical charge distributions are almost the same for similar shapes but not vice versa when shapes reach their electrical equilibrium condition, we propose a novel shape signature based on the electrical charge distribution on the shape (ECDS). Compared to other shape descriptors, ECDS is more intuitively and robust, which is computed in a global manner. Furthermore, as well as being invariant to translation, scale and rotation, ECDS is articulation insensitive and therefore exhibits better performance by the introduction of generalized coulomb potentials. This allows it to better match shapes whose parts can move independently, such as scissors. Finally, numerous experiments have done on public databases, demonstrating that ECDS has the above properties and compares well with other shape descriptors in many kinds of shape retrieval and recognition tasks.

Geometry images are important representations of 3D geometry models. Smooth geometry images contributes to low approximation error and high image compressibility. We present an adapted parameterization method to generate a smooth geometry image. It is quite challenging to directly modify the parameter domain to smooth geometry images. Our novel idea is that we use an indirect way to construct a resulting parameter domain according to the desired geometry image. We first move image pixels according to the current parameter domain to decrease the local linear error. Then we formulate a relationship between the moved image pixels and the current parameter domain. Finally, we use the relationship to update the parameter domain by restituting the image pixels to their original positions. The process will continue until the local linear error is less than a given threshold or the number of iterations is larger than a given threshold. Experimental results illustrate that geometry images generated by our method have low linear errors and low approximation errors under different sampling resolutions.

Conventional saliency detection approaches are human fixation detection and single dominant region detection. However, real-world photographs usually consist of multiple dominant regions. We propose a saliency detection method with the aim to highlight objects as a whole and distinguish objects with different saliency levels. It combines the bottom-up approach and top-down approach via two nested levels of hierarchical segmentations - the coarse level objects and fine level details. We first calculate a preliminary saliency on the fine patches with a random walk model. Then a location cue and an object-level cue are fused to refine the preliminary saliency to emphasize the objects against the background. At last, the object-level saliency map is synthesized via a heat diffusion process restricted by the coarse level patches to enhance object saliency and distinguish saliency between different objects. Extensive evaluation on a publicly available database verifies that our method outperforms the state-of-the-art algorithms.

Abstract The word ‘style’ can be interpreted in so many different ways in so many different contexts. To provide a general analysis and understanding of styles is a highly challenging problem. We pose the open question ‘how to extract styles from geometric shapes?’ and address one instance of the problem. Specifically, we present an unsupervised algorithm for identifying curve styles in a set of shapes. In our setting, a curve style is explicitly represented by a mode of curve features appearing along the 2D silhouettes of the shapes in the set. Unlike previous attempts, we do not rely on any preconceived conceptual characterisations, for example, via specific shape descriptors, to define what is or is not a style. Our definition of styles is data‐dependent; it depends on the input set but we do not require computing a shape correspondence across the set. We provide an operational definition of curve styles which focuses on separating curve features that represent styles from curve features that are content revealing. To this end, we develop a novel formulation and associated algorithm for style‐content separation. The analysis is based on a feature‐shape association matrix (FSM) whose rows correspond to modes of curve features, columns to shapes in the set, and each entry expresses the extent a feature mode is present in a shape. We make several assumptions to drive style‐content separation which only involve properties of, and relations between, rows of the FSM. Computationally, our algorithm only requires row‐wise correlation analysis in the FSM and a heuristic solution of an instance of the set cover problem. Results are demonstrated on several data sets showing the identification of curve styles. We also develop and demonstrate several style‐related applications including style exaggeration, removal, blending, and style transfer for 2D shape synthesis.

Abstract The emergence of laser/LiDAR sensors, reliable multi‐view stereo techniques and more recently consumer depth cameras have brought point clouds to the forefront as a data format useful for a number of applications. Unfortunately, the point data from those channels often incur imperfection, frequently contaminated with severe outliers and noise. This paper presents a robust consolidation algorithm for low‐quality point data from outdoor scenes, which essentially consists of two steps: 1) outliers filtering and 2) noise smoothing. We first design a connectivity‐based scheme to evaluate outlierness and thereby detect sparse outliers. Meanwhile, a clustering method is used to further remove small dense outliers. Both outlier removal methods are insensitive to the choice of the neighborhood size and the levels of outliers. Subsequently, we propose a novel approach to estimate normals for noisy points based on robust partial rankings, which is the basis of noise smoothing. Accordingly, a fast approach is exploited to smooth noise, while preserving sharp features. We evaluate the effectiveness of the proposed method on the point clouds from a variety of outdoor scenes.

We propose a robust method for detecting features on triangular meshes by combining normal tensor voting with neighbor supporting. Our method contains two stages: feature detection and feature refinement. First, the normal tensor voting method is modified to detect the initial features, which may include some pseudo features. Then, at the feature refinement stage, a novel salient measure deriving from the idea of neighbor supporting is developed. Benefiting from the integrated reliable salient measure feature, pseudo features can be effectively discriminated from the initially detected features and removed. Compared to previous methods based on the differential geometric property, the main advantage of our method is that it can detect both sharp and weak features. Numerical experiments show that our algorithm is robust, effective, and can produce more accurate results. We also discuss how detected features are incorporated into applications, such as feature-preserving mesh denoising and hole-filling, and present visually appealing results by integrating feature information.

We propose a novel curvature-aware simplification technique for point-sampled geometry based on the locally optimal projection (LOP) operator. Our algorithm includes two new developments. First, a weight term related to surface variation at each point is introduced to the classic LOP operator. It produces output points with a spatially adaptive distribution. Second, for speeding up the convergence of our method, an initialization process is proposed based on geometry-aware stochastic sampling. Owing to the initialization, the relaxation process achieves a faster convergence rate than those initialized by uniform sampling. Our simplification method possesses a number of distinguishing features. In particular, it provides resilience to noise and outliers, and an intuitively controllable distribution of simplification. Finally, we show the results of our approach with publicly available point cloud data, and compare the results with those obtained using previous methods. Our method outperforms these methods on raw scanned data.

We present a novel algorithm to reconstruct curves with self-intersections and multiple parts from unorganized strip-shaped points, which may have different local shape scales and sampling densities. We first extract an initial curve, a graph composed of polylines, to model the different structures of the points. Then a least-squares optimization is used to improve the geometric approximation. The initial curve is extracted in three steps: anisotropic farthest point sampling with an adaptable sphere, graph construction followed by non-linear region identification, and edge refinement. Our algorithm produces faithful results for points sampled from non-simple curves without pre-segmenting them. Experiments on many simulated and real data demonstrate the efficiency of our method, and more faithful curves are reconstructed compared to other existing methods.

We present an algorithm for curve skeleton extraction via Laplacian-based contraction. Our algorithm can be applied to surfaces with boundaries, polygon soups, and point clouds. We develop a contraction operation that is designed to work on generalized discrete geometry data, particularly point clouds, via local Delaunay triangulation and topological thinning. Our approach is robust to noise and can handle moderate amounts of missing data, allowing skeleton-based manipulation of point clouds without explicit surface reconstruction. By avoiding explicit reconstruction, we are able to perform skeleton-driven topology repair of acquired point clouds in the presence of large amounts of missing data. In such cases, automatic surface reconstruction schemes tend to produce incorrect surface topology. We show that the curve skeletons we extract provide an intuitive and easy-to-manipulate structure for effective topology modification, leading to more faithful surface reconstruction.

One major goal of mesh parameterization is to minimize the conformal distortion. Measured boundary parameterizations focus on lowering the distortion by setting the boundary free with the help of distance from a center vertex to all the boundary vertices. Hence these parameterizations strongly depend on the determination of the center vertex. In this paper, we introduce two methods to determine the center vertex automatically. Both of them can be used as necessary supplements to the existing measured boundary methods to minimize the common artifacts as a result of the obscure choice of the center vertex. In addition, we propose a simple and fast measured boundary parameterization method based on the Poisson’s equation. Our new approach generates less conformal distortion than the fixed boundary methods. It also generates more regular domain boundaries than other measured boundary methods. Moreover, it offers a good tradeoff between computation costs and conformal distortion compared with the fast and robust angle based flattening (ABF++).

In this paper, we propose a novel triangle mesh denoising method based on the differential coordinates. The proposed approach consists of the application of the mean filter to differential coordinates of the mesh and the reconstruction of mesh vertices' Cartesian coordinates to make them fit to the modified differential coordinates. The presented method is simple, stable and able to effectively remove large noise. Experimental results demonstrate that the proposed Mesh Mean Filter does not cause surface shrinkage and shape distortion during the denoising process, and preserves geometric detail features to a certain extent.