Text-to-image (T2I) diffusion models excel at generating photorealistic images but often fail to render accurate spatial relationships. We identify two core issues underlying this common failure: 1) the ambiguous nature of data concerning spatial relationships in existing datasets, and 2) the inability of current text encoders to accurately interpret the spatial semantics of input descriptions. We propose CoMPaSS, a versatile framework that enhances spatial understanding in T2I models. It first addresses data ambiguity with the Spatial Constraints-Oriented Pairing (SCOP) data engine, which curates spatially-accurate training data via principled constraints. To leverage these priors, CoMPaSS also introduces the Token ENcoding ORdering (TENOR) module, which preserves crucial token ordering information lost by text encoders, thereby reinforcing the prompt's linguistic structure. Extensive experiments on four popular T2I models (UNet and MMDiT-based) show CoMPaSS sets a new state of the art on key spatial benchmarks, with substantial relative gains on VISOR (+98%), T2I-CompBench Spatial (+67%), and GenEval Position (+131%). Code is available at https://github.com/blurgyy/CoMPaSS.

Online free-view navigation in volumetric videos requires high-quality rendering and real-time streaming in order to provide immersive user experiences. However, existing methods ( e.g. , dynamic NeRF and 3DGS) may not handle dynamic scenes with complex motions, and their models may not be streamable due to storage and bandwidth constraints. In this paper, we propose a novel 4D Gaussian Video (4DGV) approach that enables the creation and streaming of photorealistic, volumetric videos for dynamic scenes over the Internet. The core of our 4DGV is a novel streamable group of Gaussians (GOG) representation based on motion layering. Each GOG consists of static and dynamic points obtained via lifting 2D segmentation into 3D in motion layering, where the deformation of each dynamic point is represented as the temporal offset of its attributes. We also adaptively convert static points back to dynamic points to handle the appearance change, (e.g. , moving shadows and reflections), of static objects through optimization. To support real-time streaming of 4DGVs, we show that by applying quantization on Gaussian attributes and H.265 encoding on deformation offsets, our GOG representation can be significantly compressed (to around 6% of the original model size) without sacrificing the accuracy (PSNR loss less than 0.01dB). Extensive experiments on standard benchmarks demonstrate that our method outperforms state-of-the-art volumetric video approaches, with superior rendering quality and minimum storage overheads.

Abstract The Hausdorff distance is one of the most fundamental metrics for comparing 3D shapes. To compute the Hausdorff distance efficiently from a triangular mesh to another triangular mesh , one needs to cull the unnecessary triangles on quickly. These triangles have no chance to improve the Hausdorff distance estimation, that is the parts with local upper bound smaller than the global lower bound. The local upper bound estimation should be tight, use fast distance computation, and involve a small number of triangles in during the reduction phase for efficiency. In this paper, we propose to use point‐triangle distance, and only involve at most four triangles in in the reduction phase. Comparing with the state‐of‐the‐art proposed by Tang et al. in 2009, which uses more costly triangle‐triangle distance and may involve a large number of triangles in reduction phase, our local upper bound estimation is faster, and with only a small impact on the tightness of the bound on error estimation. Such a more economic strategy boosts the overall performance significantly. Experiments on the Thingi10K dataset show that our method can achieve several (even over 20) times speedup on average. On a few models with different placements and resolutions, we show that close placement and large difference in resolution bring big challenges to Hausdorff distance computation, and explain why our method can achieve more significant speedup on challenging cases.

This paper focuses on the problem of low-rank tensor completion, the goal of which is to recover an underlying low-rank tensor from incomplete observations. Our method is motivated by the recently proposed t-product [8] based on any invertible linear transforms. First, we define the new tensor average rank under the invertible real linear trans-forms. We then propose a tensor completion model using a nonconvex surrogate to approximate the tensor average rank. This surrogate overcomes the discontinuity of the tensor average rank and alleviates the bias problem caused by the convex relaxation. Further, we develop an efficient algorithm to solve the proposed model and establish its convergence. Finally, experimental results on both synthetic and real data demonstrate the superiority of our method.

We present a multiview method for markerless motion capture of multiple people. The main challenge in this problem is to determine crossview correspondences for the 2D joints in the presence of noise. We propose a 3D hypothesis clustering technique to solve this problem. The core idea is to transform joint matching in 2D space into a clustering problem in a 3D hypothesis space. In this way, evidence from photometric appearance, multiview geometry, and bone length can be integrated to solve the clustering problem efficiently and robustly. Each cluster encodes a set of matched 2D joints for the same person across different views, from which the 3D joints can be effectively inferred. We then assemble the inferred 3D joints to form full-body skeletons for all persons in a bottom-up way. Our experiments demonstrate the robustness of our approach even in challenging cases with heavy occlusion, closely interacting people, and few cameras. We have evaluated our method on many datasets, and our results show that it has significantly lower estimation errors than many state-of-the-art methods.

This article presents a global 3D human pose estimation method for markerless motion capture. Given two calibrated images of a person, it first obtains the 2D joint locations in the images using a pre-trained 2D Pose CNN, then constructs the 3D pose based on stereo triangulation. To improve the accuracy and the stability of the system, we propose two efficient optimization techniques for the joints. The first one, called cross-view refinement, optimizes the joints based on epipolar geometry. The second one, called cross-joint refinement, optimizes the joints using bone-length constraints. Our method automatically detects and corrects the unreliable joint, and consequently is robust against heavy occlusion, symmetry ambiguity, motion blur, and highly distorted poses. We evaluate our method on a number of benchmark datasets covering indoors and outdoors, which showed that our method is better than or on par with the state-of-the-art methods. As an application, we create a 3D human pose dataset using the proposed motion capture system, which contains about 480K images of both indoor and outdoor scenes, and demonstrate the usefulness of the dataset for human pose estimation.

This paper presents a new method for single-image pose estimation of multiple people combining the traditional bottom-up and the top-down methods. Specifically, we extract features from the input image by a residual network and use a multistage CNN to learn both the confidence maps of joints and the connection relationships, between joints. During testing, we perform the network feedforwarding in a bottom-up manner, and then use the predicted confidence maps, the connection relationships, and the corresponding bounding boxes to parse the poses of all people in a top-down manner. In contrast to the previous top-down methods, our method is robust to bounding box shift and tightness, works well for largely overlapped people, and achieves faster running speed. In contrast to the bottom-up method, our method avoids mistake propagation across different people, and addresses disconnected joints effectively. To estimate human pose from videos, we impose a weight-sharing scheme to the multi-stage CNN, and rewrite it as a recurrent neural network. Thus, we can reuse the prediction results from the previous frames so as to reduce the total stage number, yielding significantly faster speed in invoking the network on videos. And we adopt LSTM units between frames to capture the temporal correlation among video frames. We found that LSTM handles input-quality degradation in videos well and successfully stabilizes the sequential outputs.

Visual Simultaneous Localization and Mapping (SLAM) is very important in various applications such as AR, Robotics, etc. A challenging problem in SLAM is the inferior tracking performance in the low-texture environment due to their low-level feature based tactic. This paper presents a novel SLAM system which leverages feature-wise alignment and layout information to handle this challenge. The key idea is to utilize the points, lines and planes with both feature-wise constraints and layout consistency constraints to obtain robust motion estimation. Firstly, we extract the points, lines and planes from color images and depth images. Then, we find some matches between these geometric primitives. Lastly, we propose a unified solution to represent the local primitives information and the layout context among the extracted primitives, thus to stabilize the camera motion estimation. Experiments on TUM datasets demonstrate that our method outperforms other RGB-D SLAM systems in the low-texture environment, and provides comparable results in the richly-textured environment.

In this work, we propose a new method for multi-person pose estimation which combines the traditional bottom-up and the top-down methods. Specifically, we perform the network feed-forwarding in a bottom-up manner, and then parse the poses with bounding box constraints in a top-down manner. In contrast to the previous top-down methods, our method is robust to bounding box shift and tightness. We extract features from an original image by a residual network and train the network to learn both the confidence maps of joints and the connection relationships between joints. During testing, the predicted confidence maps, the connection relationships and the bounding boxes are used to parse the poses of all persons. The experimental results showed that our method learns more accurate human poses especially in challenging situations and gains better time performance, compared with the bottom-up and the top-down methods.

Abstract Representing rotational symmetry vector as a set of vectors is not suitable for design due to lacking of a consistent ordering for measurement. In this paper we introduce a spectral method to find rotation invariant harmonic functions for symmetry vector field design. This method is developed for 3D vector fields, but it is applicable in 2D. Given the finite symmetry group G of a symmetry vector fieldv(x) on a 3D domain Ω, we formulate the harmonic function h(s) as a stationary point of group G. Using the real spherical harmonic (SH) bases, we showed the coefficients of the harmonic functions are an eigenvector of the SH rotation matrices corresponding to group G. Instead of solving eigen problems to obtain the eigenvector, we developed a forward constructive method based on orthogonal group theory. The harmonic function found by our method is not only invariant under G, but also expressive and can distinguish different rotations with respect to G. At last, we demonstrate some vector field design results with tetrahedron‐symmetry, cube‐symmetry and dodecahedron‐symmetry groups.

Instead of computing on a large number of virtual point lights (VPLs), scalable many-lights rendering methods effectively simulate various illumination effects only using hundreds or thousands of representative VPLs. However, gathering illuminations from these representative VPLs, especially computing the visibility, is still a tedious and time-consuming task. In this paper, we propose a new matrix sampling-and-recovery scheme to efficiently gather illuminations by only sampling a small number of visibilities between representative VPLs and surface points. Our approach is based on the observation that the lighting matrix used in manylights rendering is of low-rank, so that it is possible to sparsely sample a small number of entries, and then numerically complete the entire matrix. We propose a three-step algorithm to explore this observation. First, we design a new VPL clustering algorithm to slice the rows and group the columns of the full lighting matrix into a number of reduced matrices, which are sampled and recovered individually. Second, we propose a novel prediction method that predicts visibility of matrix entries from sparsely and randomly sampled entries. Finally, we adapt the matrix separation technique to recover the entire reduced matrix and compute final shadings. Experimental results show that our method heavily reduces the required visibility sampling in the final gathering and achieves 3--7 times speedup compared with the state-of-the-art methods on test scenes.

Existing methods on passive facial performance capture assume that the input images are well captured. They merely consider how to deal with motion blurred images in the input sequence, which is very common in the image capture process. This paper presents a collection of novel algorithms and a thereby resulting system to reconstruct high quality facial dynamic geometry even from a partially blurred image sequence. In our method, we adopt binocular cameras to capture a stereo sequence. With this sequence, we first estimate depth map for each frame using a state-of-the-art stereo matching method. Then, based on the estimated depth map sequence, we track facial motion by leveraging constraints of both optical flow and geometry. In this step, a blur detection and regularization algorithm are devised to adaptively keep both shape and details. Finally, we synthesize temporal mesoscopic geometry on the blurred region from clear image texture of neighboring frames. We conduct extensive experiments on several sequences containing facial performance with blurred region, and the results demonstrate the effectiveness and robustness of our algorithms.

This paper presents a system for robust dynamic human surface reconstruction using a single Kinect. The single Kinect provides a self-occluded and noisy RGBD data. Thus it is challenging to track the whole human surface robustly. To overcome both incompleteness and data noise, we adopt a template to confine the shape in the un-seen part, and propose a two-stage tracking pipeline. The first stage tracks the articulated motion of the human, which improves robustness of tracking by introducing more constraints between the surface points. The second stage tracks movements of non-articulated motion. For long sequences, we stabilize the human surface in the un-seen part by directly warping the surface from the first frame to the current frame according to sequentially tracked correspondences, preventing surface from collapsing caused by error accumulation. We demonstrate our method by several real captured RGBD data, containing complex human motion. The reconstruction results show the effectiveness and robustness of our method.

This paper presents a method for quasi-rigid objects modeling from a sequence of depth scans captured at different time instances. As quasi-rigid objects, such as human bodies, usually have shape motions during the capture procedure, it is difficult to reconstruct their geometries. We represent the shape motion by a deformation graph, and propose a model-to-part method to gradually integrate sampled points of depth scans into the deformation graph. Under an as-rigid-as-possible assumption, the model-to-part method can adjust the deformation graph non-rigidly, so as to avoid error accumulation in alignment, which also implicitly achieves loop-closure. To handle the drift and topological error for the deformation graph, two algorithms are introduced. First, we use a two-stage registration to largely keep the rigid motion part. Second, in the step of graph integration, we topology-adaptively integrate new parts and dynamically control the regularization effect of the deformation graph. We demonstrate the effectiveness and robustness of our method by several depth sequences of quasi-rigid objects, and an application in human shape modeling.

Many natural and man-made objects consist of simple primitives, similar components, and various symmetry structures. This paper presents a divide-and-conquer quadrangulation approach that exploits such global structural information. Given a model represented in triangular mesh, we first segment it into a set of submeshes, and compare them with some predefined quad mesh templates. For the submeshes that are similar to a predefined template, we remesh them as the template up to a number of subdivisions. For the others, we adopt the wave-based quadrangulation technique to remesh them with extensions to preserve symmetric structure and generate compatible quad mesh boundary. To ensure that the individually remeshed submeshes can be seamlessly stitched together, we formulate a mixed-integer optimization problem and design a heuristic solver to optimize the subdivision numbers and the size fields on the submesh boundaries. With this divider-and-conquer quadrangulation framework, we are able to process very large models that are very difficult for the previous techniques. Since the submeshes can be remeshed individually in any order, the remeshing procedure can run in parallel. Experimental results showed that the proposed method can preserve the high-level structures, and process large complex surfaces robustly and efficiently.

ABSTRACT It is important in computer animation to synthesize expressive facial animation for avatars from videos. Some traditional methods track a set of semantic feature points on the face to drive the avatar. However, these methods usually suffer from inaccurate detection and sparseness of the feature points and fail to obtain high‐level understanding of facial expressions, leading to less expressive and even wrong expressions on the avatar. In this paper, we propose a state‐aware synthesis framework. Instead of simply fitting 3D face to the 2D feature points, we use expression states obtained by a set of low‐cost classifiers (based on local binary pattern and support vector machine) on the face texture to guide the face fitting procedure. Our experimental results show that the proposed hybrid framework enjoys the advantages of the original methods based on feature point and the awareness of the expression states of the classifiers and thus vivifies and enriches the face expressions of the avatar. Copyright © 2012 John Wiley & Sons, Ltd.

The traditional parameterization methods focused on preserving the local geometry properties by minimizing the distortions of angle and stretch. In this paper, we present a mesh parameterization method to preserve some global geometry properties, such as symmetry structure, of the input mesh. Given an input triangular mesh, the symmetry regions can be automatically detected using shape analysis method or manually specified by the users. For each vertex in the symmetry regions, we can identify the symmetric point and represent it as a triangle index and the corresponding bary centric coordinates. We extend the harmonic map to parameterize the input mesh by adding new error metric to measure how the symmetry property of the point pairs are preserved. Since the new error metric is non-linear, we develop an iterative update method to solve the parameterization problem. At last, we show some structure preservation parameterization results, and compare them with the results of the traditional harmonic map.

This paper proposes a new method for remeshing a surface into anisotropically sized quads. The basic idea is to construct a special standing wave on the surface to generate the global quadrilateral structure. This wave based quadrangulation method is capable of controlling the quad size in two directions and precisely aligning the quads with feature lines. Similar to the previous methods, we augment the input surface with a vector field to guide the quad orientation. The anisotropic size control is achieved by using two size fields on the surface. In order to reduce singularity points, the size fields are optimized by a new curl minimization method. The experimental results show that the proposed method can successfully handle various quadrangulation requirements and complex shapes, which is difficult for the existing state-of-the-art methods.

This article presents a global optimization approach to reconstruct surfaces from a single document image. Instead of assuming globally developable in previous works which restricted the surface to be cylindrical, conical, etc, we use a free form parametric model which is simple yet expressive enough in the reconstruction task. We then apply developable constraints locally on sample points from extracted feature curves. And in order to further achieve better stability, the developable constraint we put on it is not so strong. Instead of isometric or conformal constraints used frequently in surface parametrization tasks, we use orthogonality. We show that even this is enough to reconstruct a wide class of document surfaces even with an uncalibrated camera.

Abstract Recent soft shadow mapping techniques based on back‐projection can render high quality soft shadows in real time. However, real time high quality rendering of large penumbrae is still challenging, especially when multilayer shadow maps are used to reduce single light sample silhouette artifact. In this paper, we present an efficient algorithm to attack this problem. We first present a GPU‐friendly packet‐based approach rendering a packet of neighboring pixels together to amortize the cost of computing visibility factors. Then, we propose a hierarchical technique to quickly locate the contour edges, further reducing the computation cost. At last, we suggest a multi‐view shadow map approach to reduce the single light sample artifact. We also demonstrate its higher image quality and higher efficiency compared to the existing depth peeling approaches.

Abstract It is a fundamental but challenging problem to efficiently edit complex 3D objects. By embedding the input models into coarse tetrahedral control meshes, this paper develops a unified framework to discuss two useful editing operations: interactive deformation and deformation transfer. First, a new rigidity energy is proposed to make the tetrahedral control mesh deform as rigidly as possible, which yields intuitive detail and volume preservation even under large deformations. And an error‐driven refinement approach is presented to further improve the deformation result. Then, based on this deformation scheme, a volumetric correspondence method is introduced to perform the deformation transfer task between the tetrahedral control meshes of the source and target models, which greatly lessens the burden of the user. Experimental results show our algorithm is effective, easy to control, supports various shape representations, and well transfers deformations between non‐homeomorphous models. Copyright © 2009 John Wiley & Sons, Ltd.

It is a challenging problem to interactively deform densely sampled complex objects. This paper proposed an easy but efficient approach to it by using coarse control meshes to embed the target objects. The control mesh can be efficiently deformed by various existing methods, and then the target object can be accordingly deformed by interpolation. One of the most simplest interpolation methods is to use the barycentric coordinates, which however generates apparent first-order discontinuity artifacts across the boundary due to its piecewise linear property. To avoid such artifacts, this paper introduced a modified barycentric interpolation (modified-BI) technique. The central idea is to add a local transformation at each control vertex for interpolation, so that we can minimize the first-order discontinuity by optimizing the local transformations. We also minimize the second order derivatives of the interpolation function to avoid undesired vibrations. While focus on deforming 3D objects embedded in tetrahedron meshes, the proposed method is applicable to 2D image objects embed in planar triangular meshes. The experimental results in both 2D and 3D demonstrated the success and advantages of the proposed method.

This paper presents a new quadrangulation algorithm, extending the spectral surface quadrangulation approach where the coarse quadrangular structure is derived from the Morse-Smale complex of an eigenfunction of the Laplacian operator on the input mesh. In contrast to the original scheme, we provide flexible explicit controls of the shape, size, orientation and feature alignment of the quadrangular faces. We achieve this by proper selection of the optimal eigenvalue (shape), by adaption of the area term in the Laplacian operator (size), and by adding special constraints to the Laplace eigenproblem (orientation and alignment). By solving a generalized eigen-problem we can generate a scalar field on the mesh whose Morse-Smale complex is of high quality and satisfies all the user requirements. The final quadrilateral mesh is generated from the Morse-Smale complex by computing a globally smooth parametrization. Here we additionally introduce edge constraints to preserve user specified feature lines accurately.

It is difficult to render caustic patterns at interactive frame rates. This paper introduces new rendering techniques that relax current constraints, allowing scenes with moving, non-rigid scene objects, rigid caustic objects, and rotating directional light sources to be rendered in real-time with GPU hardware acceleration. Because our algorithm estimates the intensity and the direction of caustic light, rendering of non-Lambertian surfaces is supported. Previous caustics algorithms have separated the problem into pre-rendering and rendering phases, storing intermediate results in data structures such as photon maps or radiance transfer functions. Our central idea is to use specially parameterized spot lights, called caustic spot lights (CSLs), as the intermediate representation of a two-phase algorithm. CSLs are flexible enough that a small number can approximate the light leaving a caustic object, yet simple enough that they can be efficiently evaluated by a pixel shader program during accelerated rendering.We extend our approach to support changing lighting direction by further dividing the pre-rendering phase into per-scene and per-frame components: the per-frame phase computes frame-specific CSLs by interpolating between CSLs that were pre-computed with differing light directions.

Abstract In this paper, we introduce a new representation – radiance transfer fields (RTF) – for rendering interreflections in dynamic scenes under low frequency illumination. The RTF describes the radiance transferred by an individual object to its surrounding space as a function of the incident radiance. An important property of RTF is its independence of the scene configuration, enabling interreflection computation in dynamic scenes. Secondly, RTFs naturally fit in with the rendering framework of precomputed shadow fields, incurring negligible cost to add interreflection effects. In addition, RTFs can be used to compute interreflections for both diffuse and glossy objects. We also show that RTF data can be highly compressed by clustered principal component analysis (CPCA), which not only reduces the memory cost but also accelerates rendering. Finally, we present some experimental results demonstrating our techniques.

Abstract Shell becomes popular in a variety of modeling techniques for representing small‐scale features and increasing visual complexity. Current shell generation algorithms do not measure the volume distortion for geometric texture mapping. And when the object deforms, it is very challenging for existing algorithms to wrap the space inside the shell without large artifacts. We propose an approach to these problems by minimizing the difference between the deformation gradient of the space in the shell and the rotation component of it. Because the shell space is warped as rigid as possible, small features embedded in the shell can be preserved well. Furthermore, our algorithm can achieve shell‐like object deformation. We introduce a novel hierarchical dimension reduction method to solve the involved nonlinear optimization problem efficiently. Finally, several examples are presented to demonstrate the usefulness of our algorithm. Copyright © 2007 John Wiley & Sons, Ltd.

Abstract This paper presents a novel interactive mesh deformation method that can achieve various dynamic material effects, including elastic membrane and cloth effects. In our framework, a mesh is encoded by some differential quantities based on edge length and dihedral angle; and the deformation is formulated as a least square problem for preserving the edge length and dihedral angle via the differential quantities. In order to obtain anisotropic material effects, we further propose an edge‐weighting scheme based on a user‐specified vector field. To avoid specifying the local transformations, we set up an iterative scheme for solving the deformation. At last, several examples are presented to show that our approach can interactively generate visually pleasing deformations. Copyright © 2006 John Wiley & Sons, Ltd.

In this paper we present a general framework for performing constrained mesh deformation tasks with gradient domain techniques. We present a gradient domain technique that works well with a wide variety of linear and nonlinear constraints. The constraints we introduce include the nonlinear volume constraint for volume preservation, the nonlinear skeleton constraint for maintaining the rigidity of limb segments of articulated figures, and the projection constraint for easy manipulation of the mesh without having to frequently switch between multiple viewpoints. To handle nonlinear constraints, we cast mesh deformation as a nonlinear energy minimization problem and solve the problem using an iterative algorithm. The main challenges in solving this nonlinear problem are the slow convergence and numerical instability of the iterative solver. To address these issues, we develop a subspace technique that builds a coarse control mesh around the original mesh and projects the deformation energy and constraints onto the control mesh vertices using the mean value interpolation. The energy minimization is then carried out in the subspace formed by the control mesh vertices. Running in this subspace, our energy minimization solver is both fast and stable and it provides interactive responses. We demonstrate our deformation constraints and subspace deformation technique with a variety of constrained deformation examples.

Previous methods for soft shadows numerically integrate over many light directions at each receiver point, testing blocker visibility in each direction. We introduce a method for real-time soft shadows in dynamic scenes illuminated by large, low-frequency light sources where such integration is impractical. Our method operates on vectors representing low-frequency visibility of blockers in the spherical harmonic basis. Blocking geometry is modeled as a set of spheres; relatively few spheres capture the low-frequency blocking effect of complicated geometry. At each receiver point, we compute the product of visibility vectors for these blocker spheres as seen from the point. Instead of computing an expensive SH product per blocker as in previous work, we perform inexpensive vector sums to accumulate the log of blocker visibility. SH exponentiation then yields the product visibility vector over all blockers. We show how the SH exponentiation required can be approximated accurately and efficiently for low-order SH, accelerating previous CPU-based methods by a factor of 10 or more, depending on blocker complexity, and allowing real-time GPU implementation.

We present a fast deformation method for flexible objects. The deformation of the object is physically modeled using a linear elasticity model with a displacement based finite elements method, yielding a linear system at each time step of simulation. We solve this linear system using a precomputed force-displacement matrix, which describes the object response in terms of displacement accelerations to the forces acting on each vertex. We exploit the spatial coherence to effectively compress the force-displacement matrix to make this method practical and efficient by applying the clustered principal component analysis method. And we developed a method to efficiently handle the additional constraints for interactive user manipulation. At last large deformations are addressed based upon the compressed force-displacement matrix by combining a domain decomposition method and tracking the rotational motions. The experimental results demonstrate fast performances on complex large scale objects under interactive user manipulations.

We present a novel technique for large deformations on 3D meshes using the volumetric graph Laplacian. We first construct a graph representing the volume inside the input mesh. The graph need not form a solid meshing of the input mesh's interior; its edges simply connect nearby points in the volume. This graph's Laplacian encodes volumetric details as the difference between each point in the graph and the average of its neighbors. Preserving these volumetric details during deformation imposes a volumetric constraint that prevents unnatural changes in volume. We also include in the graph points a short distance outside the mesh to avoid local self-intersections. Volumetric detail preservation is represented by a quadric energy function. Minimizing it preserves details in a least-squares sense, distributing error uniformly over the whole deformed mesh. It can also be combined with conventional constraints involving surface positions, details or smoothness, and efficiently minimized by solving a sparse linear system.We apply this technique in a 2D curve-based deformation system allowing novice users to create pleasing deformations with little effort. A novel application of this system is to apply nonrigid and exaggerated deformations of 2D cartoon characters to 3D meshes. We demonstrate our system's potential with several examples.

The bidirectional texture function (BTF) is a 6D function that describes the appearance of a real-world surface as a function of lighting and viewing directions. The BTF can model the fine-scale shadows, occlusions, and specularities caused by surface mesostructures. In this paper, we present algorithms for efficient synthesis of BTFs on arbitrary surfaces and for hardware-accelerated rendering. For both synthesis and rendering, a main challenge is handling the large amount of data in a BTF sample. To addresses this challenge, we approximate the BTF sample by a small number of 4D point appearance functions (PAFs) multiplied by 2D geometry maps. The geometry maps and PAFs lead to efficient synthesis and fast rendering of BTFs on arbitrary surfaces. For synthesis, a surface BTF can be generated by applying a texton-based sysnthesis algorithm to a small set of 2D geometry maps while leaving the companion 4D PAFs untouched. As for rendering, a surface BTF synthesized using geometry maps is well-suited for leveraging the programmable vertex and pixel shaders on the graphics hardware. We present a real-time BTF rendering algorithm that runs at the speed of about 30 frames/second on a mid-level PC with an ATI Radeon 8500 graphics card. We demonstrate the effectiveness of our synthesis and rendering algorithms using both real and synthetic BTF samples.

In this paper, we propose a novel shape representation we call directional histogram model (DHM). It captures the shape variation of an object and is invariant to scaling and rigid transforms. The DHM is computed by first extracting a directional distribution of thickness histogram signatures, which are translation invariant. We show how the extraction of the thickness histogram distribution can be accelerated using conventional graphics hardware. Orientation invariance is achieved by computing the spherical harmonic transform of this distribution. Extensive experiments show that the DHM is capable of high discrimination power and is robust to noise.

Radiance transfer represents how generic source lighting is shadowed and scattered by an object to produce view-dependent appearance. We generalize by rendering transfer at two scales. A macro-scale is coarsely sampled over an object's surface, providing global effects like shadows cast from an arm onto a body. A meso-scale is finely sampled over a small patch to provide local texture. Low-order (25D) spherical harmonics represent low-frequency lighting dependence for both scales. To render, a coefficient vector representing distant source lighting is first transformed at the macro-scale by a matrix at each vertex of a coarse mesh. The resulting vectors represent a spatially-varying hemisphere of lighting incident to the meso-scale. A 4D function, called a radiance transfer texture (RTT), then specifies the surface's meso-scale response to each lighting basis component, as a function of a spatial index and a view direction. Finally, a 25D dot product of the macro-scale result vector with the vector looked up from the RTT performs the correct shading integral. We use an id map to place RTT samples from a small patch over the entire object; only two scalars are specified at high spatial resolution. Results show that bi-scale decomposition makes preprocessing practical and efficiently renders self-shadowing and interreflection effects from dynamic, low-frequency light sources at both scales.

In this paper, we present a novel approach to synthetically generating bidirectional texture functions (BTFs) of real-world surfaces. Unlike a conventional two-dimensional texture, a BTF is a six-dimensional function that describes the appearance of texture as a function of illumination and viewing directions. The BTF captures the appearance change caused by visible small-scale geometric details on surfaces. From a sparse set of images under different viewing/lighting settings, our approach generates BTFs in three steps. First, it recovers approximate 3D geometry of surface details using a shape-from-shading method. Then, it generates a novel version of the geometric details that has the same statistical properties as the sample surface with a non-parametric sampling method. Finally, it employs an appearance preserving procedure to synthesize novel images for the recovered or generated geometric details under various viewing/lighting settings, which then define a BTF. Our experimental results demonstrate the effectiveness of our approach.

In this paper, we present a novel fairing algorithm for the removal of noise from uniform triangular meshes without shrinkage and serious distortion. The key feature of this algorithm is to keep all triangle centers invariant at each smoothing step by including some constraints in the energy minimization functional. The constrained functional is then minimized efficiently using an iterative method. Further, we apply this smoothing technique to a multiresolution representation to remove arbitrary levels of detail. A volume‐preserving decimation algorithm is presented to generate the multiresolution representation. The experimental results demonstrate the combined algorithm's stability and efficiency.

A novel progressive geometry compression scheme is presented in this paper. In this scheme, a mesh is represented as a base mesh followed by some groups of vertex split operations using an improved simplification method in which each level of the mesh can be refined into the next level by carrying out a group of vertex split operations in any order. Consequently, the progressive mesh (PM) representation can be effectively encoded by permuting the vertex split operations in each group. Meanwhile, a geometry predictor using the Laplacian operator is designed to predict each new vertex position using its neighbours. The correction is quantized and encoded using a Huffman coding scheme. Experimental results show that our algorithm obtains higher compression ratios than previous work. It is very suitable for the progressive transmission of geometric models over the Internet.