Simulating the interactions between fluids and porous media has attracted significant attention in computer graphics. A key challenge in this domain is modeling the Poro-Elasto-Capillary (PEC) coupling effect which describes the intricate interplay of three physical phenomena in soft porous materials: pore-structure evolution, elastic deformation, and wetting driven by capillary pressure. These phenomena collectively govern dynamic behavior such as the softening and fracturing of biscuits upon water absorption or the swelling of cellulose sponges due to liquid infiltration. Most existing simulation methods model porous media either as static grids or as solid particles with augmented water content attributes, failing to capture the full spectrum of PEC-driven effects due to the lack of physical modeling for elasticity, dynamic porosity changes, and capillary interactions. We propose a multiphase particle-based framework to holistically simulate PEC coupling effects with porous media. We develop a physics-driven model that captures elasticity and dynamic pore-structure evolution under capillary action, enabling realistic simulation of softening and swelling. We derive a saturation-aware pressure Poisson equation to enforce fluid incompressibility within and around the porous medium, ensuring accurate capillary-driven flow while preserving mass and momentum. Finally, we propose a representative elementary volume-based formulation to unify the modeling of homogeneous macro-porous media and cavity-embedded structures, enhancing the representation of pore-scale PEC effects. Comparisons with prior work and real footage show the advantages of our approach in achieving visually realistic fluid-porous media interactions.

Abstract Image up-scaling and super-resolution (SR) techniques have been a hot research topic for many years due to its large impact in the field of medical imaging, surveillance etc. Especially single image super-resolution (SISR) become very popular because of the fast development of deep convolution neural network (DCNN) and the low requirement on the input. They are achieving outstanding performance. However, there are still problems in the state-of-the-art works, especially from two perspectives: 1. failed at exploiting the hierarchical characteristics from the input, resulting in loss of information and artifacts in the final high resolution (HR) image; 2. failed to handle arbitrary-sized images; the existing research works are focused on fixed size input images. To address these challenges, this paper proposed a residual dense network (RDN) and multi-scale sub-pixel convolution network (MSSPCN) which are integrated into a Collapsible Linear Block Super Efficient Super-Resolution (SESR) network. The RDNs aims to tackle the first challenge, carrying the hierarchical features from end-to-end. An adaptive cropping strategy (ACS) technique is introduced before feature extraction targeting at the image size challenge. The novelty of this work is extracting the hierarchical features and integrating RDNs with MSSPCNs. The proposed network can upscale any arbitrary-sized image (1080p) to ×2 (4K) and ×4 (8K). To secure ground truth for evaluation, this paper follows the opposite flow, generating the input LR images by down-sampling the given HR images (ground truth). To evaluate the performance, the proposed algorithm is compared with eight state-of-the-art algorithms, both quantitatively and qualitatively. The results are verified on six benchmark datasets. The extensive experiments justify that the proposed architecture performs better than other methods and upscales the images satisfactorily.

We present a Lagrangian dynamic importance Monte Carlo method without non-trivial random walks for solving the Velocity-Vorticity Poisson Equation (VVPE) in Smoothed Particle Hydrodynamics (SPH) for vortical flows. Key to our approach is the use of the Kinematic Vorticity Number (KVN) to detect vortex cores and to compute the KVN-based importance of each particle when solving the VVPE. We use Adaptive Kernel Density Estimation (AKDE) to extract a probability density distribution from the KVN for the the Monte Carlo calculations. Even though the distribution of the KVN can be non-trivial, AKDE yields a smooth and normalized result which we dynamically update at each time step. As we sample actual particles directly, the Lagrangian attributes of particle samples ensure that the continuously evolved KVN-based importance, modeled by the probability density distribution extracted from the KVN by AKDE, can be closely followed. Our approach enables effective vortical flow simulations with significantly reduced computational overhead and comparable quality to the classic Biot-Savart law that in contrast requires expensive global particle querying.

Automatic summarization of medical dialogues, which converts colloquial doctor-patient conversations into concise notes, is increasingly important due to the growing complexity of healthcare data. However, the complexity of medical language and the lack of annotated datasets pose challenges for summarization models. In this paper, we propose a MediExtract Distillation Framework (MEDF), a novel hybrid teacher-student distillation process that leverages the power of Large Language Models (LLMs) in information capturing to enhance the performance of a smaller student model. Utilizing medical key information generated by GPT-3.5-Turbo, the model training involves two feedforward branches per iteration: one using ground truth as labels and another using generated structured medical key information as an auxiliary supervision. We validated our method on the MTS-Dialogue dataset, achieving a +2.1% improvement in BLEURT compared to previous methods, demonstrating its effectiveness in summarizing medical dialogues. Additionally, using UMLS-based BERTScore, we observed a +1.8% increase in MedBERTScore for medical term extraction, highlighting our model’s practical benefits in clinical information processing. Our framework is publicly available at: https://github.com/Xiaoxiao-Liu/distill-d2n.git

The estimation of 3D motion fields, known as scene flow estimation, is an essential task in autonomous driving and robotic navigation. Existing learning-based methods either predict scene flow through flow-embedding layers or rely on local search methods to establish soft correspondences. However, these methods often neglect distant points which, in fact, represent the true matching elements. To address this challenge, we introduce GAMAFlow, a point-voxel architecture that models local motion and global motion to predict scene flow iteratively. In particular, GAMAFlow integrates the advantages of (i) the point Transformer with Grouped Attention and (ii) global Motion Aggregation to boost the efficacy of point-voxel correlation. Such an approach facilitates learning long-distance dependencies between current frame and next frame. Experiments illustrate the performance gains achieved by GAMAFlow compared to existing works on both FlyingThings3D and KITTI benchmarks.

In this paper, we present PCoTTA, an innovative, pioneering framework for Continual Test-Time Adaptation (CoTTA) in multi-task point cloud understanding, enhancing the model's transferability towards the continually changing target domain. We introduce a multi-task setting for PCoTTA, which is practical and realistic, handling multiple tasks within one unified model during the continual adaptation. Our PCoTTA involves three key components: automatic prototype mixture (APM), Gaussian Splatted feature shifting (GSFS), and contrastive prototype repulsion (CPR). Firstly, APM is designed to automatically mix the source prototypes with the learnable prototypes with a similarity balancing factor, avoiding catastrophic forgetting. Then, GSFS dynamically shifts the testing sample toward the source domain, mitigating error accumulation in an online manner. In addition, CPR is proposed to pull the nearest learnable prototype close to the testing feature and push it away from other prototypes, making each prototype distinguishable during the adaptation. Experimental comparisons lead to a new benchmark, demonstrating PCoTTA's superiority in boosting the model's transferability towards the continually changing target domain. Our source code is available at: https://github.com/Jinec98/PCoTTA.

Abstract For vortex particle methods relying on SPH‐based simulations, the direct approach of iterating all fluid particles to capture velocity from vorticity can lead to a significant computational overhead during the Biot‐Savart summation process. To address this challenge, we present a Monte Carlo vortical smoothed particle hydrodynamics (MCVSPH) method for efficiently simulating turbulent flows within an SPH framework. Our approach harnesses a Monte Carlo estimator and operates exclusively within a pre‐sampled particle subset, thus eliminating the need for costly global iterations over all fluid particles. Our algorithm is decoupled from various projection loops which enforce incompressibility, independently handles the recovery of turbulent details, and seamlessly integrates with state‐of‐the‐art SPH‐based incompressibility solvers. Our approach rectifies the velocity of all fluid particles based on vorticity loss to respect the evolution of vorticity, effectively enforcing vortex motions. We demonstrate, by several experiments, that our MCVSPH method effectively preserves vorticity and creates visually prominent vortical motions.

Physics-based fluid simulation has played an increasingly important role in the computer graphics community. Recent methods in this area have greatly improved the generation of complex visual effects and its computational efficiency. Novel techniques have emerged to deal with complex boundaries, multiphase fluids, gas–liquid interfaces, and fine details. The parallel use of machine learning, image processing, and fluid control technologies has brought many interesting and novel research perspectives. In this survey, we provide an introduction to theoretical concepts underpinning physics-based fluid simulation and their practical implementation, with the aim for it to serve as a guide for both newcomers and seasoned researchers to explore the field of physics-based fluid simulation, with a focus on developments in the last decade. Driven by the distribution of recent publications in the field, we structure our survey to cover physical background; discretization approaches; computational methods that address scalability; fluid interactions with other materials and interfaces; and methods for expressive aspects of surface detail and control. From a practical perspective, we give an overview of existing implementations available for the above methods.

Abstract 3D reconstruction is a long‐standing problem. Recently, a number of studies have emerged that utilize transformers for 3D reconstruction, and these approaches have demonstrated strong performance. However, transformer‐based 3D reconstruction methods tend to establish the transformation relationship between the 2D image and the 3D voxel space directly using transformers or rely solely on the powerful feature extraction capabilities of transformers. They ignore the crucial role played by deep multi‐scale representation of the object in the voxel feature domain, which can provide extensive global shape and local detail information about the object in a multi‐scale manner. In this article, we propose a novel framework TMSDNet (transformer with multi‐scale dense network) for single‐view and multi‐view 3D reconstruction with transformer to solve this problem. Based on our well‐designed combined‐transformer Block, which is canonical encoder–decoder architecture, voxel features with spatial order can be extracted from the input image, which are used to further extract multi‐scale global features in parallel using a multi‐scale residual attention module. Furthermore, a residual dense attention block is introduced for deep local features extraction and adaptive fusion. Finally, the reconstructed objects are produced with the voxel reconstruction block. Experiment results on the benchmarks such as ShapeNet and Pix3D datasets demonstrate that TMSDNet outperforms the existing state‐of‐the‐art reconstruction methods substantially.

Recent advances in NeRF-based 3D-aware GANs have achieved outstanding performance, especially in the realm of human facial representations, making projection of facial images back into their latent space superior and preferable compared to 2D GAN inversion. However, the direct application of 2DGAN inversion techniques to 3DGAN raises challenges due to potential appearance distortions and geometric inconsistences. To tackle these issues, this work presents a novel integrated framework that combines a composite inversion pipeline in both the SS and W+ spaces and integrates a contrastive-based training strategy, ensuring proficient disentanglement within the module. Moreover, we design a facial semantic manipulation technique based on dimensional analysis of the latent code, which is fully compatible with the proposed 3DGAN inversion pipeline. Comprehensive experimental validations substantiate the effectiveness of the proposed approach in executing 3d-aware face inversion and semantic editing tasks, presenting a robust technological solution for a diverse array of digital human modeling applications in the downstream.

Applying Artificial Intelligence (AI) techniques such as natural language generation in assisting medical treatment and diagnosis has made distinguished progress. One such technique is dialogue generation. The application of a medical dialogue system in assisting medical treatment has great potential to explore. This paper serves as a survey of digging application of AI techniques in knowledge-grounded dialogue generation for medical conversation systems. Meanwhile, we provide an academic visualization method to present such references.

Parametric curve fitting which includes curve approximation and interpolation is a critical topic in many fields, such as medical images, 3D reconstruction and data visualization, etc. To approximate or interpolate point sets with a curve, parameterization of points is usually a key step, which has a great impact on the final result. Many methods that have been proposed to tackle this problem. To our best knowledge, there is not any review paper about point set parameterization for curve fitting. In this paper, we will review the proposed techniques of parameterizing point sets for curve fitting. These methods will be divided into two categories. In the first category, the parameterization of point sets and curve fitting are treated as two separable and successive steps, specifically, the parameters obtained using some techniques in the first step are used for the curve fitting process in the second step. The two steps are solved together in the second category using more complex techniques. Methods in each of the two types will be reviewed in this paper.

Particle-based simulations have become increasingly popular in real-time applications due to their efficiency and adaptability, especially for generating highly dynamic fluid effects. However, the swift and stable simulation of interactions among distinct fluids continues to pose challenges for current mixture model techniques. When using a single-mixture flow field to represent all fluid phases, numerical discontinuities in phase fields can result in significant losses of dynamic effects and unstable conservation of mass and momentum. To tackle these issues, we present an advanced implicit mixture model for smoothed particle hydrodynamics. Instead of relying on an explicit mixture field for all dynamic computations and phase transfers between particles, our approach calculates phase momentum sources from the mixture model to derive explicit and continuous velocity phase fields. We then implicitly obtain the mixture field using a phase-mixture momentum-mapping mechanism that ensures conservation of incompressibility, mass, and momentum. In addition, we propose a mixture viscosity model and establish viscous effects between the mixture and individual fluid phases to avoid instability under extreme inertia conditions. Through a series of experiments, we show that, compared to existing mixture models, our method effectively improves dynamic effects while reducing critical instability factors. This makes our approach especially well-suited for long-duration, efficiency-oriented virtual reality scenarios.

The use of social media has become so popular that people share photos every day on them. Automatic face recognition and tagging of people's photos have caused privacy preservation issues and some methods have been proposed for hiding the identity of presented people in these images. Blurring and blacking the face area, adding physical adversarial patches to the face, and adding adversarial masks are some proposed methods for this purpose. However, these methods particularly suffer from dissimilarity of the input and output images and inadequate performance in identity concealment from automatic face recognition (AFR) systems. In this paper, we propose the Generative Mask-guided Face Image Manipulation (GMFIM) model based on Generative Adversarial Networks (GANs) to apply imperceptible edits to the input face image to preserve the identity of the person in the image. Our model consists of a face mask module, a GAN-based optimization module, and a merge module. Different criteria are considered in the objective function of the optimization step to produce high-quality images that are as similar as possible to the input image while they cannot be recognized by AFR systems. The results of the experiments on different datasets show that our model provides promising results in terms of the quality of the generated images and the identity concealment performance.

This paper proposes a real-time fluid rendering method based on the screen space rendering scheme for particle-based fluid simulation. Our method applies anisotropic transformations to the point sprites to stretch the point sprites along appropriate axes, obtaining smooth fluid surfaces based on the weighted principal components analysis of the particle distribution. Then we combine the processed anisotropic point sprite information with popular screen space filters like curvature flow and narrow-range filters to process the depth information. Experiments show that the proposed method can efficiently resolve the issues of jagged edges and unevenness on the surface that existed in previous methods while preserving sharp high-frequency details.

Abstract Aiming at obtaining structural information and 3D motion of dynamic scenes, scene flow estimation has been an interest of research in computer vision and computer graphics for a long time. It is also a fundamental task for various applications such as autonomous driving. Compared to previous methods that utilize image representations, many recent researches build upon the power of deep analysis and focus on point clouds representation to conduct 3D flow estimation. This paper comprehensively reviews the pioneering literature in scene flow estimation based on point clouds. Meanwhile, it delves into detail in learning paradigms and presents insightful comparisons between the state‐of‐the‐art methods using deep learning for scene flow estimation. Furthermore, this paper investigates various higher‐level scene understanding tasks, including object tracking, motion segmentation, etc. and concludes with an overview of foreseeable research trends for scene flow estimation.

Abstract The emergence of position‐based simulation approaches has quickly developed a group of new topics in the computer graphics community. These approaches are popular due to their advantages, including computational efficiency, controllability, stability and robustness for different scenarios, whilst they also have some weaknesses. In this survey, we will introduce the concept of the baseline position based dynamics (PBD) method and review the improvements and applications of PBD since 2018, including extensions for different materials and integrations with other techniques.

Abstract In this article, we propose a point cloud synthesis method based on stochastic differential equations. We view the point cloud generation process as smoothly transforming from a known prior distribution toward the high‐likelihood shape by point‐level denoising. We introduce a conditional corrector sampler to improve the quality of point clouds. By leveraging Markov chain Monte Carlo sample, our method can synthesize realistic point clouds. We additionally prove that our approach can be trained in an auto‐encoding fashion and reconstruct the point cloud faithfully. Furthermore, our model can be extended on a downstream application of point cloud completion. Experimental results demonstrate the effectiveness and efficiency of our method.

Abstract Fluid simulation has been one of the most critical topics in computer graphics for its capacity to produce visually realistic effects. The intricacy of fluid simulation manifests most with interacting dynamic elements. The coupling for such scenarios has always been challenging to manage due to the numerical instability arising from the coupling boundary between different elements. Therefore, we propose an implicit smoothed particle hydrodynamics fluid‐elastic coupling approach to reduce the instability issue for fluid‐fluid, fluid‐elastic, and elastic‐elastic coupling circumstances. By deriving the relationship between the universal pressure field with the incompressible attribute of the fluid, we apply the number density scheme to solve the pressure Poisson equation for both fluid and elastic material to avoid the density error for multi‐material coupling and conserve the non‐penetration condition for elastic objects interacting with fluid particles. Experiments show that our method can effectively handle the multiphase fluids simulation with elastic objects under various physical properties.

Abstract In recent years, it becomes possible to extract hair information for hair reconstruction from multiple cameras or monocular camera. Using a single image as the input avoids the high cost setups and complex calibration compared to multiviewed reconstruction. Taking advantage of an extendible hairstyle database, this paper introduced Struct2Hair, a novel single‐viewed hair modelling approach by extracting hair shape descriptor (HSD). The HSD is defined as the fundamental structure‐aware feature, which is a combination of critical shapes in a hairstyle. A complete dataset of critical hair shapes is constructed from a known database of three‐dimensional (3D) hair models. We first analyze the input two‐dimensional (2D) image to extract the orientation information and 2D hair sketch automatically. The extracted information is then used to retrieve the corresponding critical shapes with optimization to build the robust HSD. Finally, the HSD constructs a weighted 3D hair orientation field to guide full‐head hair model generation. Our method can preserve local geometric features of hair and retain the whole shape of the hairstyle globally owing to the HSD, which will benefit further hair editing and stylization.

Abstract Binary image boundary vectorization is the process of converting raster images into vector images represented with a sequence of Bézier curves. Two main factors in reconstructing parametric curves are to approximate the underlying structure of the boundaries as much as possible while using as few curves as possible. Existing methods do not perform well when considering both of these two main factors. In this article, we mimic the process of human vectorizing image boundaries by first segmenting the boundary points into multiple segments with the corner points. For the boundary points in each segment, we adopt the bisection method to find the largest number of points, which a single curve can fit. More curves will be added if the fitting error is larger than a predefined threshold. The process is repeated until all the points in the segment are fitted, thus minimizing the number of Bézier curves. Besides, symmetric image boundaries can be detected and used to further decrease the number of curves required. Our method can also choose the optimal parameterization method case by case to further reduce the fitting error. We make a comparison with both new and classical methods and show that our method outperforms them.

Abstract Surface reconstruction is an important part in automotive styling design. Existing reconstruction methods mainly rely on the proficiency of digital modelers who manually modify the surface shape to approximate the scanned data. Apart from manual modifications, the reconstructed surfaces cannot well reflect the design intent of designers since the feature curves of clay models have not been preserved accurately. In this paper, we propose a partial differential equation (PDE) based surface reconstruction method to analytically generate optimal surfaces with Cn continuity under the constraint of the feature curves. The proposed method accurately preserves automotive feature curves and achieves automatic reconstruction of Class-A surfaces without time-consuming manual work. The effectiveness of the proposed method is demonstrated by a number of experiments that reconstruct main parts of automotive exteriors.

In this article, we present two algorithms that discover the discriminative structures of sketches, given pairs of sketches and photos in sketch-based image retrieval (SBIR) scenarios. Unlike the existing approaches, we aim at the few-shot and domain adaptation (DA) problems, and set up a module with canonical correlation analysis (CCA) technique in our algorithms to improve retrieval performance. For single source domain settings, our first algorithm can effectively transfer a classifier trained on a known dataset to a new one. For multisource settings, our second algorithm sophisticatedly combines multisource domain data to yield a classifier on the target domain. To the best of our knowledge, these two works are the first research in SBIR field. Experiments on the Sketchy and TU-Berlin sketch benchmark datasets demonstrate the effectiveness of our algorithms and compelling performance. Compared with the state-of-the-art methods, our algorithms do not use text based semantic information, but achieve competitive results. Furthermore, experiments also turn out that feature representation by available trained deep networks has distinct advantages and combination with traditional machine learning methods brings substantial improvements against the state-of-the-art methods on image retrievals. The complete source code of the proposed algorithms will be released on gitHub: [ GitHub- ADAM0912/Domain-Adaptation-Problem-in-Sketch-Based-Image-Retrieval: The code repository for the articleDomain Adaptation Problem in Sketch Based Image Retrieval ] .

Abstract Point cloud generation aims to synthesize point clouds that do not exist in supervised dataset. Generating a point cloud with certain semantic labels remains an under‐explored problem. This paper proposes a formulation called DiffusionPointLabel, which completes point‐label pair generation based on a DDPM generative model (Denoising Diffusion Probabilistic Model). Specifically, we use a point cloud diffusion generative model and aggregate the intermediate features of the generator. On top of this, we propose Feature Interpreter that transforms intermediate features into semantic labels. Furthermore, we employ an uncertainty measure to filter unqualified point‐label pairs for a better quality of generated point cloud dataset. Coupling these two designs enables us to automatically generate annotated point clouds, especially when supervised point‐labels pairs are scarce. Our method extends the application of point cloud generation models and surpasses state‐of‐the‐art models.

Online games require small data of 3D models for low storage costs, quick transmission over the Internet, and efficient geometric processing to achieve real-time performance, and new techniques of facial blendshapes to create natural facial animation. Current geometric modelling and animation techniques involve big data of geometric models and widely applied facial animation using linear interpolation cannot generate natural facial animation and create special facial animation effects. In this paper, we propose a new approach to integrate the strengths of ODE (ordinary differential equation) sweeping surfaces and Newton’s second law-based facial blendshapes to create 3D models and their animation with small data, high efficiency, and ability to create special facial effects.

We present a vision-based system for real-time pose tracking of the rigid object, it can not only estimate a single pose in six degrees of freedom (6DoF), but also suitable for recovering compound movements. The system is comprised of a monocular camera, and a series of 3D printed platonic solids with squared fiducial markers attached on each single face, which is easy to setup and extend, extra cameras are allowed to incorporate into the pipeline for meeting different requirements. The system realizes object tracking by estimating the pose of the fiducial platonic solid (FPS) which can be fixed onto the surface of the target object. Different sizes and shapes of the platonic solids are allowed to combine with each other to adapt to different application scenarios, this strategy provides enormous flexibility and applicability to our system. In order to track the motion of the fiducial platonic solid accurately, a robust algorithm that combines the fiducial constraint and the statistical constraint is introduced, which is able to handle illumination changes, motion blur and partial occlusion. We evaluate the performance of the proposed approach with qualitative and quantitative experiments, in addition, a couple of mixed reality (MR) applications are developed for demonstrating the effectiveness of the system.

Brushstrokes are viewed as the artist’s “handwriting” in a painting. In many applications such as style learning and transfer, mimicking painting, and painting authentication, it is highly desired to quantitatively and accurately identify brushstroke characteristics from old masters’ pieces using computer programs. However, due to the nature of hundreds or thousands of intermingling brushstrokes in the painting, it still remains challenging. This article proposes an efficient algorithm for brush Stroke extraction based on a Deep neural network, i.e., DStroke. Compared to the state-of-the-art research, the main merit of the proposed DStroke is to automatically and rapidly extract brushstrokes from a painting without manual annotation, while accurately approximating the real brushstrokes with high reliability. Herein, recovering the faithful soft transitions between brushstrokes is often ignored by the other methods. In fact, the details of brushstrokes in a master piece of painting (e.g., shapes, colors, texture, overlaps) are highly desired by artists since they hold promise to enhance and extend the artists’ powers, just like microscopes extend biologists’ powers. To demonstrate the high efficiency of the proposed DStroke, we perform it on a set of real scans of paintings and a set of synthetic paintings, respectively. Experiments show that the proposed DStroke is noticeably faster and more accurate at identifying and extracting brushstrokes, outperforming the other methods.

Abstract We present a novel but simple physics-based method to interactively manipulate surface shapes of 3D models with $$ C^1 $$ C 1 continuity in real time. A fourth-order partial differential equation involving a sculpting force originating from elastic bending of thin plates is proposed to define physics-based deformations and achieve $$ C^1 $$ C 1 continuity at the boundary of deformation regions. In order to obtain real-time physics-based surface manipulation, we construct a mapping relationship between a deformation region in a 3D coordinate space and a unit circle on a 2D parametric plane, formulate corresponding $$ C^1 $$ C 1 continuous boundary conditions for the unit circle, and obtain a simple analytical solution to describe the physics-based deformation in the unit circle caused by a sculpting force. After that, the obtained physics-based deformation is mapped back to the 3D coordinate space, and added to the original surface to create a new surface shape with $$ C^1 $$ C 1 continuity at the boundary of the deformation region. We also develop an interactive user interface as a plug-in of the 3D modelling software package Maya to achieve real-time surface manipulation. The effectiveness, easiness, real-time performance, and better realism of our proposed method is demonstrated by testing surface deformations on several 3D models and comparing with other methods and ground-truth deformations.

Aggregate elements are ubiquitous in natural and man-made objects. Interactively authoring these elements with varying anisotropy and deformability can require high artistic skills and manual labor. To reduce input workload and enhance output quality, we present an autocomplete system that can help users distribute and align such elements over different domains. Through a brushing interface, users can place and mix a few elements, and let our system automatically populate more elements for the remaining output. Furthermore, aggregate elements often require proper direction/scalar fields for proper arrangements, but fully specifying such fields across entire domains can be difficult or inconvenient for ordinary users. To address this usability challenge, we formulate element fields that can smoothly orient all the elements based on partial user specifications without requiring full input fields in any step. We validate our prototype system with a pilot user study and show applications in design, collage, and modeling.

Semantic reconstruction of indoor scenes refers to both scene understanding and object reconstruction. Existing works either address one part of this problem or focus on independent objects. In this paper, we bridge the gap between understanding and reconstruction, and propose an end-to-end solution to jointly reconstruct room layout, object bounding boxes and meshes from a single image. Instead of separately resolving scene understanding and object reconstruction, our method builds upon a holistic scene context and proposes a coarse-to-fine hierarchy with three components: 1. room layout with camera pose; 2. 3D object bounding boxes; 3. object meshes. We argue that understanding the context of each component can assist the task of parsing the others, which enables joint understanding and reconstruction. The experiments on the SUN RGB-D and Pix3D datasets demonstrate that our method consistently outperforms existing methods in indoor layout estimation, 3D object detection and mesh reconstruction.

Point completion refers to complete the missing geometries of objects from partial point clouds. Existing works usually estimate the missing shape by decoding a latent feature encoded from the input points. However, real-world objects are usually with diverse topologies and surface details, which a latent feature may fail to represent to recover a clean and complete surface. To this end, we propose a skeleton-bridged point completion network (SK-PCN) for shape completion. Given a partial scan, our method first predicts its 3D skeleton to obtain the global structure, and completes the surface by learning displacements from skeletal points. We decouple the shape completion into structure estimation and surface reconstruction, which eases the learning difficulty and benefits our method to obtain on-surface details. Besides, considering the missing features during encoding input points, SK-PCN adopts a local adjustment strategy that merges the input point cloud to our predictions for surface refinement. Comparing with previous methods, our skeleton-bridged manner better supports point normal estimation to obtain the full surface mesh beyond point clouds. The qualitative and quantitative experiments on both point cloud and mesh completion show that our approach outperforms the existing methods on various object categories.

Abstract Mixed reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic‐based interactive MR framework that is beyond current geometry‐based approaches, offering a step change in generating high‐level context‐aware interactions. Our key insight is that by building semantic understanding in MR, we can develop a system that not only greatly enhances user experience through object‐specific behaviours, but also it paves the way for solving complex interaction design challenges. In this paper, our proposed framework generates semantic properties of the real‐world environment through a dense scene reconstruction and deep image understanding scheme. We demonstrate our approach by developing a material‐aware prototype system for context‐aware physical interactions between the real and virtual objects. Quantitative and qualitative evaluation results show that the framework delivers accurate and consistent semantic information in an interactive MR environment, providing effective real‐time semantic‐level interactions.

Abstract Superpixel segmentation is important for promoting various image processing tasks. However, existing methods still have difficulties in generating high‐quality superpixels in textured images, because they cannot separate textures from structures well. Though texture filtering can be adopted for smoothing textures before superpixel segmentation, the filtering would also smooth the object boundaries, and thus weaken the quality of generated superpixels. In this paper, we propose to use the adaptive scale box smoothing instead of the texture filtering to obtain more high‐quality texture and boundary information. Based on this, we design a novel distance metric to measure the distance between different pixels, which considers boundary, color and Euclidean distance simultaneously. As a result, our method can achieve high‐quality superpixel segmentation in textured images without texture filtering. The experimental results demonstrate the superiority of our method over existing methods, even the learning‐based methods. Benefited from using boundaries to guide superpixel segmentation, our method can also suppress noise to generate high‐quality superpixels in non‐textured images.

Abstract Human motion prediction is a fundamental problem in understanding human natural movements. This task is very challenging due to the complex human body constraints and diversity of action types. Due to the human body being a natural graph, graph convolutional network (GCN)‐based models perform better than the traditional recurrent neural network (RNN)‐based models on modeling the natural spatial and temporal dependencies lying in the motion data. In this paper, we develop the GCN‐based models further by adding densely connected links to increase their feature utilizations and address oversmoothing problem. More specifically, the GCN block is used to learn the spatial relationships between the nodes and each feature map of the GCN block propagates directly to every following block as input rather than residual linked. In this way, the spatial dependency of human motion data is exploited more sufficiently and the features of different level of scale are fused more efficiently. Extensive experiments demonstrate our model achieving the state‐of‐the‐art results on CMU dataset.

Abstract Sketch‐based modeling aims to recover three‐dimensional (3D) shape from two‐dimensional line drawings. However, due to the sparsity and ambiguity of the sketch, it is extremely challenging for computers to interpret line drawings of physical objects. Most conventional systems are restricted to specific scenarios such as recovering for specific shapes, which are not conducive to generalize. Recent progress of deep learning methods have sparked new ideas for solving computer vision and pattern recognition issues. In this work, we present an end‐to‐end learning framework to predict 3D shape from line drawings. Our approach is based on a two‐steps strategy, it converts the sketch image to its normal image, then recover the 3D shape subsequently. A differentiable renderer is proposed and incorporated into this framework, it allows the integration of the rendering pipeline with neural networks. Experimental results show our method outperforms the state‐of‐art, which demonstrates that our framework is able to cope with the challenges in single sketch‐based 3D shape modeling.

This special issue contains 28 full papers selected from the Computer Animation and Social Agents 2020 Conference (CASA2020). This conference was founded by the Computer Graphics Society in 1988 in Geneva and is the oldest conference on Computer Animation in the world. It has been held in many countries around the world and in recent years in Beijing, China (2018), Paris, France (2019) and this year in Bournemouth, United Kingdom. Because of the Covid-19 pandemic, this year, the conference will be held online through the Youtube Channel. The best paper award will be announced on the conference website after the conference. We would like to thank the authors for sharing their research findings by submitting papers to CASA2020. We are very grateful to the Program Committee members for reviewing the papers and to all the people who have contributed to the success of CASA2020 in Bournemouth. The conference is organized by Bournemouth University under the guidance of the Computer Graphics Society (CGS). Conference co-chairs Jian Jun Zhang (Bournemouth University, UK) Nadia Magnenat Thalmann (University of Geneva, Switzerland and Nanyang Technological University, Singapore) Program co-chairs Daniel Thalmann (EPFL, Switzerland) Xiaosong Yang (Bournemouth University, UK) Weiwei Xu (Zhejiang University, China) Publicity chair Jian Chang (Bournemouth University, UK) Local chair Feng Tian (Bournemouth University, UK) International program committee Nadine Aburumman, Brunel University, UK Norman Badler, University of Pennsylvania, USA Selim Balcisoy, Sabanci University, Turkey Loic Barthe, IRIT—Université de Toulouse, France Jan Bender, RWTH Aachen University, Germany Raphaëlle Chaine, LIRIS Université Lyon 1, France Jian Chang, Bournemouth University, UK Fred Charles, Bournemouth University, UK Parag Chaudhuri, Indian Institute of Technology, Bombay, India Marc Christie, INRIA, France Justin Dauwels, Nanyang Technological University, Singapore Shujie Deng, King's College London, UK Zhigang Deng, University of Houston, USA Etienne de Sevin, SANPSY University of Bordeaux, France Petros Faloutsos, York University, Canada Christos Gatzidis, Bournemouth University, UK Ugur Gudukbay, Bilkent University, Turkey Shihui Guo, Xiamen University, China Xiaohu Guo, The University of Texas at Dallas, USA James Hahn, George Washington University, USA Carlo Harvey, Birmingham City University, UK Gaoqi He, East China Normal University, China Ying He, Nanyang Technological University, Singapore Kemao Qian, Nanyang Technological University, Singapore Ruizhen Hu, Shenzhen University, China Jinyuan Jia, Tongji University, China Tao Jiang, University of Surrey, UK Xiaogang Jin, Zhejiang University, China Marcelo Kallmann, University of California, Merced, USA Prem Kalra, IIT Delhi, India Dongwann Kang, Seoul National University of Science and Technology, Korea Mubbasir Kapadia, Rutgers University, USA Min H. Kim, Korea Advanced Institute of Science and Technology, Korea Scott King, Texas A&M University—Corpus Christi, USA Taesoo Kwon, Hanyang University, China Sung-Hee Lee, Korea Advanced Institute of Science and Technology, Korea Wonsook Lee, University of Ottawa, Canada Tsai-Yen Li, National Chengchi University, Taiwan Guoliang Luo, East China Jiaotong University, China Chongyang Ma, Snap Inc., USA Anderson Maciel, Universidade Federal do Rio Grande do Sul, Brazil Nadia Magnenat Thalmann, University Of Geneva, Switzerland Shigeo Morishima, Waseda University, Japan Soraia Musse, Pontificia Universidade Catolica do Roi Grande do Sul, PUCRS, Brazil Rahul Narain, Indian Institute of Technology, Delhi, India Junjun Pan, Beihang University, China Nuria Pelechano, Universitat Politècnica de Catalunya, Spain Julien Pettre, INRIA, France Nicolas Pronost, Université Claude Bernard Lyon 1, France Kun Qian, King's College London, UK Craig Schroeder, University of California, Riverside, USA Ari Shapiro, Embody Digital, USA Hubert P. H. Shum, Northumbria University, UK Shinjiro Sueda, Texas A&M University, USA Daniel Thalmann, Ecole Polytechnique Fédérale de Lausanne, Switzerland Feng Tian, Bournemouth University, UK Yiying Tong, Michigan State University, USA Meili Wang, Northwest A&F University, China Zhao Wang, Zhejiang University, China Enhua Wu, University of Macau & ISCAS, China Zhongke Wu, Beijing Normal University, China Weiwei Xu, Zhejiang University, China Yachun Fan, Beijing Normal University, China Bailin Yang, Zhejiang Gongshang University, China Yin Yang, University of New Mexico, USA Xiaosong Yang, Bournemouth University, UK Yuting Ye, Oculus Research, USA Lihua You, Bournemouth University, UK Hongchuan Yu, Bournemouth University, UK Zerrin Yumak, Utrecht University, Netherlands Wenshu Zhang, Cardiff Metropolitan University Jian Zhang, Bournemouth University, UK Jianmin Zheng, Nanyang Technological University, Singapore

Abstract Virtual characters are 3D geometric models of characters. They have a lot of applications in multimedia. In this paper, we propose a new physics-based deformation method and efficient character modelling framework for creation of detailed 3D virtual character models. Our proposed physics-based deformation method uses PDE surfaces. Here PDE is the abbreviation of Partial Differential Equation, and PDE surfaces are defined as sculpting force-driven shape representations of interpolation surfaces. Interpolation surfaces are obtained by interpolating key cross-section profile curves and the sculpting force-driven shape representation uses an analytical solution to a vector-valued partial differential equation involving sculpting forces to quickly obtain deformed shapes. Our proposed character modelling framework consists of global modeling and local modeling. The global modeling is also called model building, which is a process of creating a whole character model quickly with sketch-guided and template-based modeling techniques. The local modeling produces local details efficiently to improve the realism of the created character model with four shape manipulation techniques. The sketch-guided global modeling generates a character model from three different levels of sketched profile curves called primary, secondary and key cross-section curves in three orthographic views. The template-based global modeling obtains a new character model by deforming a template model to match the three different levels of profile curves. Four shape manipulation techniques for local modeling are investigated and integrated into the new modelling framework. They include: partial differential equation-based shape manipulation, generalized elliptic curve-driven shape manipulation, sketch assisted shape manipulation, and template-based shape manipulation. These new local modeling techniques have both global and local shape control functions and are efficient in local shape manipulation. The final character models are represented with a collection of surfaces, which are modeled with two types of geometric entities: generalized elliptic curves (GECs) and partial differential equation-based surfaces. Our experiments indicate that the proposed modeling approach can build detailed and realistic character models easily and quickly.

Surgical smoke removal algorithms can improve the quality of intra-operative imaging and reduce hazards in image-guided surgery, a highly desirable post-process for many clinical applications. These algorithms also enable effective computer vision tasks for future robotic surgery. In this article, we present a new unsupervised learning framework for high-quality pixel-wise smoke detection and removal. One of the well recognized grand challenges in using convolutional neural networks (CNNs) for medical image processing is to obtain intra-operative medical imaging datasets for network training and validation, but availability and quality of these datasets are scarce. Our novel training framework does not require ground-truth image pairs. Instead, it learns purely from computer-generated simulation images. This approach opens up new avenues and bridges a substantial gap between conventional non-learning based methods and which requiring prior knowledge gained from extensive training datasets. Inspired by the Generative Adversarial Network (GAN), we have developed a novel generative-collaborative learning scheme that decomposes the de-smoke process into two separate tasks: smoke detection and smoke removal. The detection network is used as prior knowledge, and also as a loss function to maximize its support for training of the smoke removal network. Quantitative and qualitative studies show that the proposed training framework outperforms the state-of-the-art de-smoking approaches including the latest GAN framework (such as PIX2PIX). Although trained on synthetic images, experimental results on clinical images have proved the effectiveness of the proposed network for detecting and removing surgical smoke on both simulated and real-world laparoscopic images.

Image style transfer has attracted much attention in recent years. However, results produced by existing works still have lots of distortions. This paper investigates the CNN-based artistic style transfer work specifically and finds out the key reasons for distortion coming from twofold: the loss of spatial structures of content image during content-preserving process and unexpected geometric matching introduced by style transformation process. To tackle this problem, this paper proposes a novel approach consisting of a dual-stream deep convolution network as the loss network and edge-preserving filters as the style fusion model. Our key contribution is the introduction of an additional similarity loss function that constrains both the detail reconstruction and style transfer procedures. The qualitative evaluation shows that our approach successfully suppresses the distortions as well as obtains faithful stylized results compared to state-of-the-art methods.

We present a unified learning framework for recovering both 3D mesh and camera pose of the object from a single image. Our approach learns to recover outer shape and surface geometric details of the mesh without relying on 3D supervision. We adopt multi-view normal maps as the 2D supervision so that the silhouette and geometric details information can be transferred to neural network. A normal mismatch based objective function is introduced to train the network, and the camera pose is parameterized into the objective, it integrates pose estimation with the mesh reconstruction in a same optimization procedure. We demonstrate the abilities of the proposed approach in generating 3D mesh and estimating camera pose with qualitative and quantitative experiments.

The collection of internet images has been growing in an astonishing speed. It is undoubted that these images contain rich visual information that can be useful in many applications, such as visual media creation and data-driven image synthesis. In this article, we focus on the methodologies for building a visual object database from a collection of internet images. Such database is built to contain a large number of high-quality visual objects that can help with various data-driven image applications. Our method is based on dense proposal generation and objectness-based re-ranking. A novel deep convolutional neural network is designed for the inference of proposal objectness , the probability of a proposal containing optimally located foreground object. In our work, the objectness is quantitatively measured in regard of completeness and fullness , reflecting two complementary features of an optimal proposal: a complete foreground and relatively small background. Our experiments indicate that object proposals re-ranked according to the output of our network generally achieve higher performance than those produced by other state-of-the-art methods. As a concrete example, a database of over 1.2 million visual objects has been built using the proposed method, and has been successfully used in various data-driven image applications.

Relief is an art form part way between 3D sculpture and 2D painting. We present a novel approach for generating a texture-mapped high-relief model from a single brush painting. Our aim is to extract the brushstrokes from a painting and generate the individual corresponding relief proxies rather than recovering the exact depth map from the painting, which is a tricky computer vision problem, requiring assumptions that are rarely satisfied. The relief proxies of brushstrokes are then combined together to form a 2.5D high-relief model. To extract brushstrokes from 2D paintings, we apply layer decomposition and stroke segmentation by imposing boundary constraints. The segmented brushstrokes preserve the style of the input painting. By inflation and a displacement map of each brushstroke, the features of brushstrokes are preserved by the resultant high-relief model of the painting. We demonstrate that our approach is able to produce convincing high-reliefs from a variety of paintings(with humans, animals, flowers, etc.). As a secondary application, we show how our brushstroke extraction algorithm could be used for image editing. As a result, our brushstroke extraction algorithm is specifically geared towards paintings with each brushstroke drawn very purposefully, such as Chinese paintings, Rosemailing paintings, etc.

Non-isometric surface registration is an important task in computer graphics and computer vision. It, however, remains challenging to deal with noise from scanned data and distortion from transformation. In this paper, we propose a Huber- $$L_1$$ -based non-isometric surface registration and solve it by the alternating direction method of multipliers. With a Huber- $$L_1$$ -regularized model constrained on the transformation variation and position difference, our method is robust to noise and produces piecewise smooth results while still preserving fine details on the target. The introduced as-similar-as-possible energy is able to handle different size of shapes with little stretching distortion. Extensive experimental results have demonstrated that our method is more accurate and robust to noise in comparison with the state-of-the-arts.

Human motion prediction is a challenging problem due to the complicated human body constraints and high-dimensional dynamics. Recent deep learning approaches adopt RNN, CNN or fully connected networks to learn the motion features which do not fully exploit the hierarchical structure of human anatomy. To address this problem, we propose a convolutional hierarchical autoencoder model for motion prediction with a novel encoder which incorporates 1D convolutional layers and hierarchical topology. The new network is more efficient compared to the existing deep learning models with respect to size and speed. We train the generic model on Human3.6M and CMU benchmark and conduct extensive experiments. The qualitative and quantitative results show that our model outperforms the state-of-the-art methods in both short-term prediction and long-term prediction.

The techniques for photographic style transfer have been researched for a long time, which explores effective ways to transfer the style features of a reference photo onto another content photograph. Recent works based on convolutional neural networks present an effective solution for style transfer, especially for paintings. The artistic style transformation results are visually appealing, however, the photorealism is lost because of content-mismatching and distortions even when both input images are photographic. To tackle this challenge, this paper introduces a similarity loss function and a refinement method into the style transfer network. The similarity loss function can solve the content-mismatching problem, however, the distortion and noise artefacts may still exist in the stylized results due to the content-style trade-off. Hence, we add a post-processing refinement step to reduce the artefacts. The robustness and effectiveness of our approach has been evaluated through extensive experiments which show that our method can obtain finer content details and less artefacts than state-of-the-art methods, and transfer style faithfully. In addition, our approach is capable of processing photographic style transfer in almost real-time, which makes it a potential solution for video style transfer.

Abstract A novel unified particle‐based method is proposed for real‐time dissolution simulation that is fast, predictable, independent of sampling resolution, and visually plausible. The dissolution model is derived from collision theory and integrated into a smoothed particle hydrodynamics fluid solver. Dissolution occurs when a solute is submerged in solvent. Physical laws govern the local excitation of solute particles based on kinetic energy: when the local excitation energy exceeds a user‐specified threshold (activation energy), the particle will be dislodged from the solid. Solute separation during dissolution is handled using a new Graphics Processing Unit (GPU)‐based region growing method. The use of smoothed particle hydrodynamics sampling for both solute and solvent guarantees a predictable and smooth dissolution process and provides user control of the volume change during the phase transition. A mathematical relationship between the activation energy and dissolution time allows for intuitive artistic control over the global dissolution rate. We demonstrate this method using a number of practical examples, including antacid pills dissolving in water, hydraulic erosion of nonhomogeneous terrains, and melting.

Abstract This paper presents a novel approach for bas‐relief generation and synthesis. In contrast to previous methods, we divide this problem into two parts: the selection of the best view and arrangement of the relief layout. Taking these into account, we incorporate the visual saliency and photographic composition rules into the bas‐relief generation. Additionally, a nonlinear compression function is used to compress the models, and finally, we implement surface parameterization by directly manipulating the mesh triangles to generate curved surface bas‐relief. We validate our approach through a variety of models. The results indicate that the proposed approach is effective to adapt different types of target surface with topology unchanged. Comparing with conventional methods, our approach is able to effectively produce bas‐relief with a reasonable layout and distinct details.

Abstract We present an automatic approach for the semantic modeling of indoor scenes based on a single photograph, instead of relying on depth sensors. Without using handcrafted features, we guide indoor scene modeling with feature maps extracted by fully convolutional networks. Three parallel fully convolutional networks are adopted to generate object instance masks, a depth map, and an edge map of the room layout. Based on these high‐level features, support relationships between indoor objects can be efficiently inferred in a data‐driven manner. Constrained by the support context, a global‐to‐local model matching strategy is followed to retrieve the whole indoor scene. We demonstrate that the proposed method can efficiently retrieve indoor objects including situations where the objects are badly occluded. This approach enables efficient semantic‐based scene editing.

The objective of motion capture (mocap) data completion is to recover missing measurement of the body markers from mocap. It becomes increasingly challenging as the missing ratio and duration of mocap data grow. Traditional approaches usually recast this problem as a low-rank matrix approximation problem based on the nuclear norm. However, the nuclear norm defined as the sum of all the singular values of a matrix is not a good approximation to the rank of mocap data. This paper proposes a novel approach to solve mocap data completion problem by adopting a new matrix norm, called truncated nuclear norm. An efficient iterative algorithm is designed to solve this problem based on the augmented Lagrange multiplier. The convergence of the proposed method is proved mathematically under mild conditions. To demonstrate the effectiveness of the proposed method, various comparative experiments are performed on synthetic data and mocap data. Compared to other methods, the proposed method is more efficient and accurate.

This paper aims to automate the process of generating a meaningful single still image from a temporal input of scene sequences. The success of our extraction relies on evaluating the optimal pose of characters selection, which should maximize the information conveyed. We define the information entropy of the still image candidates as the evaluation criteria.

Abstract Shell structures are extensively used in engineering due to their efficient load‐carrying capacity relative to material volume. However, large‐span shells require additional supporting structures to strengthen fragile regions. The problem of designing optimal stiffeners is therefore becoming a major challenge for shell applications. To address it, we propose a computational framework to design and optimize rib layout on arbitrary shell to improve the overall structural stiffness and mechanical performance. The essential of our method is to place ribs along the principal stress lines which reflect paths of material continuity and indicates trajectories of internal forces. Given a surface and user‐specified external loads, we perform a Finite Element Analysis. Using the resulting principal stress field, we generate a quad‐mesh whose edges align with this cross field. Then we extract an initial rib network from the quad‐mesh. After simplifying rib network by removing ribs with little contribution, we perform a rib flow optimization which allows ribs to swing on surface to further adjust rib distribution. Finally, we optimize rib cross‐section to maximally reduce material usage while achieving certain structural stiffness requirements. We demonstrate that our rib‐reinforced shell structures achieve good static performances. And experimental results by 3D printed objects show the effectiveness of our method.

Abstract Geodesic based Voronoi diagrams play an important role in many applications of computer graphics. Constructing such Voronoi diagrams usually resorts to exact geodesics. However, exact geodesic computation always consumes lots of time and memory, which has become the bottleneck of constructing geodesic based Voronoi diagrams. In this paper, we propose the window‐VTP algorithm, which can effectively reduce redundant computation and save memory. As a result, constructing Voronoi diagrams using the proposed window‐VTP algorithm runs 3–8 times faster than Liu et al.'s method [ LCT11 ], 1.2 times faster than its FWP‐MMP variant and more importantly uses 10–70 times less memory than both of them.

We present a novel approach for automatically detecting and tracking facial landmarks across poses and expressions from in-the-wild monocular video data, e.g., YouTube videos and smartphone recordings. Our method does not require any calibration or manual adjustment for new individual input videos or actors. Firstly, we propose a method of robust 2D facial landmark detection across poses, by combining shape-face canonical-correlation analysis with a global supervised descent method. Since 2D regression-based methods are sensitive to unstable initialization, and the temporal and spatial coherence of videos is ignored, we utilize a coarse-todense 3D facial expression reconstruction method to refine the 2D landmarks. On one side, we employ an in-the-wild method to extract the coarse reconstruction result and its corresponding texture using the detected sparse facial landmarks, followed by robust pose, expression, and identity estimation. On the other side, to obtain dense reconstruction results, we give a face tracking flow method that corrects coarse reconstruction results and tracks weakly textured areas; this is used to iteratively update the coarse face model. Finally, a dense reconstruction result is estimated after it converges. Extensive experiments on a variety of video sequences recorded by ourselves or downloaded from YouTube show the results of facial landmark detection and tracking under various lighting conditions, for various head poses and facial expressions. The overall performance and a comparison with state-of-art methods demonstrate the robustness and effectiveness of our method.

Abstract This paper simulates the behaviour of collective transport where a group of ants transports an object in a cooperative fashion. Different from humans, the task coordination of collective transport, with ants, is not achieved by direct communication between group individuals, but through indirect information transmission via mechanical movements of the object. This paper proposes a stochastic probability model to model the decision‐making procedure of group individuals and trains a neural network via reinforcement learning to represent the force policy. Our method is scalable to different numbers of individuals and is adaptable to users' input, including transport trajectory, object shape, external intervention, etc. Our method can reproduce the characteristic strategies of ants, such as realign and reposition. The simulations show that with the strategy of reposition, the ants can avoid deadlock scenarios during the task of collective transport.

Abstract With the rapid development of human–computer interaction technologies, the new media generation demands novel learning experiences with natural interaction and immersive experience. Considering that digital storytelling is a powerful pedagogical tool for young children, in this paper, we design an immersive storytelling environment that allows multiple players to use naturally interactive hand gestures to manipulate virtual puppetry for assisting narration. A set of multimodal interaction techniques is presented for a hybrid user interface that integrates existing 3D visualization and interaction devices including head‐mounted displays and depth motion sensor. In this system, the young players could intuitively use hand gestures to manipulate virtual puppets to perform a story and interact with props in a virtual stereoscopic environment. We have conducted a user experiment with four young children for pedagogical evaluation, as well as system acceptability and interactivity evaluation by postgraduate students. The results show that our framework has great potential to stimulate learning abilities of young children through collaboration tasks. The stereoscopic head‐mounted display outperformed the traditional monoscopic display in a comparison between the two.

Abstract Laparoscopic surgery is a complex minimum invasive operation that requires long learning curve for the new trainees to have adequate experience to become a qualified surgeon. With the development of virtual reality technology, virtual reality‐based surgery simulation is playing an increasingly important role in the surgery training. The simulation of laparoscopic surgery is challenging because it involves large non‐linear soft tissue deformation, frequent surgical tool interaction and complex anatomical environment. Current researches mostly focus on very specific topics (such as deformation and collision detection) rather than a consistent and efficient framework. The direct use of the existing methods cannot achieve high visual/haptic quality and a satisfactory refreshing rate at the same time, especially for complex surgery simulation. In this paper, we proposed a set of tailored key technologies for laparoscopic surgery simulation, ranging from the simulation of soft tissues with different properties, to the interactions between surgical tools and soft tissues to the rendering of complex anatomical environment. Compared with the current methods, our tailored algorithms aimed at improving the performance from accuracy, stability and efficiency perspectives. We also abstract and design a set of intuitive parameters that can provide developers with high flexibility to develop their own simulators. Copyright © 2016 John Wiley & Sons, Ltd.

Abstract Face alignment and tracking play important roles in facial performance capture. Existing data‐driven methods for monocular videos suffer from large variations of pose and expression. In this paper, we propose an efficient and robust method for this task by introducing a novel supervised coordinate descent method with 3D bilinear representation. Instead of learning the mapping between the whole parameters and image features directly with a cascaded regression framework in current methods, we learn individual sets of parameters mappings separately step by step by a coordinate descent mean. Because different parameters make different contributions to the displacement of facial landmarks, our method is more discriminative to current whole‐parameter cascaded regression methods. Benefiting from a 3D bilinear model learned from public databases, the proposed method can handle the head pose changes and extreme expressions out of plane better than other 2D‐based methods. We present the reliable result of face tracking under various head poses and facial expressions on challenging video sequences collected online. The experimental results show that our method outperforms state‐of‐art data‐driven methods.

We introduce a unified particle framework which integrates the phase-field method with multi-material simulation to allow modeling of both liquids and solids, as well as phase transitions between them. A simple elasto-plastic model is used to capture the behavior of various kinds of solids, including deformable bodies, granular materials, and cohesive soils. States of matter or phases , particularly liquids and solids, are modeled using the non-conservative Allen-Cahn equation. In contrast, materials---made of different substances---are advected by the conservative Cahn-Hilliard equation. The distributions of phases and materials are represented by a phase variable and a concentration variable, respectively, allowing us to represent commonly observed fluid-solid interactions. Our multi-phase, multi-material system is governed by a unified Helmholtz free energy density. This framework provides the first method in computer graphics capable of modeling a continuous interface between phases. It is versatile and can be readily used in many scenarios that are challenging to simulate. Examples are provided to demonstrate the capabilities and effectiveness of this approach.

Non-isometric surface registration, aiming to align two surfaces with different sizes and details, has been widely used in computer animation industry. Various existing surface registration approaches have been proposed for accurate template fitting; nevertheless, two challenges remain. One is how to avoid the mesh distortion and fold over of surfaces during transformation. The other is how to reduce the amount of landmarks that have to be specified manually. To tackle these challenges simultaneously, we propose a consistent as-similar-as-possible (CASAP) surface registration approach. With a novel defined energy, it not only achieves the consistent discretization for the surfaces to produce accurate result, but also requires a small number of landmarks with little user effort only. Besides, CASAP is constrained as-similar-as-possible so that angles of triangle meshes are preserved and local scales are allowed to change. Extensive experimental results have demonstrated the effectiveness of CASAP in comparison with the state-of-the-art approaches.

Digital techniques have been used to assist narrative and storytelling, especially in many pedagogical practices. With the rapid development of HCI techniques, saturated with digital media in their daily lives, young children, demands more interactive learning methods and meaningful immersive learning experiences. In this paper, we propose a novel hand gesture-based puppetry storytelling system which provides a more intuitive and natural human computer interaction method for young children to develop narrative ability in virtual story world. Depth motion sensing and hand gestures control technology is utilized in the implementation of user-friendly interaction. Young players could intuitively use hand gestures to manipulate virtual puppet to perform story and interact with different items in virtual environment to assist narration. Based on the result of the evaluation, this novel digital storytelling system shows positive pedagogical functions on children’s narrating ability as well as the competencies of cognitive and motor coordination. The usability of the system is preliminary examined in our test, and the results which showed that young children can benefit from playing with Puppet Narrator.

Motion capture (MOCAP) is an important technique that is widely used in many areas such as computer animation, film industry, physical training and so on. Even with professional MOCAP system, the missing marker problems always occur. Motion refinement is an essential preprocessing step for MOCAP data based applications. Although many existing approaches for motion refinement have been developed, it is still a challenging task due to the complexity and diversity of human motion. A data driven based motion refinement method is proposed in this paper, which modifies the traditional sparse coding process for special task of motion recovery from missing parts. Meanwhile, the objective function is derived by taking both statistical and kinematical property of motion data into account. Poselet model and moving window grouping are applied in the proposed method to achieve a fine-grained feature representation, which preserves the embedded spatial-temporal kinematic information. 5 motion dictionaries are learnt for each kind of poselet from training data in parallel. The motion refine problem is finally solved as an ℓ1-minimization problem. Compared with several state-of-art motion refine methods, the experimental result shows that our approach outperforms the competitors.

Eye movements have been shown to lead hand movements in tracing tasks where subjects have to move their fingers along a predefined trace. The question remained, whether the leading relationship was similar when tracing with a pointing device, such as a mouse; more importantly, whether tasks that required more or less gaze–mouse coordination would introduce variation in this pattern of behaviour, in terms of both spatial and temporal leading of gaze position to mouse movement. A three-level gaze–mouse coordination demand paradigm was developed to address these questions. A substantial dataset of 1350 trials was collected and analysed. The linear correlation of gaze–mouse movements, the statistical distribution of the lead time, as well as the lead distance between gaze and mouse cursor positions were all considered, and we proposed a new method to quantify lead time in gaze–mouse coordination. The results supported and extended previous empirical findings that gaze often led mouse movements. We found that the gaze–mouse coordination demands of the task were positively correlated to the gaze lead, both spatially and temporally. However, the mouse movements were synchronised with or led gaze in the simple straight line condition, which demanded the least gaze–mouse coordination.

Abstract Estimation of 3D body shapes from dressed‐human photos is an important but challenging problem in virtual fitting. We propose a novel automatic framework to efficiently estimate 3D body shapes under clothes. We construct a database of 3D naked and dressed body pairs, based on which we learn how to predict 3D positions of body landmarks (which further constrain a parametric human body model) automatically according to dressed‐human silhouettes. Critical vertices are selected on 3D registered human bodies as landmarks to represent body shapes, so as to avoid the time‐consuming vertices correspondences finding process for parametric body reconstruction. Our method can estimate 3D body shapes from dressed‐human silhouettes within 4 seconds, while the fastest method reported previously need 1 minute. In addition, our estimation error is within the size tolerance for clothing industry. We dress 6042 naked bodies with 3 sets of common clothes by physically based cloth simulation technique. To the best of our knowledge, We are the first to construct such a database containing 3D naked and dressed body pairs and our database may contribute to the areas of human body shapes estimation and cloth simulation.

Abstract Facial expression transfer has been actively researched in the past few years. Existing methods either suffer from depth ambiguity or require special hardware. We present a novel marker‐less, real‐time facial transfer method that requires only a single video camera. We develop a robust model, which is adaptive to user‐specific facial data. It computes expression variances in real time and rapidly transfers them onto a target character either from images or videos. Our method can be applied to videos without prior camera calibration and focal adjustment. It enables realistic online facial expression editing and performance transferring in many scenarios such as video conference, news broadcasting, lip‐syncing for song performances and so on. With low computational cost and hardware requirement, our method tracks a single user at an average of 38fps and runs smoothly even in web browsers. Copyright © 2016 John Wiley & Sons, Ltd.

Abstract With the development of virtual reality technology, surgery simulation has become an effective way to train the operation skills for surgeons. Soft tissue dissection, as one of the most frequently performed operations in surgery, is indispensable to an immersive and high‐fidelity surgery simulator. Energized dissection tools are much more commonly used than the traditional sharp scalpels for patient safety. Unfortunately, the interaction of such tools with the soft tissues has been largely ignored in the research of surgical simulators. In this paper, we have proposed an energized soft tissue dissection model. We categorize the soft tissues into three types (fascia, membrane, and fat) and simulate their physical property accordingly. The dissection algorithm we propose employs an edge‐based structure, which offers an effective mechanism for the generation of incisions dissected with energized tools. The mesh topology will not be changed when it is dissected by an energized tool, rather it is controlled by the heat transfer model. Our dissection method is highly compatible and efficient to the physically based simulation resolved by a pre‐factorized linear system. Copyright © 2016 John Wiley & Sons, Ltd.

Human motion retrieval plays an important role in many motion data based applications. In the past, many researchers tended to use a single type of visual feature as data representation. Because different visual feature describes different aspects about motion data, and they have dissimilar discriminative power with respect to one particular class of human motion, it led to poor retrieval performance. Thus, it would be beneficial to combine multiple visual features together for motion data representation. In this article, we present an Adaptive Multi-view Feature Selection (AMFS) method for human motion retrieval. Specifically, we first use a local linear regression model to automatically learn multiple view-based Laplacian graphs for preserving the local geometric structure of motion data. Then, these graphs are combined together with a non-negative view-weight vector to exploit the complementary information between different features. Finally, in order to discard the redundant and irrelevant feature components from the original high-dimensional feature representation, we formulate the objective function of AMFS as a general trace ratio optimization problem, and design an effective algorithm to solve the corresponding optimization problem. Extensive experiments on two public human motion database, i.e., HDM05 and MSR Action3D, demonstrate the effectiveness of the proposed AMFS over the state-of-art methods for motion data retrieval. The scalability with large motion dataset, and insensitivity with the algorithm parameters, make our method can be widely used in real-world applications.

Computing discrete geodesic distance over triangle meshes is one of the fundamental problems in computational geometry and computer graphics. In this problem, an effective window pruning strategy can significantly affect the actual running time. Due to its importance, we conduct an in-depth study of window pruning operations in this paper, and produce an exhaustive list of scenarios where one window can make another window partially or completely redundant. To identify a maximal number of redundant windows using such pairwise cross checking, we propose a set of procedures to synchronize local window propagation within the same triangle by simultaneously propagating a collection of windows from one triangle edge to its two opposite edges. On the basis of such synchronized window propagation, we design a new geodesic computation algorithm based on a triangle-oriented region growing scheme. Our geodesic algorithm can remove most of the redundant windows at the earliest possible stage, thus significantly reducing computational cost and memory usage at later stages. In addition, by adopting triangles instead of windows as the primitive in propagation management, our algorithm significantly cuts down the data management overhead. As a result, it runs 4--15 times faster than MMP and ICH algorithms, 2-4 times faster than FWP-MMP and FWP-CH algorithms, and also incurs the least memory usage.

We propose a semi-Lagrangian method for multiphase interface tracking. In contrast to previous methods, our method maintains an explicit polygonal mesh, which is reconstructed from an unsigned distance function and an indicator function, to track the interface of arbitrary number of phases. The surface mesh is reconstructed at each step using an efficient multiphase polygonization procedure with precomputed stencils while the distance and indicator function are updated with an accurate semi-Lagrangian path tracing from the meshes of the last step. Furthermore, we provide an adaptive data structure, multiphase distance tree, to accelerate the updating of both the distance function and the indicator function. In addition, the adaptive structure also enables us to contour the distance tree accurately with simple bisection techniques. The major advantage of our method is that it can easily handle topological changes without ambiguities and preserve both the sharp features and the volume well. We will evaluate its efficiency, accuracy and robustness in the results part with several examples.

Collision detection and resolution are of great importance to physically based animation. Real time responses are essential for many applications, which largely rely on the efficiency of localising the potentially colliding geometry and calculating the polygon intersections. It is an extremely heavy computation task using the existing polygon based methods, especially for deformable objects. To improve this issue, we present an implicit circumsphere based collision detection and resolution method for deformable objects which takes into consideration both local geometry features and the material properties. Our method approximates the mesh in question with an implicit circumsphere surface, which is used to perform finest level collision detection and resolution instead of the original polygonal mesh. The dynamic deformation as a result of collision is determined by both the geometry and the material properties of the surface. Due to the simplicity of sphere overlap test, our method is not only computationally efficient, but also stable and comparatively accurate, outperforming the existing methods in overall performance. Our implicit circumsphere method can also provide better prevention to collision tunnelling than existing methods. Besides, this method is compatible with all existing broad phase and narrow phase collision query techniques.

To better capture the shapes as well as the rich dynamics of hair, image based modelling techniques have been developed for reconstructing their 3D geometry and important visual features. Most hair images contain inevitable noises which impair reconstructed hair models. Therefore we propose to pre-process hair images and provide an orientation map of hair strands to enhance the follow-on modelling. To demonstrate the usage of pre-processing techniques, we apply our pre-processing results for bas-relief stylisation and modelling of hair from image inputs. We compare different techniques to estimate hair orientations, adopting four types of filter mechanisms. Our analysis of their performance sheds insight on designing a suitable pre-processing technique for hair reconstruction from images. Several examples of bas-relief creation validate the effectiveness of the proposed approach.

With the development of computer graphic and haptic devices, training surgeons with virtual reality technology has proven to be very effective in surgery simulation. Many successful simulators have been deployed for training medical students. However, due to the various unsolved technical issues, the laparoscopic surgery simulation has not been widely used. Such issues include modeling of complex anatomy structure, large soft tissue deformation, frequent surgical tools interactions, and the rendering of complex material under the illumination of headlight. A successful laparoscopic surgery simulator should integrate all these required components in a balanced and efficient manner to achieve both visual/haptic quality and a satisfactory refreshing rate. In this paper, we propose an efficient framework integrating a set of specially tailored and designed techniques, ranging from deformation simulation, collision detection, soft tissue dissection and rendering. We optimize all the components based on the actual requirement of laparoscopic surgery in order to achieve an improved overall performance of fidelity and responding speed.

Dehydrated core/shell fruits, such as jujubes, raisins and plums, show very complex buckles and wrinkles on their exocarp. It is a challenging task to model such complicated patterns and their evolution in a virtual environment even for professional animators. This paper presents a unified physically-based approach to simulate the morphological transformation for the core/shell fruits in the dehydration process. A finite element method (FEM), which is based on the multiplicative decomposition of the deformation gradient into an elastic part and a dehydrated part, is adopted to model the morphological evolution. In the method, the dehydration pattern can be conveniently controlled through physically prescribed parameters according to the geometry and material of the real fruits. The effects of the parameters on the final dehydrated surface patterns are investigated and summarized in detail. Experiments on jujubes, wolfberries, raisins and plums are given, which demonstrate the efficacy of the method.

We present an efficient interactive system for dressing a naked bird with feathers. In our system, a skeleton associated with guide feathers is used to describe the distribution of the body feathers. The special skeleton can be easily built by the user, given a 3D bird model as input. To address the problem of interpenetrations among feathers, the growth priority between the feather roots is defined, with which we obtain the growth order from a greedily constructed directed acyclic graph. Each feather is then adjusted in that order by a height field based collision resolution process. The height field not only provides an efficient way to detect the collision but also enables us to finely control the degree of collision during feather adjustments. The results show that our approach is capable of resolving the collisions among thousands of feathers in a few seconds. If model animation is desired, the feathers can be adjusted on the fly at interactive framerates. Details of our implementation are provided with several examples to demonstrate the effectiveness of our system.

As an important autumn feature, scenes with large numbers of falling leaves are common in movies and games. However, it is a challenge for computer graphics to simulate such scenes in an authentic and efficient manner. This paper proposes a GPU based approach for simulating the falling motion of many leaves in real time. Firstly, we use a motionsynthesis based method to analyze the falling motion of the leaves, which enables us to describe complex falling trajectories using low-dimensional features. Secondly, we transmit a primitive-motion trajectory dataset together with the low-dimensional features of the falling leaves to video memory, allowing us to execute the appropriate calculations on the GPU.

Abstract Creation of detailed character models is a very challenging task in animation production. Sketch‐based character model creation from a 3D template provides a promising solution. However, how to quickly find correct correspondences between user's drawn sketches and the 3D template model, how to efficiently deform the 3D template model to exactly match user's drawn sketches, and realize real‐time interactive modeling is still an open topic. In this paper, we propose a new approach and develop a user interface to effectively tackle this problem. Our proposed approach includes using user's drawn sketches to retrieve a most similar 3D template model from our dataset and marrying human's perception and interactions with computer's highly efficient computing to extract occluding and silhouette contours of the 3D template model and find correct correspondences quickly. We then combine skeleton‐based deformation and mesh editing to deform the 3D template model to fit user's drawn sketches and create new and detailed 3D character models. The results presented in this paper demonstrate the effectiveness and advantages of our proposed approach and usefulness of our developed user interface. Copyright © 2015 John Wiley & Sons, Ltd.

Abstract Puppetry has been a popular art form for many centuries in different cultures, which becomes a valuable and fascinating heritage assert. Traditional Chinese marionette art with over 2000 years history is one of the most representative forms offering a mixture of stage performance of singing, dancing, music, poetry, opera, story narrative and action. Apart from a set of string rules, which controls the dynamics, head carving skill is another important pillar in this art form. This paper addresses the heritage preservation of the marionette head carving by digitalizing the head models with a novel modelling technique using ordinary differential equations (ODEs). The technique has been specially tailored to suit the modelling complexity and the need of accurate description of shapes. It offers smoothly sewing ODE swept patches to represent the distinct features of a marionette head with sharp variance of local geometry. Such features otherwise are difficult to model and capture accurately, which may require a great effort and tedious handcrafting of an experienced modeller, when using other representation forms like polygons. Copyright © 2015 John Wiley & Sons, Ltd.

Human motion denoising is an indispensable step of data preprocessing for many motion data based applications. In this paper, we propose a data-driven based human motion denoising method that sparsely selects the most correlated subset of motion bases for clean motion reconstruction. Meanwhile, it takes the statistic property of two common noises, i.e., Gaussian noise and outliers, into account in deriving the objective functions. In particular, our method firstly divides each human pose into five partitions termed as poselets to gain a much fine-grained pose representation. Then, these poselets are reorganized into multiple overlapped poselet groups using a lagged window moving across the entire motion sequence to preserve the embedded spatial–temporal motion patterns. Afterward, five compacted and representative motion dictionaries are constructed in parallel by means of fast K-SVD in the training phase; they are used to remove the noise and outliers from noisy motion sequences in the testing phase by solving ℓ 1 -minimization problems. Extensive experiments show that our method outperforms its competitors. More importantly, compared with other data-driven based method, our method does not need to specifically choose the training data , it can be more easily applied to real-world applications.

Motion capture is an important technique with a wide range of applications in areas such as computer vision, computer animation, film production, and medical rehabilitation. Even with the professional motion capture systems, the acquired raw data mostly contain inevitable noises and outliers. To denoise the data, numerous methods have been developed, while this problem still remains a challenge due to the high complexity of human motion and the diversity of real-life situations. In this paper, we propose a data-driven-based robust human motion denoising approach by mining the spatial-temporal patterns and the structural sparsity embedded in motion data. We first replace the regularly used entire pose model with a much fine-grained partlet model as feature representation to exploit the abundant local body part posture and movement similarities. Then, a robust dictionary learning algorithm is proposed to learn multiple compact and representative motion dictionaries from the training data in parallel. Finally, we reformulate the human motion denoising problem as a robust structured sparse coding problem in which both the noise distribution information and the temporal smoothness property of human motion have been jointly taken into account. Compared with several state-of-the-art motion denoising methods on both the synthetic and real noisy motion data, our method consistently yields better performance than its counterparts. The outputs of our approach are much more stable than that of the others. In addition, it is much easier to setup the training dataset of our method than that of the other data-driven-based methods.

We propose a novel algorithm for blue noise sampling inspired by the Smoothed Particle Hydrodynamics (SPH) method. SPH is a well-known method in fluid simulation -- it computes particle distributions to minimize the internal pressure variance. We found that this results in sample points (i.e., particles) with a high quality blue-noise spectrum. Inspired by this, we tailor the SPH method for blue noise sampling. Our method achieves fast sampling in general dimensions for both surfaces and volumes. By varying a single parameter our method can generate a variety of blue noise samples with different distribution properties, ranging from Lloyd's relaxation to Capacity Constrained Voronoi Tessellations (CCVT). Our method is fast and supports adaptive sampling and multi-class sampling. We have also performed experimental studies of the SPH kernel and its influence on the distribution properties of samples. We demonstrate with examples that our method can generate a variety of controllable blue noise sample patterns, suitable for applications such as image stippling and re-meshing.

Multiple-fluid interaction is an interesting and common visual phenomenon we often observe. In this paper, we present an energy-based Lagrangian method that expands the capability of existing multiple-fluid methods to handle various phenomena, such as extraction, partial dissolution, etc. Based on our user-adjusted Helmholtz free energy functions, the simulated fluid evolves from high-energy states to low-energy states, allowing flexible capture of various mixing and unmixing processes. We also extend the original Cahn-Hilliard equation to be better able to simulate complex fluid-fluid interaction and rich visual phenomena such as motion-related mixing and position based pattern. Our approach is easily integrated with existing state-of-the-art smooth particle hydrodynamic (SPH) solvers and can be further implemented on top of the position based dynamics (PBD) method, improving the stability and incompressibility of the fluid during Lagrangian simulation under large time steps. Performance analysis shows that our method is at least 4 times faster than the state-of-the-art multiple-fluid method. Examples are provided to demonstrate the new capability and effectiveness of our approach.

Abstract Natural‐looking insect animation is very difficult to simulate. The fast movement and small scale of insects often challenge the standard motion capture techniques. As for the manual key‐framing or physics‐driven methods, significant amounts of time and efforts are necessary due to the delicate structure of the insect, which prevents practical applications. In this paper, we address this challenge by presenting a two‐level control framework to efficiently automate the modeling and authoring of insects’ locomotion. On the top level, we design a Triangle Placement Engine to automatically determine the location and orientation of insects’ foot contacts, given the user‐defined trajectory and settings, including speed, load, path and terrain etc. On the low‐level, we relate the Central Pattern Generator to the triangle profiles with the assistance of a Controller Look‐Up Table to fast simulate the physically‐based movement of insects. With our approach, animators can directly author insects’ behavior among a wide range of locomotion repertoire, including walking along a specified path or on an uneven terrain, dynamically adjusting to external perturbations and collectively transporting prey back to the nest.

Inspired by the development of the matrix completion theories and algorithms, a low-rank based motion capture (mocap) data refinement method has been developed, which has achieved encouraging results. However, it does not guarantee a stable outcome if we only consider the low-rank property of the motion data. To solve this problem, we propose to exploit the temporal stability of human motion and convert the mocap data refinement problem into a robust matrix completion problem, where both the low-rank structure and temporal stability properties of the mocap data as well as the noise effect are considered. An efficient optimization method derived from the augmented Lagrange multiplier algorithm is presented to solve the proposed model. Besides, a trust data detection method is also introduced to improve the degree of automation for processing the entire set of the data and boost the performance. Extensive experiments and comparisons with other methods demonstrate the effectiveness of our approaches on both predicting missing data and de-noising.

ABSTRACT Even though there is an explosive growth of motion capture data, there is still a lack of efficient and reliable methods to automatically annotate all the motions in a database. Moreover, because of the popularity of mocap devices in home entertainment systems, real‐time human motion annotation or recognition becomes more and more imperative. This paper presents a new motion annotation method that achieves both the aforementioned two targets at the same time. It uses a probabilistic pose feature based on the Gaussian Mixture Model to represent each pose. After training a clustered pose feature model, a motion clip could be represented as an action string. Then, a dynamic programming‐based string matching method is introduced to compare the differences between action strings. Finally, in order to achieve the real‐time target, we construct a hierarchical action string structure to quickly label each given action string. The experimental results demonstrate the efficacy and efficiency of our method. Copyright © 2014 John Wiley & Sons, Ltd.

We propose an energy-based method for dissolution simulation. Both the fluid (solvent) and solid (solute) are presented by particles. The dissolution priority of solute particles is based on the kinetic activation energy. The position of solute is determined using rigid body dynamics. We demonstrate plausible dissolving behaviours by using examples.

Partial differential equation-based surface modelling is a new approach of creating and manipulating three-dimensional geometric models. It uses the solution to a vector-valued partial differential equation subjected to suitably defined boundary constraints to carry out surface modeling. This paper provides a survey on this approach which summarizes various mathematical models of partial differential equation-based surface modelling, accurate and approximate analytical solutions as well as numerical solutions of the mathematical models, and the applications of partial differential equation-based surface modelling. It also discusses some future research directions of partial differential equation-based surface modelling.

This paper describes the impact of display resolution and luminance on the responses of participants in a behavioral study that used a projection-based Immersive Virtual Reality System. The scenario was a virtual bar where participants witnessed a violent attack of one person on another due to an argument about support for a soccer club. The major response variable was the number of interventions made by participants. The study was between-groups with 10 participants in two groups pre-upgrade and post-upgrade, both in the same 4-screen Cave-like system. However, the post-upgrade group experienced the scenario with projectors that had a significantly higher level of resolution and luminance than those experienced by the pre-upgrade group. The results show that, other things being equal, the number of both verbal and physical interventions was greater amongst those in the post-upgrade group compared to the pre-upgrade group.

ABSTRACT With the explosive growth of motion capture data, it becomes very imperative in animation production to have an efficient search engine to retrieve motions from large motion repository. However, because of the high dimension of data space and complexity of matching methods, most of the existing approaches cannot return the result in real time. This paper proposes a high level semantic feature in a low dimensional space to represent the essential characteristic of different motion classes. On the basis of the statistic training of Gauss Mixture Model, this feature can effectively achieve motion matching on both global clip level and local frame level. Experiment results show that our approach can retrieve similar motions with rankings from large motion database in real‐time and also can make motion annotation automatically on the fly. Copyright © 2013 John Wiley & Sons, Ltd.

Bipedal walking is not fully understood. Motion generated from methods employed in robotics literature is stiff and is not nearly as energy efficient as what we observe in nature. In this paper, we propose validity conditions for motion adaptation from biological principles in terms of the topology of the dynamic system. This allows us to provide a closed-form solution to the problem of motion adaptation to environmental perturbations. We define both global and local controllers that improve structural and state stability, respectively. Global control is achieved by coupling the dynamic system with a neural oscillator, which preserves the periodic structure of the motion primitive and ensures stability by entrainment. A group action derived from Lie group symmetry is introduced as a local control that transforms the underlying state space while preserving certain motor invariants. We verify our method by evaluating the stability and energy consumption of a synthetic passive dynamic walker and compare this with motion data of a real walker. We also demonstrate that our method can be applied to a variety of systems.

Abstract In this paper, we present an overview of the achievements accomplished to date in the field of computer‐aided relief generation. We delineate the problem, classify different solutions, analyse similarities, investigate developments and review the approaches according to their particular relative strengths and weaknesses. Moreover, we describe remaining challenges and point out prospective extensions. In consequence, this survey is addressed to both researchers and artists, through providing valuable insights into the theory behind the different concepts in this field and augmenting the options available among the methods presented with regard to practical application.

Texture mapping has long been used in computer graphics to enhance the realism of virtual scenes. However, to match the 3D model feature points with the corresponding pixels in a texture image, surface parameterization must satisfy specific positional constraints. However, despite numerous research efforts, the construction of a mathematically robust, foldover-free parameterization that is subject to positional constraints continues to be a challenge. In the present paper, this foldover problem is addressed by developing radial basis function (RBF)-based reparameterization. Given initial 2D embedding of a 3D surface, the proposed method can reparameterize 2D embedding into a foldover-free 2D mesh, satisfying a set of user-specified constraint points. In addition, this approach is mesh free. Therefore, generating smooth texture mapping results is possible without extra smoothing optimization.

In this paper, a novel feature representation to multimodal face recognition is proposed, which possesses three properties: completeness, robustness and compactness. This feature descriptor allows all information of an object to be reproduced and its representation is invariant to rigid motion. In order to effectively take advantage of the proposed feature descriptor, we amend our previous ND-PCA scheme with multidirectional decomposition technique, and provide the estimation of the upper bound error of the amended classifier. It is proved to be linear optimal compared to other linear classifiers. To investigate the numerical performance of the presented feature descriptor, we apply it to both multiple modal and single modal samples, and the revised ND-PCA classifier is performed on the resulting feature representations. The experiments of verification and identification are carried out on two different gallery-probe face databases in order for the results to be evaluated by ROC and CMC curves independently.

A least-squares mesh is a surface representation consisting of a small set of anchor points and the differential and topological properties of the surface. In this paper, we present a novel method to identify motion-sensitive anchor points for least-squares meshes from a set of examples. We present a new method, called clustered teleconnection analysis, to identify the maximally excited points in a subset of basis vectors deduced using principal component analysis. We demonstrate by means of examples that our approach has a smaller reconstruction error and equivalent performance to the current best approaches.

Abstract This paper develops an efficient, physically based shape manipulation technique. It defines a 3D model with profile curves, and uses spine curves generated from the profile curves to control the motion and global shape of 3D models. Profile and spine curves are changed into profile and spine wires by specifying proper material and geometric properties together with external forces. The underlying physics is introduced to deform profile and spine wires through the closed form solution to ordinary differential equations for axial and bending deformations. With the proposed approach, global shape changes are achieved through manipulating spine wires, and local surface details are created by deforming profile wires. A number of examples are presented to demonstrate the applications of our proposed approach in shape manipulation. Copyright © 2010 John Wiley & Sons, Ltd.

We present a fast physics and example based skin deformation method for character animation in this paper. The main idea is to represent the skin surface and its deformations with a group of curves whose computation incurs much less computing overheads than the direct surface-based approach. A fourth order ordinary differential equation is introduced to describe the deformations of the curves between two example skin shapes which relates geometric and physical properties and externally applied forces to shape changes of the curves. The skin deformation is determined from these deformed curves. Several examples are given in this paper to demonstrate the application of the method.

In this paper, we present a method to create and manipulate free form surfaces. A surface is generated from boundary constraints through a vector-valued fourth order partial differential equation involving an externally applied force. In addition to this function, the boundary constraints and the externally applied force also serve as shape manipulation tools. The proposed approach is applied in surface creation, surface manipulation, surface rebuilding and surface animation. It can tackle the problems of tangential continuity in patch modeling and is cheap to compute. The examples given in this paper indicate that our proposed method can create and manipulate free from surfaces easily and quickly.

Abstract Animating an articulated 3D character requires the specification of its interior skeleton structure which defines how the skin surface is deformed during animation. Currently this task is to a large extent accomplished manually, which consumes a large amount of animators' time. This paper presents an automatic rigging method making use of a new geometry entity called the 3D silhouette. The first step is to extract a coarse 3D curve skeleton and some skeletal joints of a character. This curve skeleton is then refined with a perpendicular silhouette. According to the connectivity of the skeletal joints, the hierarchical animation skeleton is finally constructed. By avoiding complicated computation such as voxelization and pruning, this method is simple and efficient, much faster than existing methods. It proves very useful for quick animation production, with applications including games design and prototype graphical systems. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract Motion capture is prevalent in the pipeline of realistic articulated character animation. To define accurate joint positions and joint orientations for the movement of a hierarchical human‐like character without using a pre‐defined skeleton remains a challenge for motion capture studios. In this paper, we present a method for automatically estimating and determining the topology of a hierarchical human skeleton from optical motion capture data based on the human biomechanical information. Through the use of a novel per‐frame based recursive method with joint angle minimization, human skeleton mapping from optical markers and joint angle estimation are achieved in real time. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract Skin sliding is the phenomenon of the skin moving over underlying layers of fat, muscle and bone. Due to the complex interconnections between these separate layers and their differing elasticity properties, it is difficult to model and expensive to compute. We present a novel method to simulate this phenomenon at real‐time by remeshing the surface based on a parameter space resampling. In order to evaluate the surface parametrization, we borrow a technique from structural engineering known as the force density method (FDM)which solves for an energy minimizing form with a sparse linear system. Our method creates a realistic approximation of skin sliding in real‐time, reducing texture distortions in the region of the deformation. In addition it is flexible, simple to use, and can be incorporated into any animation pipeline. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract By simulating motion and deformation of a set of 3D characteristic curves defining surfaces, we introduce a new skin deformation technique to animate skin deformation of character models. This technique consists of two parts, representation of characteristic curves and dynamic skin deformation. The first part is to extract characteristic curves of an existing model, that is a polygon model or a NURBS model, represent the characteristic curves with time‐dependent 3D trigonometric curves, and relate the surface points of the model to these trigonometric curves. The advantage of such a treatment is that it can transform an original 3D problem into a 2D problem, making it much quicker and easier to process and compute. In the second part, a vector‐valued dynamic fourth‐order differential equation is employed to govern the behaviour of the time‐dependent 3D trigonometric curves. These dynamic differential equations incorporate both the time component and the physical properties of the material, which are given by the user. Thus, the character model can be made to animate (deform) following the user‐specified behaviour parameters. Our experiments demonstrate that this technique is able to produce realistic skin deformations efficiently and avoid the undesirable skinning problems suffered by some existing skinning techniques. Copyright © 2008 John Wiley & Sons, Ltd.

For various 3D Animation processing, representing mesh geometry in local rather the world coordinate systems is very useful. One can investigate the representation of vertex locations relative to a local coordinate frame (LCF) in the compression of dynamic 3D meshes. Unlike the world coordinates, which scatter in a wide range and show non linear behavior of the vertices, the local coordinates exhibit a large clustering behavior of the vertex over time. This property is very useful for exploiting a large coherence over the vertex trajectory and between neighboring vertices. In this paper, we discuss the use of the LCF in mesh encoding and we introduce a new and simple predictive scheme for single-rate compression for animated meshes. Our geometry encoding strategy is based on a region growing encoding order and only the delta vectors between original and predicted locations are encoded in a local coordinate system, which splits into two tangential and one normal components. Our approach is simple, efficient and well suited for real time applications.

Abstract Due to their simplicity and intuitiveness, swept surfaces are widely used in many surface modelling applications. In this paper, we present a versatile swept surface technique called the boundary constrained swept surfaces. The most distinct feature is its ability to satisfy boundary constraints, including the shape and tangent conditions at the boundaries of a swept surface. This permits significantly varying surfaces to be both modelled and smoothly assembled, leading to the construction of complex objects. The representation, similar to an ordinary swept surface, is analytical in nature and thus it is light in storage cost and numerically very stable to compute. We also introduce a number of useful shape manipulation tools, such as sculpting forces, to deform a surface both locally and globally. In addition to being a complementary method to the mainstream surface modelling and deformation techniques, we have found it very effective in automatically rebuilding existing complex models. Model reconstruction is arguably one of the most laborious and expensive tasks in modelling complex animated characters. We demonstrate how our technique can be used to automate this process.

Abstract The Cosserat beam model is traditionally used to describe the mechanics of a flexible beam, which is a one‐dimensional entity. In this paper we apply the Cosserat beam model to an arbitrary three‐dimensional object for fast simulation of its vibration behaviours. We encage a detailed mesh model of an object inside a much simpler supporting framework whose edges are effectively the beams (struts) of the Cosserat model. The paper focuses first on extracting the modes of vibration of the framework. Once this is done, dynamic deformations can then be quickly simulated with respect to any applied constraints and dynamic stimuli. The aim of this method is to offer realism traditionally only afforded by the finite element methods (FEMs) while providing more sophistication than the mass–spring method. Copyright © 2007 John Wiley & Sons, Ltd.

Abstract In this paper we present a physically motivated technique for the deformation of animated characters, called thebar‐net driven skinning. We use a bar‐network (bar‐net) as a deforming mechanism. This technique can be used similarly to a conventional skinning tool, but can also make a skin surface behave in a physically plausible manner due to the inherent physical properties of the network. A bar‐net is a structure commonly used in structural engineering. Its shape depends on the structural and material properties and the forces acting upon it. Computing the rest shape of an arbitrary bar‐net is a time‐consuming non‐linear problem. In order to speed up the computation and also for such a bar‐net to be used intuitively to help computer animation, we have defined a set of properties that a desirable bar‐net should satisfy. This allows a bar‐net shape finding problem to be solved using linear equations. We adopt a two‐layer structure for the representation of a skin surface, including a coarse mesh and a fine mesh. To deform a skin surface, we couple a bar‐net to its coarse mesh, which in turn deforms the fine mesh when the coupled bar‐net is deformed. The fine surface mesh can be of different forms, including Nurbs, subdivision surfaces and polygons. Copyright © 2007 John Wiley & Sons, Ltd.

Abstract The skeleton driven skinning technique is still the most popular method for animating deformable human and creature characters. Albeit an industryde factodue to its computational performance and intuitiveness, it suffers from problems like collapsing elbow and candy wrapper joint. To remedy these problems, one needs to formulate the non‐linear relationship between the skeleton and the skin shape of a character properly, which however proves mathematically very challenging. Placing additional joints where the skin bends increases the sampling rate and is an ad hoc way of approximating this non‐linear relationship. In this paper, we propose a method that is able to accommodate the inherent non‐linear relationships between the movement of the skeleton and the skin shape. We use the so‐called curve skeletons along with the joint‐based skeletons to animate the skin shape. Since the deformation follows the tangent of the curve skeleton and also due to higher sampling rates received from the curve points, collapsing skin and other undesirable skin deformation problems are avoided. The curve skeleton retains the advantages of the current skeleton driven skinning. It is easy to use and allows full control over the animation process. As a further enhancement, it is also fairly simple to build realistic muscle and fat bulge effect. A practical implementation in the form of a Maya plug‐in is created to demonstrate the viability of the technique. Copyright © 2006 John Wiley & Sons, Ltd.

Abstract As skin shape depends on the underlying anatomical structure, the anatomy‐based techniques usually afford greater realism than the traditional skeleton‐driven approach. On the downside, however, it is against the current animation workflow, as the animator has to model many individual muscles before the final skin layer arrives, resulting in an unintuitive modelling process. In this paper, we present a new anatomy‐based technique that allows the animator to start from an already modelled character. Muscles having visible influence on the skin shape at the rest pose are extracted automatically by studying the surface geometry of the skin. The extracted muscles are then used to deform the skin in areas where there exist complex deformations. The remaining skin areas, unaffected or hardly affected by the muscles, are handled by the skeleton‐driven technique, allowing both techniques to play their strengths. In order for the extracted muscles to produce realistic local skin deformation during animation, muscle bulging and special movements are both represented. Whereas the former ensues volume preservation, the latter allows a muscle not only to deform along a straight path, but also to slide and bend around joints and bones, resulting in the production of sophisticated muscle movements and deformations. Copyright © 2006 John Wiley & Sons, Ltd.

Mesh-free deformation is an effective method for the simulation of deformable objects and characters in computer animation. It bears the merits of flexible control, good accuracy and easy implementation. Similar to other physically-based deformation techniques, however, computational cost remains a pressing issue, especially for large scale problems. Based on our previous work, in this paper we investigate the underlying structure of the numerical approach in order to speed up the computation. Three algorithms are presented and their efficiency and convergence are analysed. Our results show that we are able to significantly reduce the computation time and memory usage with little loss in accuracy. The new technique is capable of achieving interactive frame rates with large models.

In this paper, a simple, accurate and efficient method is presented for free-form surface blending with curvature continuity. The method is based on an approximate analytical solution to a sixth order partial differential equation (PDE) subjected to blending boundary constraints. The most accurate and fast solution to a PDE is the closed form solution, which however is either very difficult to come by or does not exist. Comparison studies show that our proposed method has similar accuracy and computational efficiency to those of the closed form solution. We also examine the effects of the vector-valued parameters on the shape of blending surfaces and apply our proposed method in various surface blending examples. The outcomes indicate that the proposed approach is able to solve not only simple surface blending problems, but also complex ones which otherwise could only be solved using expensive numerical methods. In addition, this method can be extended to satisfy up to curvature continuities along 3 or 4 boundaries of a 3- or 4-sided patch and serve as an effective tool for the generation and manipulation of complex surfaces.

Optimization techniques have proven to be a powerful approach for generating new motions. In this paper, we present a physically based optimization method to synthesize motions by using motion capture data as input. We assume that the captured motion data is physically plausible. We start by defining and estimating the physical properties of human characters. The procedure of motion synthesis is from coarse to fine according to the objective function and physical constraints. Our motion synthesis is like a motion editing method, which is appropriate for motion correction and extrapolation. By this means, we can correct and eliminate unrealistic motion data.

The reliability of facial recognition techniques is often affected by the variation of illumination, such as shadows and illumination direction changes. In this paper, we present a novel framework, called the self-quotient image, for the elimination of the lighting effect in the image. Although this method has a similar invariant form to the quotient image by Shashua etc. (2001), it does not need the alignment and bootstrap images. Our method combines the image processing technique of edge-preserved filtering with the Retinex applications of by Jobson, et al., (1997) and Gross and Brajovie (2003). We have analyzed this algorithm with a 3D imaging model and formulated the conditions where illumination-invariant and -variant properties can be realized, respectively. A fast anisotropic filter is also presented. The experiment results show that our method is effective in removing the effect of illumination for robust face recognition.

This work presents a new deformation algorithm for computer graphics and animation applications. The algorithm is motivated by the mass-spring systems and the ChainMail technique and possesses advantages of both methods. It uses a one-step approach to find the deformation, and therefore is very fast. Simulation parameters can easily be chosen in order to represent deformation characteristics. The algorithm is applicable to various simulation applications and produces very promising results.

This work surveys the numerical techniques developed in computer graphics for editing and revising data driven character animation. We evaluate systems that have been developed for interactive, off-line and on-line animation, using motion capture data or animator generated data, which enable the control of complex character animation.

It has been reported a fourth order partial differential equation is effective in solving surface blending problems. In this paper, we present a new approximate solution to the fourth order partial differential equation and apply it to a number of surface blending tasks. The approximate solution consists of two parts: a blended part of boundary functions is used to accurately satisfy the original boundary conditions that define the blending surfaces; and a bivariate polynomial with zeroed boundary conditions, which is used to minimize the error of the fourth order partial differential equation. Using the developed method, a number of examples are investigated to demonstrate the applications of the proposed method in surface blending.

Blending is an interesting subject in computer graphics and CAD. Although surface blending has been well studied, to our knowledge, solid blending remains a blank research area. In this paper, we develop a partial differential equation (PDE) based solid blending method where a vector-valued fourth order PDE together with some surface boundary conditions are involved. In order to solve the partial differential equation, we propose an approximate analytical method which satisfies all the boundary conditions exactly. Unlike surface blending which only gives the information on the blending surface, solid blending represents properly not only the external surface, but also the interior of the solid. Three examples are given in the paper to demonstrate the applications of the proposed method in solid blending.

Abstract Although the control‐point based parametric approach is used most widely in free‐form surface modelling, complementary techniques co‐exist to meet various specialised requirements. The partial differential equation (PDE) based modelling approach is especially suitable for satisfying surface boundary constraints. They are also effective for the generation of families of free‐form surfaces, which share a common base and differ in their secondary features. In this paper, we present a fast surface modelling method using a sixth order PDE. This PDE provides enough degrees of freedom not only to accommodate tangent, but also curvature boundary conditions and offers more shape control parameters to serve as user controls for the manipulation of surface shapes. In order to achieve real‐time performance, we have constructed a surface function and developed a high‐precision approximate solution to the 6th order PDE. Unlike some existing PDE‐based techniques, this resolution method can satisfy the boundary conditions exactly, and is able to create free‐form surfaces as fast and almost as accurately as the closed‐form (analytical) solutions. Due to the fact that it has sufficient degrees of freedom to accommodate the continuity of 3‐sided and 4‐sided surface patches at their boundaries, this method is able to model complex surfaces consisting of multiple patches. Compared with existing PDE‐based modelling methods, this method is both fast and can solve a larger class of surface modelling problems. Categories and Subject Descriptors (according to ACM CCS): I.3.5 [Computer Graphics]: Curves, surfaces, solid, and object representations; physically based modelling

Abstract Existing physically based deformation techniques, such as the finite element method (FEM) and the mass‐spring systems (MSS), require the deformed object to be properly meshed. This is arguably the most expensive manual intervention process. In this paper, we propose a mesh‐free deformation technique where only unconnected points are involved. The idea is to develop an approximate analytical solution using the Kelvin solution. Due to the fact that no mesh is involved, deforming a complex shape is as straightforward as deforming a simple one. Furthermore, the trade‐off between efficiency and accuracy is easy to achieve by redistributing the points concerned. Experiments show that this method is fast and offers similar accuracy to the FEM. Copyright © 2004 John Wiley & Sons, Ltd.

Abstract Two approaches, image‐space and object‐space, exist for non‐photorealistic rendering (NPR). The object‐space approach has an advantage that it is able to access the 3D shape information of the scene with a 3D model. In this paper, we present a non‐stroke based image‐space technique that has the strength similar to that of the object‐space approach, but without involving explicit 3D models. The basic idea is to factorise the Lambertian model to obtain the shape and texture information implicitly contained in the image. We also develop a specific anisotropic low‐pass image filter, which can smooth an image without blurring important features. With the developed techniques, we present two NPR styles, the sketching style and the painting style. Although our work does not aim to mimic any specific existing art form, the sketching style can generate effects similar to those of the pen‐and‐ink and the painting style is able to produce the colour permeating effects of watercolour. Copyright © 2004 John Wiley & Sons, Ltd.

Dynamic surface modeling is an important subject of geometric modeling due to their extensive applications in engineering design, entertainment and medical visualization. Many deformable objects in the real world are dynamic objects as their shapes change over time. Traditional geometric modeling methods are mainly concerned with static problems, therefore unsuitable for the representation of dynamic objects. Apart from the definition of a dynamic modeling problem, another key issue is how to solve the problem. Because of the complexity of the representations, currently the finite element method or finite difference method is usually used. Their major shortcoming is the excessive computational cost, hence not ideal for applications requiring real-time performance. We propose a representation of dynamic surface modeling with a set of fourth order dynamic partial differential equations (PDEs). To solve these dynamic PDEs accurately and efficiently, we also develop an effective resolution method. This method is further extended to achieve local deformation and produce n-sided patches. It is demonstrated that this new method is almost as fast and accurate as the analytical closed form resolution method and much more efficient and accurate than the numerical methods.

This paper addresses the problem faced by animators in creating accurate and realistic walk cycles, as well as coherent interaction with the immediate production environment. Modern software packages often contain modules for implementing dynamics, without the need for specialist programs. In this paper, how to best use these modules to create the desired realistic effects is explored and expanded upon, including driving forces for balanced bipedal motion, friction/slippage between the feet of the model and the surface walked upon. A simple "wind-up" toy model is used as an example for the demonstration of the process, but the approaches taken could be similarly extended into more complicated models and applications.

Texture-mapping a triangular or rectangular surface is easy to perform. However it is not so for irregular surfaces (n-sided surfaces where n > 4) due to lack of proper global parameters. In this paper we propose a mapping function capable of texturing an irregular surface based on the modelling expression of Zheng-Ball n-sided surfaces.

Surface representation using the solution to a partial differential equation (PDE) is an important topic of computer graphics and computer-aided design. In existing references, various free-from surfaces were created with a 4th-order PDE, which is only able to meet the tangential conditions at the surface boundaries. The need for a 6th-order PDE in surface modelling arises in two situations: one is to generate surfaces with curvature continuity and the other is to use curvature values as a user handle for surface shape manipulation. In this paper, we introduce such a 6th-order PDE for free-form surface generation and develop a finite difference method to solve this PDE. We also investigate the effects of boundary curvature and the vector-valued shape parameters on the surface shape. It was found that their variation has a strong influence on the shape of the surfaces and that, therefore, they can be used as flexible user handles.

Partial differential equations (PDEs) are powerful tools for the generation of free-form surfaces. In this paper, techniques of surface representation using PDEs of different orders are investigated. In order to investigate the real-time performance and capacity of surface generation based on the PDE method, the forms of three types of partial differential equations are put forward, which are the second, mixed and fourth order PDEs. The closed form solutions of these PDEs are derived. The advantages and disadvantages of each of them are discussed. A number of examples are given to demonstrate the use and effectiveness of the techniques.