Simulating the interactions between fluids and porous media has attracted significant attention in computer graphics. A key challenge in this domain is modeling the Poro-Elasto-Capillary (PEC) coupling effect which describes the intricate interplay of three physical phenomena in soft porous materials: pore-structure evolution, elastic deformation, and wetting driven by capillary pressure. These phenomena collectively govern dynamic behavior such as the softening and fracturing of biscuits upon water absorption or the swelling of cellulose sponges due to liquid infiltration. Most existing simulation methods model porous media either as static grids or as solid particles with augmented water content attributes, failing to capture the full spectrum of PEC-driven effects due to the lack of physical modeling for elasticity, dynamic porosity changes, and capillary interactions. We propose a multiphase particle-based framework to holistically simulate PEC coupling effects with porous media. We develop a physics-driven model that captures elasticity and dynamic pore-structure evolution under capillary action, enabling realistic simulation of softening and swelling. We derive a saturation-aware pressure Poisson equation to enforce fluid incompressibility within and around the porous medium, ensuring accurate capillary-driven flow while preserving mass and momentum. Finally, we propose a representative elementary volume-based formulation to unify the modeling of homogeneous macro-porous media and cavity-embedded structures, enhancing the representation of pore-scale PEC effects. Comparisons with prior work and real footage show the advantages of our approach in achieving visually realistic fluid-porous media interactions.

ABSTRACT We propose a novel simulation method for surface tension effects based on the Smoothed Particle Hydrodynamics framework, capturing versatile tension effects using a unified interface energy description. Guided by the principle of energy minimization, we compute the interface energy from multiple interfaces solely using the original kernel function estimation, which eliminates the dependence on second‐order derivative discretization. Subsequently, we incorporate an inertia term into the energy function to strike a balance between tension effects and other forces. To simulate tension, we propose an energy diffusion‐based method for minimizing the objective energy function. The particles at the interface are iteratively shifted from high‐energy regions to low‐energy regions through several iterations, thereby achieving global interface energy minimization. Furthermore, our approach incorporates surface tension parameters as variable quantities within the energy framework, enabling automatic resolution of tension spatial gradients without requiring explicit computation of interfacial gradients. Experimental results demonstrate that our method effectively captures the wetting, capillary, and Marangoni effects, showcasing significant improvements in both the accuracy and stability of tension simulation.

Abstract Image up-scaling and super-resolution (SR) techniques have been a hot research topic for many years due to its large impact in the field of medical imaging, surveillance etc. Especially single image super-resolution (SISR) become very popular because of the fast development of deep convolution neural network (DCNN) and the low requirement on the input. They are achieving outstanding performance. However, there are still problems in the state-of-the-art works, especially from two perspectives: 1. failed at exploiting the hierarchical characteristics from the input, resulting in loss of information and artifacts in the final high resolution (HR) image; 2. failed to handle arbitrary-sized images; the existing research works are focused on fixed size input images. To address these challenges, this paper proposed a residual dense network (RDN) and multi-scale sub-pixel convolution network (MSSPCN) which are integrated into a Collapsible Linear Block Super Efficient Super-Resolution (SESR) network. The RDNs aims to tackle the first challenge, carrying the hierarchical features from end-to-end. An adaptive cropping strategy (ACS) technique is introduced before feature extraction targeting at the image size challenge. The novelty of this work is extracting the hierarchical features and integrating RDNs with MSSPCNs. The proposed network can upscale any arbitrary-sized image (1080p) to ×2 (4K) and ×4 (8K). To secure ground truth for evaluation, this paper follows the opposite flow, generating the input LR images by down-sampling the given HR images (ground truth). To evaluate the performance, the proposed algorithm is compared with eight state-of-the-art algorithms, both quantitatively and qualitatively. The results are verified on six benchmark datasets. The extensive experiments justify that the proposed architecture performs better than other methods and upscales the images satisfactorily.

High-resolution images come with rich color information and texture details. Due to the rapid upgrading of display devices and rendering technologies, high-resolution real-time rendering faces the computational overhead challenge. To address this, the current mainstream solution is to render at a lower resolution and then upsample to the target resolution by supersampling techniques. However, while many prior supersampling approaches have attempted to exploit rich rendered data such as color, depth, motion vectors at low resolution, there is little discussion on how to harness high-frequency information that is readily available in the high-resolution (HR) G-buffers of modern renders. In this article, we seek to investigate how to fully leverage information from HR G-buffers to maximize the visual quality of supersampling results. We propose a neural network for real-time supersampling of rendered content, which is based on several core designs, including gated G-buffers encoder, G-buffers attended encoder and reflection-aware loss. These designs are especially made for the sake of effectively using HR G-buffers, enabling faithful recovery of a variety of high-frequency scene details from low-resolution, highly aliased inputs. Furthermore, a simple occlusion-aware blender is proposed to efficiently rectify dis-occluded features in the warped previous frame, allowing us to better exploit history information to improve temporal stability. The experiments show that our method, equipped with strong ability to harness HR G-buffer information, significantly improves the visual fidelity of high-resolution reconstructions upon previous state-of-the-art methods, even for challenging $4 \times 4$4×4 upsampling, while still being compute-efficient.

We present a Lagrangian dynamic importance Monte Carlo method without non-trivial random walks for solving the Velocity-Vorticity Poisson Equation (VVPE) in Smoothed Particle Hydrodynamics (SPH) for vortical flows. Key to our approach is the use of the Kinematic Vorticity Number (KVN) to detect vortex cores and to compute the KVN-based importance of each particle when solving the VVPE. We use Adaptive Kernel Density Estimation (AKDE) to extract a probability density distribution from the KVN for the the Monte Carlo calculations. Even though the distribution of the KVN can be non-trivial, AKDE yields a smooth and normalized result which we dynamically update at each time step. As we sample actual particles directly, the Lagrangian attributes of particle samples ensure that the continuously evolved KVN-based importance, modeled by the probability density distribution extracted from the KVN by AKDE, can be closely followed. Our approach enables effective vortical flow simulations with significantly reduced computational overhead and comparable quality to the classic Biot-Savart law that in contrast requires expensive global particle querying.

Automatic summarization of medical dialogues, which converts colloquial doctor-patient conversations into concise notes, is increasingly important due to the growing complexity of healthcare data. However, the complexity of medical language and the lack of annotated datasets pose challenges for summarization models. In this paper, we propose a MediExtract Distillation Framework (MEDF), a novel hybrid teacher-student distillation process that leverages the power of Large Language Models (LLMs) in information capturing to enhance the performance of a smaller student model. Utilizing medical key information generated by GPT-3.5-Turbo, the model training involves two feedforward branches per iteration: one using ground truth as labels and another using generated structured medical key information as an auxiliary supervision. We validated our method on the MTS-Dialogue dataset, achieving a +2.1% improvement in BLEURT compared to previous methods, demonstrating its effectiveness in summarizing medical dialogues. Additionally, using UMLS-based BERTScore, we observed a +1.8% increase in MedBERTScore for medical term extraction, highlighting our model’s practical benefits in clinical information processing. Our framework is publicly available at: https://github.com/Xiaoxiao-Liu/distill-d2n.git

In this paper, we present PCoTTA, an innovative, pioneering framework for Continual Test-Time Adaptation (CoTTA) in multi-task point cloud understanding, enhancing the model's transferability towards the continually changing target domain. We introduce a multi-task setting for PCoTTA, which is practical and realistic, handling multiple tasks within one unified model during the continual adaptation. Our PCoTTA involves three key components: automatic prototype mixture (APM), Gaussian Splatted feature shifting (GSFS), and contrastive prototype repulsion (CPR). Firstly, APM is designed to automatically mix the source prototypes with the learnable prototypes with a similarity balancing factor, avoiding catastrophic forgetting. Then, GSFS dynamically shifts the testing sample toward the source domain, mitigating error accumulation in an online manner. In addition, CPR is proposed to pull the nearest learnable prototype close to the testing feature and push it away from other prototypes, making each prototype distinguishable during the adaptation. Experimental comparisons lead to a new benchmark, demonstrating PCoTTA's superiority in boosting the model's transferability towards the continually changing target domain. Our source code is available at: https://github.com/Jinec98/PCoTTA.

Abstract For vortex particle methods relying on SPH‐based simulations, the direct approach of iterating all fluid particles to capture velocity from vorticity can lead to a significant computational overhead during the Biot‐Savart summation process. To address this challenge, we present a Monte Carlo vortical smoothed particle hydrodynamics (MCVSPH) method for efficiently simulating turbulent flows within an SPH framework. Our approach harnesses a Monte Carlo estimator and operates exclusively within a pre‐sampled particle subset, thus eliminating the need for costly global iterations over all fluid particles. Our algorithm is decoupled from various projection loops which enforce incompressibility, independently handles the recovery of turbulent details, and seamlessly integrates with state‐of‐the‐art SPH‐based incompressibility solvers. Our approach rectifies the velocity of all fluid particles based on vorticity loss to respect the evolution of vorticity, effectively enforcing vortex motions. We demonstrate, by several experiments, that our MCVSPH method effectively preserves vorticity and creates visually prominent vortical motions.

Physics-based fluid simulation has played an increasingly important role in the computer graphics community. Recent methods in this area have greatly improved the generation of complex visual effects and its computational efficiency. Novel techniques have emerged to deal with complex boundaries, multiphase fluids, gas–liquid interfaces, and fine details. The parallel use of machine learning, image processing, and fluid control technologies has brought many interesting and novel research perspectives. In this survey, we provide an introduction to theoretical concepts underpinning physics-based fluid simulation and their practical implementation, with the aim for it to serve as a guide for both newcomers and seasoned researchers to explore the field of physics-based fluid simulation, with a focus on developments in the last decade. Driven by the distribution of recent publications in the field, we structure our survey to cover physical background; discretization approaches; computational methods that address scalability; fluid interactions with other materials and interfaces; and methods for expressive aspects of surface detail and control. From a practical perspective, we give an overview of existing implementations available for the above methods.

Recent advances in NeRF-based 3D-aware GANs have achieved outstanding performance, especially in the realm of human facial representations, making projection of facial images back into their latent space superior and preferable compared to 2D GAN inversion. However, the direct application of 2DGAN inversion techniques to 3DGAN raises challenges due to potential appearance distortions and geometric inconsistences. To tackle these issues, this work presents a novel integrated framework that combines a composite inversion pipeline in both the SS and W+ spaces and integrates a contrastive-based training strategy, ensuring proficient disentanglement within the module. Moreover, we design a facial semantic manipulation technique based on dimensional analysis of the latent code, which is fully compatible with the proposed 3DGAN inversion pipeline. Comprehensive experimental validations substantiate the effectiveness of the proposed approach in executing 3d-aware face inversion and semantic editing tasks, presenting a robust technological solution for a diverse array of digital human modeling applications in the downstream.

Applying Artificial Intelligence (AI) techniques such as natural language generation in assisting medical treatment and diagnosis has made distinguished progress. One such technique is dialogue generation. The application of a medical dialogue system in assisting medical treatment has great potential to explore. This paper serves as a survey of digging application of AI techniques in knowledge-grounded dialogue generation for medical conversation systems. Meanwhile, we provide an academic visualization method to present such references.

Particle-based simulations have become increasingly popular in real-time applications due to their efficiency and adaptability, especially for generating highly dynamic fluid effects. However, the swift and stable simulation of interactions among distinct fluids continues to pose challenges for current mixture model techniques. When using a single-mixture flow field to represent all fluid phases, numerical discontinuities in phase fields can result in significant losses of dynamic effects and unstable conservation of mass and momentum. To tackle these issues, we present an advanced implicit mixture model for smoothed particle hydrodynamics. Instead of relying on an explicit mixture field for all dynamic computations and phase transfers between particles, our approach calculates phase momentum sources from the mixture model to derive explicit and continuous velocity phase fields. We then implicitly obtain the mixture field using a phase-mixture momentum-mapping mechanism that ensures conservation of incompressibility, mass, and momentum. In addition, we propose a mixture viscosity model and establish viscous effects between the mixture and individual fluid phases to avoid instability under extreme inertia conditions. Through a series of experiments, we show that, compared to existing mixture models, our method effectively improves dynamic effects while reducing critical instability factors. This makes our approach especially well-suited for long-duration, efficiency-oriented virtual reality scenarios.

The use of social media has become so popular that people share photos every day on them. Automatic face recognition and tagging of people's photos have caused privacy preservation issues and some methods have been proposed for hiding the identity of presented people in these images. Blurring and blacking the face area, adding physical adversarial patches to the face, and adding adversarial masks are some proposed methods for this purpose. However, these methods particularly suffer from dissimilarity of the input and output images and inadequate performance in identity concealment from automatic face recognition (AFR) systems. In this paper, we propose the Generative Mask-guided Face Image Manipulation (GMFIM) model based on Generative Adversarial Networks (GANs) to apply imperceptible edits to the input face image to preserve the identity of the person in the image. Our model consists of a face mask module, a GAN-based optimization module, and a merge module. Different criteria are considered in the objective function of the optimization step to produce high-quality images that are as similar as possible to the input image while they cannot be recognized by AFR systems. The results of the experiments on different datasets show that our model provides promising results in terms of the quality of the generated images and the identity concealment performance.

This paper proposes a real-time fluid rendering method based on the screen space rendering scheme for particle-based fluid simulation. Our method applies anisotropic transformations to the point sprites to stretch the point sprites along appropriate axes, obtaining smooth fluid surfaces based on the weighted principal components analysis of the particle distribution. Then we combine the processed anisotropic point sprite information with popular screen space filters like curvature flow and narrow-range filters to process the depth information. Experiments show that the proposed method can efficiently resolve the issues of jagged edges and unevenness on the surface that existed in previous methods while preserving sharp high-frequency details.

Abstract Virtual reality (VR) and augmented reality (AR) applications are becoming increasingly prevalent. However, constructing realistic 3D hands, especially when two hands are interacting, from a single RGB image remains a major challenge due to severe mutual occlusion and the enormous diversity of hand poses. In this article, we propose a disturbing graph contrastive learning strategy for two‐hand 3D reconstruction. This involves a graph disturbance network designed to generate graph feature pairs to enhance the consistency of the two‐hand pose features. A contrastive learning module leverages high‐quality generative features for a strong feature expression. We further propose a similarity distinguish method to divide positive and negative features for accelerating the model convergence. Additionally, a multi‐term loss is designed to balance the relation among the hand pose, the visual scale and the viewpoint position. Our model has achieved state‐of‐the‐art results in the InterHand2.6M benchmark. Ablation studies show the model's great ability to correct unreasonable hand movements. In subjective assessments, our graph disturbance learning method significantly improves the construction of realistic 3D hands, especially when two hands are interacting.

Abstract In this article, we propose a point cloud synthesis method based on stochastic differential equations. We view the point cloud generation process as smoothly transforming from a known prior distribution toward the high‐likelihood shape by point‐level denoising. We introduce a conditional corrector sampler to improve the quality of point clouds. By leveraging Markov chain Monte Carlo sample, our method can synthesize realistic point clouds. We additionally prove that our approach can be trained in an auto‐encoding fashion and reconstruct the point cloud faithfully. Furthermore, our model can be extended on a downstream application of point cloud completion. Experimental results demonstrate the effectiveness and efficiency of our method.

Abstract Fluid simulation has been one of the most critical topics in computer graphics for its capacity to produce visually realistic effects. The intricacy of fluid simulation manifests most with interacting dynamic elements. The coupling for such scenarios has always been challenging to manage due to the numerical instability arising from the coupling boundary between different elements. Therefore, we propose an implicit smoothed particle hydrodynamics fluid‐elastic coupling approach to reduce the instability issue for fluid‐fluid, fluid‐elastic, and elastic‐elastic coupling circumstances. By deriving the relationship between the universal pressure field with the incompressible attribute of the fluid, we apply the number density scheme to solve the pressure Poisson equation for both fluid and elastic material to avoid the density error for multi‐material coupling and conserve the non‐penetration condition for elastic objects interacting with fluid particles. Experiments show that our method can effectively handle the multiphase fluids simulation with elastic objects under various physical properties.

Abstract Fluid simulation is well‐known for being visually stunning while computationally expensive. Spatial adaptivity can effectively ease the computational cost by discretizing the simulation space with varying resolutions. Adaptive methods nowadays mainly focus on the mechanism of refining the fluid surfaces to obtain more vivid splashes and wave effects. But such techniques hinder further performance gain under the condition where most of the vast fluid surface is tranquil. Moreover, energetic flow beneath the surface cannot be adequately captured with the interior of the fluid still being simulated under coarse discretization. This article proposes a novel boundary‐distance based adaptive method for smoothed particle hydrodynamics fluid simulation. The signed‐distance field constructed with respect to the coupling boundary is introduced to determine particle resolution in different spatial positions. The resolution is maximal within a specific distance to the boundary and decreases smoothly as the distance increases until a threshold is reached. The sizes of the particles are then adjusted towards the resolution via splitting and merging. Additionally, a wake flow preservation mechanism is introduced to keep the particle resolution at a high level for a period of time after a particle flows through the boundary object to prevent the loss of flow details. Experiments show that our method can refine fluid–solid coupling details more efficiently and effectively capture dynamic effects beneath the surface.

Abstract In recent years, it becomes possible to extract hair information for hair reconstruction from multiple cameras or monocular camera. Using a single image as the input avoids the high cost setups and complex calibration compared to multiviewed reconstruction. Taking advantage of an extendible hairstyle database, this paper introduced Struct2Hair, a novel single‐viewed hair modelling approach by extracting hair shape descriptor (HSD). The HSD is defined as the fundamental structure‐aware feature, which is a combination of critical shapes in a hairstyle. A complete dataset of critical hair shapes is constructed from a known database of three‐dimensional (3D) hair models. We first analyze the input two‐dimensional (2D) image to extract the orientation information and 2D hair sketch automatically. The extracted information is then used to retrieve the corresponding critical shapes with optimization to build the robust HSD. Finally, the HSD constructs a weighted 3D hair orientation field to guide full‐head hair model generation. Our method can preserve local geometric features of hair and retain the whole shape of the hairstyle globally owing to the HSD, which will benefit further hair editing and stylization.

Abstract Point cloud generation aims to synthesize point clouds that do not exist in supervised dataset. Generating a point cloud with certain semantic labels remains an under‐explored problem. This paper proposes a formulation called DiffusionPointLabel, which completes point‐label pair generation based on a DDPM generative model (Denoising Diffusion Probabilistic Model). Specifically, we use a point cloud diffusion generative model and aggregate the intermediate features of the generator. On top of this, we propose Feature Interpreter that transforms intermediate features into semantic labels. Furthermore, we employ an uncertainty measure to filter unqualified point‐label pairs for a better quality of generated point cloud dataset. Coupling these two designs enables us to automatically generate annotated point clouds, especially when supervised point‐labels pairs are scarce. Our method extends the application of point cloud generation models and surpasses state‐of‐the‐art models.

Abstract In this article, we propose a novel technique based on Head Movement tracking to explore multilayer digital content. We extend an existing method by Kazemi et al. dealing with the extraction of facial landmarks to define the “head‐gaze” of the user. We use the “head‐gaze” to calculate the users' on‐screen coordinates. Hovering the cursor over an interactive area for a given time threshold allows users to explore the next layer contents. Our experimental sessions allowed us to measure the technique's level of control and usability. Our results were promising, and users were able to interact with considerably small regions. Furthermore, our lightweight method can be used with a low‐cost camera or webcam and a wide range of screen sizes and distances.

Abstract It is significant to achieve automatic arrangement for bas‐relief layout which can be noticeably more efficient than the time‐consuming manual process. In fact, nearly none work has been reported in terms of bas‐relief layout arrangement. In this paper, we propose a novel approach to tackle this problem. Specifically, we first identify the evaluation indicators to account for different aesthetic factors, and model the goodness of each indicator. We then cast the bas‐relief layout as a combinatorial optimization problem based on those evaluation indicators and a geometric mean model. The contribution of this paper is to propose an objective function for bas‐relief layout and apply simulated annealing algorithm for optimization. Experiments show that our method is effective, in terms of layout arrangement for bas‐relief generation. In addition, this method can synthesize a few models arrangement and investigate which evaluation indicators will affect the aesthetic perception of the bas‐relief.

Semantic reconstruction of indoor scenes refers to both scene understanding and object reconstruction. Existing works either address one part of this problem or focus on independent objects. In this paper, we bridge the gap between understanding and reconstruction, and propose an end-to-end solution to jointly reconstruct room layout, object bounding boxes and meshes from a single image. Instead of separately resolving scene understanding and object reconstruction, our method builds upon a holistic scene context and proposes a coarse-to-fine hierarchy with three components: 1. room layout with camera pose; 2. 3D object bounding boxes; 3. object meshes. We argue that understanding the context of each component can assist the task of parsing the others, which enables joint understanding and reconstruction. The experiments on the SUN RGB-D and Pix3D datasets demonstrate that our method consistently outperforms existing methods in indoor layout estimation, 3D object detection and mesh reconstruction.

Point completion refers to complete the missing geometries of objects from partial point clouds. Existing works usually estimate the missing shape by decoding a latent feature encoded from the input points. However, real-world objects are usually with diverse topologies and surface details, which a latent feature may fail to represent to recover a clean and complete surface. To this end, we propose a skeleton-bridged point completion network (SK-PCN) for shape completion. Given a partial scan, our method first predicts its 3D skeleton to obtain the global structure, and completes the surface by learning displacements from skeletal points. We decouple the shape completion into structure estimation and surface reconstruction, which eases the learning difficulty and benefits our method to obtain on-surface details. Besides, considering the missing features during encoding input points, SK-PCN adopts a local adjustment strategy that merges the input point cloud to our predictions for surface refinement. Comparing with previous methods, our skeleton-bridged manner better supports point normal estimation to obtain the full surface mesh beyond point clouds. The qualitative and quantitative experiments on both point cloud and mesh completion show that our approach outperforms the existing methods on various object categories.

Abstract Superpixel segmentation is important for promoting various image processing tasks. However, existing methods still have difficulties in generating high‐quality superpixels in textured images, because they cannot separate textures from structures well. Though texture filtering can be adopted for smoothing textures before superpixel segmentation, the filtering would also smooth the object boundaries, and thus weaken the quality of generated superpixels. In this paper, we propose to use the adaptive scale box smoothing instead of the texture filtering to obtain more high‐quality texture and boundary information. Based on this, we design a novel distance metric to measure the distance between different pixels, which considers boundary, color and Euclidean distance simultaneously. As a result, our method can achieve high‐quality superpixel segmentation in textured images without texture filtering. The experimental results demonstrate the superiority of our method over existing methods, even the learning‐based methods. Benefited from using boundaries to guide superpixel segmentation, our method can also suppress noise to generate high‐quality superpixels in non‐textured images.

This special issue contains 28 full papers selected from the Computer Animation and Social Agents 2020 Conference (CASA2020). This conference was founded by the Computer Graphics Society in 1988 in Geneva and is the oldest conference on Computer Animation in the world. It has been held in many countries around the world and in recent years in Beijing, China (2018), Paris, France (2019) and this year in Bournemouth, United Kingdom. Because of the Covid-19 pandemic, this year, the conference will be held online through the Youtube Channel. The best paper award will be announced on the conference website after the conference. We would like to thank the authors for sharing their research findings by submitting papers to CASA2020. We are very grateful to the Program Committee members for reviewing the papers and to all the people who have contributed to the success of CASA2020 in Bournemouth. The conference is organized by Bournemouth University under the guidance of the Computer Graphics Society (CGS). Conference co-chairs Jian Jun Zhang (Bournemouth University, UK) Nadia Magnenat Thalmann (University of Geneva, Switzerland and Nanyang Technological University, Singapore) Program co-chairs Daniel Thalmann (EPFL, Switzerland) Xiaosong Yang (Bournemouth University, UK) Weiwei Xu (Zhejiang University, China) Publicity chair Jian Chang (Bournemouth University, UK) Local chair Feng Tian (Bournemouth University, UK) International program committee Nadine Aburumman, Brunel University, UK Norman Badler, University of Pennsylvania, USA Selim Balcisoy, Sabanci University, Turkey Loic Barthe, IRIT—Université de Toulouse, France Jan Bender, RWTH Aachen University, Germany Raphaëlle Chaine, LIRIS Université Lyon 1, France Jian Chang, Bournemouth University, UK Fred Charles, Bournemouth University, UK Parag Chaudhuri, Indian Institute of Technology, Bombay, India Marc Christie, INRIA, France Justin Dauwels, Nanyang Technological University, Singapore Shujie Deng, King's College London, UK Zhigang Deng, University of Houston, USA Etienne de Sevin, SANPSY University of Bordeaux, France Petros Faloutsos, York University, Canada Christos Gatzidis, Bournemouth University, UK Ugur Gudukbay, Bilkent University, Turkey Shihui Guo, Xiamen University, China Xiaohu Guo, The University of Texas at Dallas, USA James Hahn, George Washington University, USA Carlo Harvey, Birmingham City University, UK Gaoqi He, East China Normal University, China Ying He, Nanyang Technological University, Singapore Kemao Qian, Nanyang Technological University, Singapore Ruizhen Hu, Shenzhen University, China Jinyuan Jia, Tongji University, China Tao Jiang, University of Surrey, UK Xiaogang Jin, Zhejiang University, China Marcelo Kallmann, University of California, Merced, USA Prem Kalra, IIT Delhi, India Dongwann Kang, Seoul National University of Science and Technology, Korea Mubbasir Kapadia, Rutgers University, USA Min H. Kim, Korea Advanced Institute of Science and Technology, Korea Scott King, Texas A&M University—Corpus Christi, USA Taesoo Kwon, Hanyang University, China Sung-Hee Lee, Korea Advanced Institute of Science and Technology, Korea Wonsook Lee, University of Ottawa, Canada Tsai-Yen Li, National Chengchi University, Taiwan Guoliang Luo, East China Jiaotong University, China Chongyang Ma, Snap Inc., USA Anderson Maciel, Universidade Federal do Rio Grande do Sul, Brazil Nadia Magnenat Thalmann, University Of Geneva, Switzerland Shigeo Morishima, Waseda University, Japan Soraia Musse, Pontificia Universidade Catolica do Roi Grande do Sul, PUCRS, Brazil Rahul Narain, Indian Institute of Technology, Delhi, India Junjun Pan, Beihang University, China Nuria Pelechano, Universitat Politècnica de Catalunya, Spain Julien Pettre, INRIA, France Nicolas Pronost, Université Claude Bernard Lyon 1, France Kun Qian, King's College London, UK Craig Schroeder, University of California, Riverside, USA Ari Shapiro, Embody Digital, USA Hubert P. H. Shum, Northumbria University, UK Shinjiro Sueda, Texas A&M University, USA Daniel Thalmann, Ecole Polytechnique Fédérale de Lausanne, Switzerland Feng Tian, Bournemouth University, UK Yiying Tong, Michigan State University, USA Meili Wang, Northwest A&F University, China Zhao Wang, Zhejiang University, China Enhua Wu, University of Macau & ISCAS, China Zhongke Wu, Beijing Normal University, China Weiwei Xu, Zhejiang University, China Yachun Fan, Beijing Normal University, China Bailin Yang, Zhejiang Gongshang University, China Yin Yang, University of New Mexico, USA Xiaosong Yang, Bournemouth University, UK Yuting Ye, Oculus Research, USA Lihua You, Bournemouth University, UK Hongchuan Yu, Bournemouth University, UK Zerrin Yumak, Utrecht University, Netherlands Wenshu Zhang, Cardiff Metropolitan University Jian Zhang, Bournemouth University, UK Jianmin Zheng, Nanyang Technological University, Singapore

Sunken relief is an art form whereby the depicted shapes are sunk into a given flat plane with a shallow overall depth. In this paper, we propose an efficient sunken relief generation algorithm based on a single image by the technique of feature line enhancement. Our method starts from a single image. First, we smoothen the image with morphological operations such as opening and closing operations and extract the feature lines by comparing the values of adjacent pixels. Then we apply unsharp masking to sharpen the feature lines. After that, we enhance and smoothen the local information to obtain an image with less burrs and jaggies. Differential operations are applied to produce the perceptive relief-like images. Finally, we construct the sunken relief surface by triangularization which transforms two-dimensional information into a three-dimensional model. The experimental results demonstrate that our method is simple and efficient.

Traditional vision registration technologies require the design of precise markers or rich texture information captured from the video scenes, and the vision-based methods have high computational complexity while the hardware-based registration technologies lack accuracy. Therefore, in this paper, we propose a novel registration method that takes advantages of RGB-D camera to obtain the depth information in real-time, and a binocular system using the Time of Flight (ToF) camera and a commercial color camera is constructed to realize the three-dimensional registration technique. First, we calibrate the binocular system to get their position relationships. The systematic errors are fitted and corrected by the method of B-spline curve. In order to reduce the anomaly and random noise, an elimination algorithm and an improved bilateral filtering algorithm are proposed to optimize the depth map. For the real-time requirement of the system, it is further accelerated by parallel computing with CUDA. Then, the Camshift-based tracking algorithm is applied to capture the real object registered in the video stream. In addition, the position and orientation of the object are tracked according to the correspondence between the color image and the 3D data. Finally, some experiments are implemented and compared using our binocular system. Experimental results are shown to demonstrate the feasibility and effectiveness of our method.

Abstract This paper presents a novel approach for bas‐relief generation and synthesis. In contrast to previous methods, we divide this problem into two parts: the selection of the best view and arrangement of the relief layout. Taking these into account, we incorporate the visual saliency and photographic composition rules into the bas‐relief generation. Additionally, a nonlinear compression function is used to compress the models, and finally, we implement surface parameterization by directly manipulating the mesh triangles to generate curved surface bas‐relief. We validate our approach through a variety of models. The results indicate that the proposed approach is effective to adapt different types of target surface with topology unchanged. Comparing with conventional methods, our approach is able to effectively produce bas‐relief with a reasonable layout and distinct details.

Abstract We present an automatic approach for the semantic modeling of indoor scenes based on a single photograph, instead of relying on depth sensors. Without using handcrafted features, we guide indoor scene modeling with feature maps extracted by fully convolutional networks. Three parallel fully convolutional networks are adopted to generate object instance masks, a depth map, and an edge map of the room layout. Based on these high‐level features, support relationships between indoor objects can be efficiently inferred in a data‐driven manner. Constrained by the support context, a global‐to‐local model matching strategy is followed to retrieve the whole indoor scene. We demonstrate that the proposed method can efficiently retrieve indoor objects including situations where the objects are badly occluded. This approach enables efficient semantic‐based scene editing.

This paper aims to automate the process of generating a meaningful single still image from a temporal input of scene sequences. The success of our extraction relies on evaluating the optimal pose of characters selection, which should maximize the information conveyed. We define the information entropy of the still image candidates as the evaluation criteria.

Abstract This paper simulates the behaviour of collective transport where a group of ants transports an object in a cooperative fashion. Different from humans, the task coordination of collective transport, with ants, is not achieved by direct communication between group individuals, but through indirect information transmission via mechanical movements of the object. This paper proposes a stochastic probability model to model the decision‐making procedure of group individuals and trains a neural network via reinforcement learning to represent the force policy. Our method is scalable to different numbers of individuals and is adaptable to users' input, including transport trajectory, object shape, external intervention, etc. Our method can reproduce the characteristic strategies of ants, such as realign and reposition. The simulations show that with the strategy of reposition, the ants can avoid deadlock scenarios during the task of collective transport.

Abstract With the rapid development of human–computer interaction technologies, the new media generation demands novel learning experiences with natural interaction and immersive experience. Considering that digital storytelling is a powerful pedagogical tool for young children, in this paper, we design an immersive storytelling environment that allows multiple players to use naturally interactive hand gestures to manipulate virtual puppetry for assisting narration. A set of multimodal interaction techniques is presented for a hybrid user interface that integrates existing 3D visualization and interaction devices including head‐mounted displays and depth motion sensor. In this system, the young players could intuitively use hand gestures to manipulate virtual puppets to perform a story and interact with props in a virtual stereoscopic environment. We have conducted a user experiment with four young children for pedagogical evaluation, as well as system acceptability and interactivity evaluation by postgraduate students. The results show that our framework has great potential to stimulate learning abilities of young children through collaboration tasks. The stereoscopic head‐mounted display outperformed the traditional monoscopic display in a comparison between the two.

We introduce a unified particle framework which integrates the phase-field method with multi-material simulation to allow modeling of both liquids and solids, as well as phase transitions between them. A simple elasto-plastic model is used to capture the behavior of various kinds of solids, including deformable bodies, granular materials, and cohesive soils. States of matter or phases , particularly liquids and solids, are modeled using the non-conservative Allen-Cahn equation. In contrast, materials---made of different substances---are advected by the conservative Cahn-Hilliard equation. The distributions of phases and materials are represented by a phase variable and a concentration variable, respectively, allowing us to represent commonly observed fluid-solid interactions. Our multi-phase, multi-material system is governed by a unified Helmholtz free energy density. This framework provides the first method in computer graphics capable of modeling a continuous interface between phases. It is versatile and can be readily used in many scenarios that are challenging to simulate. Examples are provided to demonstrate the capabilities and effectiveness of this approach.

In this paper, we present a novel pairwise-force smoothed particle hydrodynamics (PF-SPH) model to enable simulation of various interactions at interfaces in real time. Realistic capture of interactions at interfaces is a challenging problem for SPH-based simulations, especially for scenarios involving multiple interactions at different interfaces. Our PF-SPH model can readily handle multiple types of interactions simultaneously in a single simulation; its basis is to use a larger support radius than that used in standard SPH. We adopt a novel anisotropic filtering term to further improve the performance of interaction forces. The proposed model is stable; furthermore, it avoids the particle clustering problem which commonly occurs at the free surface. We show how our model can be used to capture various interactions. We also consider the close connection between droplets and bubbles, and show how to animate bubbles rising in liquid as well as bubbles in air. Our method is versatile, physically plausible and easy-to-implement. Examples are provided to demonstrate the capabilities and effectiveness of our approach.

Digital techniques have been used to assist narrative and storytelling, especially in many pedagogical practices. With the rapid development of HCI techniques, saturated with digital media in their daily lives, young children, demands more interactive learning methods and meaningful immersive learning experiences. In this paper, we propose a novel hand gesture-based puppetry storytelling system which provides a more intuitive and natural human computer interaction method for young children to develop narrative ability in virtual story world. Depth motion sensing and hand gestures control technology is utilized in the implementation of user-friendly interaction. Young players could intuitively use hand gestures to manipulate virtual puppet to perform story and interact with different items in virtual environment to assist narration. Based on the result of the evaluation, this novel digital storytelling system shows positive pedagogical functions on children’s narrating ability as well as the competencies of cognitive and motor coordination. The usability of the system is preliminary examined in our test, and the results which showed that young children can benefit from playing with Puppet Narrator.

Eye movements have been shown to lead hand movements in tracing tasks where subjects have to move their fingers along a predefined trace. The question remained, whether the leading relationship was similar when tracing with a pointing device, such as a mouse; more importantly, whether tasks that required more or less gaze–mouse coordination would introduce variation in this pattern of behaviour, in terms of both spatial and temporal leading of gaze position to mouse movement. A three-level gaze–mouse coordination demand paradigm was developed to address these questions. A substantial dataset of 1350 trials was collected and analysed. The linear correlation of gaze–mouse movements, the statistical distribution of the lead time, as well as the lead distance between gaze and mouse cursor positions were all considered, and we proposed a new method to quantify lead time in gaze–mouse coordination. The results supported and extended previous empirical findings that gaze often led mouse movements. We found that the gaze–mouse coordination demands of the task were positively correlated to the gaze lead, both spatially and temporally. However, the mouse movements were synchronised with or led gaze in the simple straight line condition, which demanded the least gaze–mouse coordination.

Abstract Estimation of 3D body shapes from dressed‐human photos is an important but challenging problem in virtual fitting. We propose a novel automatic framework to efficiently estimate 3D body shapes under clothes. We construct a database of 3D naked and dressed body pairs, based on which we learn how to predict 3D positions of body landmarks (which further constrain a parametric human body model) automatically according to dressed‐human silhouettes. Critical vertices are selected on 3D registered human bodies as landmarks to represent body shapes, so as to avoid the time‐consuming vertices correspondences finding process for parametric body reconstruction. Our method can estimate 3D body shapes from dressed‐human silhouettes within 4 seconds, while the fastest method reported previously need 1 minute. In addition, our estimation error is within the size tolerance for clothing industry. We dress 6042 naked bodies with 3 sets of common clothes by physically based cloth simulation technique. To the best of our knowledge, We are the first to construct such a database containing 3D naked and dressed body pairs and our database may contribute to the areas of human body shapes estimation and cloth simulation.

To better capture the shapes as well as the rich dynamics of hair, image based modelling techniques have been developed for reconstructing their 3D geometry and important visual features. Most hair images contain inevitable noises which impair reconstructed hair models. Therefore we propose to pre-process hair images and provide an orientation map of hair strands to enhance the follow-on modelling. To demonstrate the usage of pre-processing techniques, we apply our pre-processing results for bas-relief stylisation and modelling of hair from image inputs. We compare different techniques to estimate hair orientations, adopting four types of filter mechanisms. Our analysis of their performance sheds insight on designing a suitable pre-processing technique for hair reconstruction from images. Several examples of bas-relief creation validate the effectiveness of the proposed approach.

As an important autumn feature, scenes with large numbers of falling leaves are common in movies and games. However, it is a challenge for computer graphics to simulate such scenes in an authentic and efficient manner. This paper proposes a GPU based approach for simulating the falling motion of many leaves in real time. Firstly, we use a motionsynthesis based method to analyze the falling motion of the leaves, which enables us to describe complex falling trajectories using low-dimensional features. Secondly, we transmit a primitive-motion trajectory dataset together with the low-dimensional features of the falling leaves to video memory, allowing us to execute the appropriate calculations on the GPU.

Abstract Puppetry has been a popular art form for many centuries in different cultures, which becomes a valuable and fascinating heritage assert. Traditional Chinese marionette art with over 2000 years history is one of the most representative forms offering a mixture of stage performance of singing, dancing, music, poetry, opera, story narrative and action. Apart from a set of string rules, which controls the dynamics, head carving skill is another important pillar in this art form. This paper addresses the heritage preservation of the marionette head carving by digitalizing the head models with a novel modelling technique using ordinary differential equations (ODEs). The technique has been specially tailored to suit the modelling complexity and the need of accurate description of shapes. It offers smoothly sewing ODE swept patches to represent the distinct features of a marionette head with sharp variance of local geometry. Such features otherwise are difficult to model and capture accurately, which may require a great effort and tedious handcrafting of an experienced modeller, when using other representation forms like polygons. Copyright © 2015 John Wiley & Sons, Ltd.

Multiple-fluid interaction is an interesting and common visual phenomenon we often observe. In this paper, we present an energy-based Lagrangian method that expands the capability of existing multiple-fluid methods to handle various phenomena, such as extraction, partial dissolution, etc. Based on our user-adjusted Helmholtz free energy functions, the simulated fluid evolves from high-energy states to low-energy states, allowing flexible capture of various mixing and unmixing processes. We also extend the original Cahn-Hilliard equation to be better able to simulate complex fluid-fluid interaction and rich visual phenomena such as motion-related mixing and position based pattern. Our approach is easily integrated with existing state-of-the-art smooth particle hydrodynamic (SPH) solvers and can be further implemented on top of the position based dynamics (PBD) method, improving the stability and incompressibility of the fluid during Lagrangian simulation under large time steps. Performance analysis shows that our method is at least 4 times faster than the state-of-the-art multiple-fluid method. Examples are provided to demonstrate the new capability and effectiveness of our approach.

Abstract Natural‐looking insect animation is very difficult to simulate. The fast movement and small scale of insects often challenge the standard motion capture techniques. As for the manual key‐framing or physics‐driven methods, significant amounts of time and efforts are necessary due to the delicate structure of the insect, which prevents practical applications. In this paper, we address this challenge by presenting a two‐level control framework to efficiently automate the modeling and authoring of insects’ locomotion. On the top level, we design a Triangle Placement Engine to automatically determine the location and orientation of insects’ foot contacts, given the user‐defined trajectory and settings, including speed, load, path and terrain etc. On the low‐level, we relate the Central Pattern Generator to the triangle profiles with the assistance of a Controller Look‐Up Table to fast simulate the physically‐based movement of insects. With our approach, animators can directly author insects’ behavior among a wide range of locomotion repertoire, including walking along a specified path or on an uneven terrain, dynamically adjusting to external perturbations and collectively transporting prey back to the nest.

Abstract In this paper, we present an overview of the achievements accomplished to date in the field of computer‐aided relief generation. We delineate the problem, classify different solutions, analyse similarities, investigate developments and review the approaches according to their particular relative strengths and weaknesses. Moreover, we describe remaining challenges and point out prospective extensions. In consequence, this survey is addressed to both researchers and artists, through providing valuable insights into the theory behind the different concepts in this field and augmenting the options available among the methods presented with regard to practical application.

Abstract The Cosserat beam model is traditionally used to describe the mechanics of a flexible beam, which is a one‐dimensional entity. In this paper we apply the Cosserat beam model to an arbitrary three‐dimensional object for fast simulation of its vibration behaviours. We encage a detailed mesh model of an object inside a much simpler supporting framework whose edges are effectively the beams (struts) of the Cosserat model. The paper focuses first on extracting the modes of vibration of the framework. Once this is done, dynamic deformations can then be quickly simulated with respect to any applied constraints and dynamic stimuli. The aim of this method is to offer realism traditionally only afforded by the finite element methods (FEMs) while providing more sophistication than the mass–spring method. Copyright © 2007 John Wiley & Sons, Ltd.

Mesh-free deformation is an effective method for the simulation of deformable objects and characters in computer animation. It bears the merits of flexible control, good accuracy and easy implementation. Similar to other physically-based deformation techniques, however, computational cost remains a pressing issue, especially for large scale problems. Based on our previous work, in this paper we investigate the underlying structure of the numerical approach in order to speed up the computation. Three algorithms are presented and their efficiency and convergence are analysed. Our results show that we are able to significantly reduce the computation time and memory usage with little loss in accuracy. The new technique is capable of achieving interactive frame rates with large models.

Abstract Existing physically based deformation techniques, such as the finite element method (FEM) and the mass‐spring systems (MSS), require the deformed object to be properly meshed. This is arguably the most expensive manual intervention process. In this paper, we propose a mesh‐free deformation technique where only unconnected points are involved. The idea is to develop an approximate analytical solution using the Kelvin solution. Due to the fact that no mesh is involved, deforming a complex shape is as straightforward as deforming a simple one. Furthermore, the trade‐off between efficiency and accuracy is easy to achieve by redistributing the points concerned. Experiments show that this method is fast and offers similar accuracy to the FEM. Copyright © 2004 John Wiley & Sons, Ltd.