Background 3D botanical tree reconstruction from a single image plays a vital role in the field of computer graphics. However, accurately capturing the intricate branching patterns and detailed morphologies of trees remains a challenge. Methods In this study, we proposed a novel approach for single-image tree reconstruction using a conditional generative adversarial network to infer the 3D skeleton of a tree in the form of a 2D skeleton depth map. Based on the 2D skeleton depth map, a corresponding branching structure (3D skeleton) that inherits the tree shape in the input image and leaves can be generated using a procedural modeling technique. Result Experimental results show that the proposed method accurately reconstructs diverse tree structures across species. Both quantitative and qualitative evaluations demonstrate improved skeleton completeness, branching accuracy, and visual realism over baseline methods, while requiring no user input. Conclusions Our proposed approach for generating lifelike 3D tree models from a single image with no user input shows its proficiency in achieving efficient and reliable reconstruction. These results showcase the capability of the proposed model to recreate complex tree architectures while capturing their visual authenticity.

ABSTRACT The spectrum‐based method, usually referred to as the Fourier approach, is considered to be an efficient solution for synthesizing and visualizing realistic ocean waves, and is widely used in films and video games. It is based on a spectra function describing the distribution relationship between the wave energies and various individual waves of different frequencies. However, the spectrum method is based on the assumption that the sea surface is homogeneous everywhere, and cannot reproduce the various inhomogeneous real‐world sea conditions for maritime simulators. This paper proposes a new model for ocean wave simulation which combines the spectral methods and wave‐particle theory to result in a flexible and physically accurate model for ocean wave simulation. In the proposed model, input environmental parameters can dynamically affect the shape of the spectrum, then control the properties of the wave particles by the spectral function, thus achieving local control of waves instead of global control. Through experiments, we verify that the proposed model can capture the spatially varying characteristics and reproduce real sea state. Experimental results also show that our model has considerable physical accuracy and can achieve good compliance with the statistical characteristics of actual sea.

Abstract More and more applications require accurate estimation of objects' poses. For example, Augmented Reality (AR) needs that for interactions among virtual and real objects. However, recent object‐level Simultaneous Localization and Mapping (SLAM) systems care more about trajectories than poses of objects. So, in this paper, we present SSG SLAM: a monocular SLAM system based on Spatial Semantic Graph (SSG), leveraging spatial relations to achieve good estimation of objects' poses. First, we put forward the design of SSG to organize objects and their spatial relations. Then, SSG is utilized in a monocular object SLAM and new constraints are proposed based on spatial relations. Finally, experimental evaluations performed on SSG‐Dataset show that our approach outperforms the baseline object SLAM system, as well as providing visual consistency in an AR demo.

We present a speaker-independent isolated word recognition approach with audio derivation and convolutional neural network(CNN) in this paper. In contrast with traditional sophisticated phonetic-based features extracted from audio, we utilize the spectrogram of audio as training data for convolutional neural network which transforms the isolated word recognition problem into the image recognition problem. Deep learning has high demands of training data, but it will reduce efficiency of the system to make such corpora. We present an audio-level data derivation approach, which makes it possible to obtain high recognition rate with a small number of audio seed data collected. It is achieved by formant perturbation, pitch shifting, time stretching and volume perturbation while maintaining semantic content. The approach presented in this paper reduces seed data amount demand of deep learning in isolated word recognition. Results show that accuracy improvement is significant with derived data and only 7.57%-15.14% of seed data is needed to achieve the same level accuracy.

We present MagicToon, an interactive modeling system with mobile augmented reality (AR) that allows children to build 3D cartoon scenes creatively from their own 2D cartoon drawings on paper. Our system consists of two major components: an automatic 2D-to-3D cartoon model creator and an interactive model editor to construct more complicated AR scenes. The model creator can generate textured 3D cartoon models according to 2D drawings automatically and overlay them on the real world, bringing life to flat cartoon drawings. With our interactive model editor, the user can perform several optional operations on 3D models such as copying and animating in AR context through a touchscreen of a handheld device. The user can also author more complicated AR scenes by placing multiple registered drawings simultaneously. The results of our user study have shown that our system is easier to use compared with traditional sketch-based modeling systems and can give more play to children's innovations compared with AR coloring books.

We introduce Pmomo (acronym of projection mapping on movable object), a dynamic projection mapping system that tracks the 6-DOF position of real-world object, and shades it with virtual 3D contents by projection. The system can precisely lock the projection on the moving object in real-time, even the one with complex geometry. Based on depth camera, we developed a novel and robust tracking method that samples the structure of the object into low-density point cloud, then performs an adaptive searching scheme for the registration procedure. As a fully interactive system, our method can handle both internal and external complex occlusions, and can quickly track back the object even when losing track. In order to further improve the realism of the projected virtual textures, our system innovatively culls occlusions away from projection, which is achieved by a facet-covering method. As a result, the Pmomo system enables the possibility of new interactive Augmented Reality applications that require high-quality dynamic projection effect.

Abstract We propose a personality trait exaggeration system emphasizing the impression of human face in images, based on multi‐level features learning and exaggeration. These features are called Personality Trait Model (PTM). Abstract level of PTM is social psychology trait of face perception such as amiable, mean, cute and so on. Concrete level of PTM is shape feature and texture feature. A training phase is presented to learn multi‐level features of faces from different images. Statistical survey is taken to label sample images with people's first impressions. From images with the same labels, we capture not only shape features but also texture features to enhance exaggeration effect. Texture feature is expressed by matrix to reflect depth of facial organs, wrinkles and so on. In application phase, original images will be exaggerated using PTM iteratively. And exaggeration rate for each iteration is constrained to keep likeness with the original face. Experimental results demonstrate that our system can emphasize chosen social psychology traits effectively.

In this paper, we present a key point recognition scheme, which consists of a novel feature detector and an efficient descriptor. Inspired by FAST (features from accelerated segment test), our feature detector is easy to compute and has high repeatability. Scale-invariance and optimized robustness are gained by extending traditional FAST to scale space.We combine this detector with an adapted version of SURF (speed up robust features) descriptor, providing the system with all means to do feature matching and object detection. Experimental evaluation and comparison with standard SURF using Hessian matrix-based detector are included in this paper, showing improvement in speed with comparable robustness.

Visibility precomputation has become essential for PRT-based rendering involving soft shadows. While graphic hardware rasterization has been increasingly used to compute visibility information for a mass of precomputation tasks, the linear feature of rasterization makes it unadaptive to sample visibility in a coordinates system, to which the Cartesian coordinates are non-linearly mapped. With the emergence of Geometry Shader in the framework of DirectX 10, this problem can be solved. In this paper, we present an efficient solution for visibility sampling on the spherical polar coordinates in the precomputation stage of PRT by means of the Geometry Shader within the DirectX 10 pipelines. In the mean time, a spherical reparameterization scheme is introduced for shadow rendering, under which spherical functions as lighting, BRDF and visibility are mapped onto the spherical polar coordinates.

This paper presents an approach that can relight captured image with real incident light source. We first captured basis images of objects illuminated with a projector as the basis light source. We then established a pixel level mapping relationship between a real incident light source and the basis light source. Finally we developed a relighting algorithm that can simulate the relit effects with the real light source. The techniques can be applied to mixed reality.

Radiometric compensation has made it possible for a projector to display on ordinary surface with colors and textures. For previous methods, itpsilas necessary to calibrate both the projector and camera at first. The calibration can be time-consuming and needs to be redone once the system settings change. We present a method that simplifies the calibration process. As a result, the system is more practicable for ad-hoc setups.

This poster describes our recently developed real-time projector-camera system for interaction with Chinese Ink-and-Wash Cartoons. We implement a real-time interactive water simulation under the acceleration of GPU, together with a sensing subsystem using computer vision techniques. Combined with Chinese stylized fish rendered in process, the system provides real-time interactions with traditional Chinese paintings. By stirring up the still water, fish and other essential elements of Chinese paintings, we hope to present new interaction techniques and more lively Chinese painting sceneries compared to those in traditional static settings.

This paper presents a real-time rendering system for generating a Chinese ink-and-wash cartoon. The objective is to free the animators from laboriously designing traditional Chinese painting appearance. The system constitutes a morphing animation framework and a rendering process. The whole rendering process is based on Graphic Process Unit (GPU), including interior shading, silhouette extracting and background shading. Moreover, the morphing framework is created to automatically generate Chinese painting cartoon from a set of surface mesh models. These techniques can be applied to real-time Chinese style entertainment application.

This paper proposes a framework named TIPMR, for designing tangible user interfaces (TUIs) within projector-based mixed reality applications. The framework divides the target application into three parts: GUI-based application, TUI and an assistant. The assistant is employed as an adapter to translate between TUI operations and general GUI commands like mouse or keyboard events. This architecture makes it easier to focus on designing GUI-based applications and TUI separately. We built a tourist guidance system with two different tangible interaction modes based on TIPMR to demonstrate its usefulness and efficiency.

A similarity based image inpainting method is introduced in this paper, which can be used to automatically fill the regions needing to be erased by using the surrounding information in the image. The method is based on analyzing the previous image inpainting methods, especially improving the method proposed by Criminisi et al. In our method, a more robust method is adopted to compute the filling order, which overcome the problem arising when the image gradients of some pixels on the source region contour are just zeros. And a new mapping term is proposed in computing the matching cost to penalize the source regions repeatedly sampled, which practically avoid the artificial repetition problem in the inpainted results.

Motion editing is an efficient tool for creating human animation. In this paper, we present a new motion editing method based on intermediate model. Our approach can clone motion capture data to a new path. Instead of editing original motions with full DOFs, we firstly reduce the DOFs of the original motion and produce intermediate motion; then the intermediate motion is edited and cloned onto new paths; finally, the reduced DOFs are added back to the edited motions. Experimental results and analysis are given in the end