Abstract This study examines how science fiction films and real-world interaction with a humanoid social robot shape teenagers’ emotional responses, technology acceptance, and expectations of AI robots. To address the lack of comparative research on mediated and embodied experiences in adolescents, we conducted a three-phase study with 55 teenagers using questionnaires and thematic analysis. Quantitative results show that while films increased nervousness and reduced comfort, direct interaction moderated these effects without significantly changing perceptions of technology acceptance. Qualitatively, teenagers primarily imagined robots in practical roles, though some expressed skepticism after exposure. These insights provide a foundation for designing adolescent-centered robots and developing targeted media literacy initiatives to promote realistic perceptions of AI robots among teenagers.

This issue contains 10 papers. In the first paper, Diego Mazala and Claudio Esperança, from Universidade Federal do Rio de Janeiro in Brazil, and Ricardo Marroquim, from Delft University of Technology in Netherlands, present a novel method for blending human faces in order to create a new one. In a nutshell, their proposal uses Laplacian smoothing to segregate layers of details from one or more faces, which are then integrated into a base face with the help of an interactive and visual editor. In particular, their method supports blending multiple faces and multiple subregions in those faces. Since their approach is intuitive and relatively easy to implement, it can be integrated into artistic pipelines aiming at designing human face models from preexisting ones. In the second paper, Ulas Gulec, from TED University Ankara, Turkey, Ilkin Sevgi Isler, from University of Central Florida in Orlando, USA, Mehmet Harun Doganay and Muruvet Gokcen, from Milsoft Software Technologies, in Ankara, Turkey, Mert Ali Gozcu, from Simsoft Information Technologies in Ankara, Turkey, and Merve Denizci Nazligul, from Yeditepe University in Istanbul, Turkey, propose a study that aims to increase the motivation levels of powerlifters during training sessions by developing a virtual competition environment. In this environment, the athletes experience a virtual competition environment by using HTC Vive. To understand the efficiency of the virtual environment, it was tested with 32 professional athletes. The findings illustrated that using VR technology was beneficial to increase the level of motivation of powerlifters during training sessions. In the third paper, Jinyu Li, Xin Zhou, Bangbang Yang, Guofeng Zhang, Xun Wang, and Hujun Bao, from Zhejiang University in Hangzhou, China, propose RLP-VIO—a robust and lightweight monocular visual-inertial odometry system using multiplane priors. With planes extracted from the point cloud, visual-inertial-plane PnP uses the plane information for fast localization. For sensor fusion, their sliding-window optimization uses a novel structureless plane-distance error cost, which prevents the fill-in effect that poisons the BA problem's sparsity and permits the use of a smaller sliding window while maintaining good accuracy. The total computational cost is further reduced with their modified marginalization strategy. To further improve the tracking robustness, the landmark depths are constrained using the planes during degenerated motion. The whole system is parallelized with a three-stage pipeline. Their system achieves competitive accuracy and works robustly even on long and challenging sequences. In the fourth paper, Ruizhe Li, Ryo Oji, and Issei Fujishiro, from Keio University in Yokohama, Japan, present an anime-like character customization system, where each customizing parameter can adjust the shape or color for the corresponding part of the character model. Based on this system, the authors propose an improved approach for generating a rich variety of 3D anime-like NPCs including body models and clothing items in different styles. They introduce a neural network to control the facial appearances, Gaussian mixture models to control the colors of hair and clothes, and a Bayesian network to control the outfits of clothing items. They demonstrate the proposed approach can maintain variety and stability for generated characters. The fifth paper by Cheng Shang, Hongke Zhao, Meili Wang, Xiao Long Wang, Yu Jiang, and Qiang Gao, from Northwest Agriculture and Forestry University in Yangling, China, focuses on the identification of cashmere goats with similar characteristics. First, the single shot detection network was used to process the data set. Next, the authors innovatively proposed the multibranch fusion optimization structure of triplet loss function and Label Smoothing CrossEntropy Loss function, as well as they added a small number of images of 24 different breeds of sheep to each cashmere goat dataset with different ID to promote the distance between training individuals, and then used the trained model to find the number of goats with the lowest recognition accuracy. Unlike previous studies using the Cycle-GAN, the authors took the novel approach of using this network to learn and combine the features seen in photos of cashmere goats. Since the learned features were all observed in the same goats, this method achieved better results in learning the features of the goats. In the sixth paper, Jia Chen, Haidongqing Yuan, Yi Zhang, Ruhan He, and Jinxing Liang, from Wuhan Textile University in China, propose a fashion image retrieval framework based on dilated convolutional residual network that consists of two major parts, image feature extraction and feature distance measurement. For image feature extraction, they first extract the shallow features of the input image by a multiscale convolutional network, and then develop a novel dilated convolutional residual network to obtain the deep features of the image. Finally, the extracted features are transformed into high-dimensional features vector by a binary retrieval vector module. For feature distance measurement, the authors first use PCA to reduce the dimension of the extracted high-dimensional vectors. Then they propose a mixed distance measurement algorithm combined with cosine distance and Mahalanobis distance to calculate the spatial distance of the feature vectors for similarity ranking. In the seventh paper, Yi-Jheng Huang, from Yuan Ze University in Taoyuan, Taiwan, proposes an algorithm for detecting edges based on the color of a mesh surface. His approach is based on the data structure of a quad mesh, which makes the data structure of 3D meshes resemble the data structure of images. As a result, image-processing methods can be applied on the 3D meshes. In this paper, six classical edge detection filters are implemented on the 3D meshes. The experimental results demonstrate that his method can identify areas of high color gradient on 3D meshes. A comparison with two other methods for detecting color boundaries on 3D meshes reveals that his method is more effective at detecting boundaries. Lastly, he proposes two novel applications that utilize the information of color boundaries on a 3D mesh surface. In the eighth paper, Xue Du, Juan Xiu Sun, Kunpeng Wang, Junlong Yang, and Jiang Chuan Wang, from Shandong University of Science and Technology in China, propose an underwater image enhancement method based on entropy weight fusion for underwater images. First, white balance processing can effectively correct the blue (green) color appearance of the image. Then the white-balanced images are converted from RGB space to LAB space, and L channel is processed with improved adaptive gamma correction, and then converted back to RGB space. CLAHE and bilateral filtering are performed in RGB space. The RGB space is converted to HSV space, the V channel is processed by single-scale Retinex algorithm combined with guided filtering, stretching the R channel with s-cosine curve, and then converted back to RGB space. Finally, the three results are fused by entropy weight to obtain the final enhanced image. Experimental results show that the proposed algorithm can improve the contrast and clarity of underwater images, and effectively remove color cast. In the ninth paper, Ana Agić, Lidija Mandić, and Lea Skorin-Kapov, from University of Zagreb in Croatia, report on the results of a user study aimed to investigate and compare three types of locomotion techniques in VR in terms of their impact on cybersickness. The different locomotion techniques are tested in two different contrast scene settings, daytime and nighttime, to further explore the potential impact of scene contrast adjustments. For the evaluation of cybersickness, the authors used a questionnaire to obtain subjective ratings, and heart rate monitoring as an objective metric. Results show that a linear movement locomotion technique provokes the highest level of cybersickness, and that women have a higher heart rate as compared with men when navigating and interacting in a VR scene. Regarding the influence of scene contrast, results showed that scenes with daylight were better suited to participants in almost all tested scenarios. In addition to reported findings related to locomotion techniques and the impact on cybersickness, they highlight that a key contribution is the utilized test methodology. The last paper by Zafar Masood, Jiangbin Zheng, Muhammad Irfan, and Idrees Ahmad, from Northwestern Polytechnical University in Xi'an, China, presents a novel method for a high-performance large-scale terrain rendering for high-fidelity display systems using game engine. The proposed method performs patch-based hierarchical culling of a multiresolution terrain model to reduce rendering load. A view-based algorithm simplifies the patches with error control on GPU. Simplified patches are efficiently submitted for drawing using indirect mesh instancing feature of game engine. The method utilizes hardware tessellation feature for high-performance model tessellation and accurate Earth's surface construction using displacement mapping. The proposed method is evaluated by rendering scenes for high-quality output on consumer-level hardware. Flights are performed with various settings and results are compared with clipmap-based and state-of-the-art hardware tessellation based adaptive methods. The method achieved 750, 575, and 540 frames-per-second (fps) for HD, full-HD, and ultra-HD display resolutions.

VRLAB

This issue contains 12 regular papers. In the first paper, Hong Li et al. present an animation translation method based on edge enhancement and coordinate attention, which is called FAEC-GAN. They design a novel edge discrimination network to identify the edge features of images, so that the generated anime images can present clear and coherent lines. And the coordinate attention module is introduced in the encoder to adapt the model to the geometric changes in translation, to produce more realistic animation images. In addition, the method combines the focal frequency loss and pixel loss, which can pay attention to both the frequency domain information and pixel information of the generated image to improve the visual effect of the image. In the second paper, Rahul Jain et al. propose an algorithm to convert a depth video into a single dynamic image known as a linked motion image (LMI). The LMI has been given to a classifier consisting of an ensemble of three modified pre-trained convolutional neural networks (CNNs). The experiments were conducted using two datasets: a multimodal large-scale EgoGesture dataset and The MSR Gesture 3D dataset. For the EgoGesture dataset, the proposed method achieved an accuracy of 92.91%, which is better than the state-of-the-art methods. For the MSR Gesture 3D dataset, the proposed method accuracy is 100%, which outperforms the state-of-the-art methods. The recognition accuracy and precision of each gesture are also highlighted in this work. In the third paper, Rustam Akhunov et al. propose a set of experiments to aid the evaluation of the main categories of fluid-boundary interactions that are important in computer animation, i.e. no motion (resting) fluid, tangential and normal motion of a fluid with respect to the boundary, and a fluid impacting a corner. They propose 10 experiments, comprising experimental setup and quantitative evaluation with optional visual inspections, that are arranged in four groups which focus on one of the main category of fluid-boundary interactions. The authors use these experiments to evaluate three particle-based boundary handling methods, that is, Pressure Mirroring (PM), Pressure Boundaries (PB) and Moving Least Squares Pressure Extrapolation (MLS), in combination with two incompressible SPH fluid simulation methods, namely IISPH and DFSPH. In the fourth paper, Shenghuan Zhao et al. present three Extended Reality (XR) apps (AR, MR, and VR) to interactively visualize façade fenestration geometries and indoor illuminance simulations. Then XR technologies are assessed by 120 students and young architects, from task performance and engagement level two aspects. The task performance is measured by correct rate and time consumption two indicators, while the engagement level is measured by usability and interest two indicators. Evaluation results show that compared to AR and VR, MR is the best XR technology for this aim. VR outperforms AR on three indicators except for usability. By exposing three different XR technologies' performances in aiding fenestration design, this study increases the practical value of applying XR to the building design field. The fifth paper by Jing Zhao et al. focuses on a multiple-fluid coupling simulation algorithm based on MPM and PFM. First, based on the MPM, they model multiphase flow on Eulerian grids and capture the sharp interfaces between immiscible fluids combined with the PFM. The gas phase is further treated as a fluid during the gas–liquid interaction. Second, to demonstrate the natural fluid moving evolution from the high energy state to the low energy state, the paper proposes the local minimize bulk energy function to control the low energy state. Finally, the paper designs and achieves various groups of multiple-fluid coupling comparison experiments. Experimental results showed that the proposed approach can simulate various effects of rapid diffusion in the multiple-fluid coupling, such as complete dissolution, mutual solubility, extraction, and other phenomena. In the sixth paper, Jiwei Zhang et al. propose a novel method fusing multiple heterogeneous features through a multi-feature subspace representation network (MFSRN) to maximize the classification performance while keeping the disparity among features as small as possible, that is, common-subspace constraints. The authors conducted comparative experiments with state-of-the-art models on the bird's-eye view person dataset, and extensive experimental results demonstrated that the proposed MFSRN could achieve better recognition performance. Furthermore, the validity and stability of the method are confirmed. In the seventh paper, Sahadeb Shit et al. propose a convolutional neural network (CNN)-based image dehazing and detection approach, called End to End Dehaze and Detection Network (EDD-N), for proper image visualization and detection. This network is trained on real-time hazy images that are directly used to recover dehaze images without a transmission map. EDD-N is robust, and accuracy is higher than any other proposed model. The authors also conducted extensive experiments using real-time foggy images. The quantitative and qualitative evaluations of the hazy dataset verify the proposed method's superiority over other dehazing methods. Moreover, the proposed method validated real-time object detection tasks in adverse weather conditions and improved the intelligent transportation system. In the eighth paper, Chaehan So et al. designed a virtual being from a deep learning-generated face and a conversational AI model acting as a virtual conversation partner in an online conferencing software and evaluated it in 11 perceptions of social attributes. Compared to prior expectations, participants perceived the virtual being as distinctly higher in warmth (engaging, empathic, and approachable) but lower in realism and credibility after 5 days of 10-min daily conversations (Study 1). Further, the authors explored the idea of simplifying the technical setup to reduce the technical entry barrier for such experiments (Study 2). To this aim, they conducted several trials of fine-tuning a small conversational model of 90 million parameters until its performance metrics improved. Testing this fine-tuned model with users revealed that this model was not perceived differently from a large conversational model. In the ninth paper, Di Qi et al. propose a novel split and join approach to simulate a side-to-side stapled intestinal anastomosis in virtual reality. They mimic the intestine model using a new hybrid representation—a grid-linked particles model for physics simulation and a surface mesh for rendering. The proposed split and join operations handle the updates of both the grid-linked particles model and the surface mesh during the anastomosis procedure. The simulation results demonstrate the feasibility of the proposed approach in simulating intestine models and the side-to-side anastomosis operation. The tenth paper by Lanfeng Zhou et al. present a novel graph convolution mixed with point cloud deep learning method. In this method, the skinned multi-person linear model is regarded as a graph structure input, and the coarsened graph is obtained by graph convolution. After feeding the coarsened graph into the PointNet network, the coordinates of Dazhui are output. Different from the existing methods, the proposed method can directly label the results on the adaptive model, thus improving the accuracy on different models. An optimization method based on graph structure is introduced for better fit the predicted acupoints to the surface. In addition, a dataset marked with Dazhui is constructed for training. Experiments show that the accuracy of positioning could meet the requirements of needle application under certain circumstances. In the eleventh paper, Jian Lu et al. propose to reduce the interference of the factors in skeleton-based action recognition, by taking the joint coordinate modal information of 2D skeleton to represent the change of human posture as the research point: first, the joint coordinates are obtained from RGB video or image using a detector. Then the feature extraction network is combined to perform multi-level feature learning to establish correspondence between actions and corresponding multi-level features. Finally, the hierarchical attention mechanism is introduced to design the CHAN model. By calculating the association between elements, the weight of the action classification is redistributed. The proposed method has good performance on UT-Kinect, KTH and NTU RGB + D datasets. The last paper by Numan Ali et al. first conducted a subjective study with field experts to investigate about the practical implementation of their existing virtual chemistry laboratory (VCL). To consider the suggestions of the field experts, they propose task specific aids based virtual reality chemistry laboratory (TSA-VRCL) to minimize students' cognitive load and enhance their performance. The task specific aids consist of an arrow, animation and audio aids that are separately rendered with each step of the experimental tasks. During evaluations, 80 students performed the experiments in four different groups using four different experimental conditions. Evaluations revealed that the proposed TSA-VRCL minimizes students' cognitive load and enhances their performance. Professor Magnenat Thalmann started her career in Canada at the University of Montreal where she pioneered the field of 3D Virtual Humans. In 1989, she created the laboratory MIRALab at the University of Geneva, a ground-breaking research lab developing Virtual Humans and Social Robots. From 2019 to 2022, she was the Director of the research Center Being there and the Research Institute IMI in Nanyang Technological University (NTU) in Singapore. In NTU, Singapore, she revolutionized social robotics by unveiling the first social robot Nadine that can show mood and emotions and remember people and actions. Besides having bachelor's and master's degrees in disciplines such as psychology, biology, chemistry and computer science, Professor Thalmann completed her PhD in quantum physics at the University of Geneva. She has received honorary doctorates from Leibniz University of Hannover and the University of Ottawa in Canada and several prestigious other Awards as the Humboldt Research Award in Germany, the Eurographics Career Award and the Canadian Human Computer Communications Society Award. She is a life Member of the Swiss Academy of Engineering Sciences. For more information about her academic achievements, see google scholar. Prof. Daniel Thalmann is a Swiss and Canadian computer scientist. He is currently an honorary professor at the EPFL in Switzerland and the executive director of R&D at MIRALab Sarl. He is co-editor-in-chief of Wiley's Journal of Computer Animation and Virtual Worlds (CAVW) and on the editorial boards of several other journals. Daniel Thalmann is programme chair and co-chair of CASA2023 and CGI2023. After receiving his Ph.D. in Computer Science from the University of Geneva in 1977, Daniel Thalmann began at the University of Montreal in Canada. He later became a professor at EPFL, Switzerland, where he founded the Virtual Reality Lab (VRlab). From 2009 to 2017, he was a visiting professor at the Nanyang Technological University in Singapore. Throughout his successful career, Professor Daniel Thalmann has received many awards, including an Honorary Doctorate from Paul Sabatier University in Toulouse, France in 2003, the Eurographics Distinguished Career Award in 2010, the Canadian Human Computer Communications Society Achievement Award in 2012, and the CGI Career Achievement Award in 2015. More can be found on Daniel Thalmannin Wikipedia.

Estimating personality traits of a person through visual or multimodal signals has attracted increasing attention in cognitive multimodal interfaces and human factors in XR. Existing methods place a great emphasis on individual’s facial features and use these features to predict the apparent personality, but overlook the importance of environment and real personality. In this paper, we propose a deep learning approach to predict both real and apparent personality based on pure visual information in dyadic human-human interaction scenarios. We use the nonverbal information of both the target person and the interlocutor to learn their body and facial representations through a multi-branch ResNet-Attention network, and output real and apparent personality prediction in the form of five-dimensional personality traits (openness, conscientiousness, extraversion, agreeableness, and neuroticism). We conduct experiments to evaluate the proposed method. The experimental results show that the proposed method achieves good performance for both real and appearance personality prediction.

Abstract Intangible heritage is often linked to human actions and performances. The use of digital humans (DHs) for its digital representation and therefore its preservation, allows reframing the way to transmit and deal with content that is difficult to visualize. To that end, the digital human becomes an important element establishing the connection between the action, the objects, the knowledge, and the environment. In this article, we describe the development of DHs acting as practitioners and storytellers for traditional craft processes within virtual environments. We present the process and the tasks involved in modeling, designing, and animating DHs, detailing the underlying technological background. Animations were completely based on real humans' motion extraction while working on the corresponding craft. As a result, we present the different DHs models created for three specific heritage crafts: mastic cultivation, glass blowing, and silk weaving as well as an AR application, built to augment exhibits of the Chios Mastic Museum. This article is a practical description of the steps to model and animate virtual humans. The work aims to bring a methodology for achieving DHs creation for CH applications.

This issue contains five papers. In the first paper, Yuling Yan, Lijun Zhang, from Suzhou University, China and Minye Chen, from Shanghai University of Science and Technology, all in China propose a virtual training system of aircraft maintenance based on gesture recognition interaction. Leap Motion is used as a sensor to construct a hybrid machine learning gesture recognition model, to obtain natural human–computer interaction experience. In the recognition model, the initial weight matrix, and the number of hidden layer nodes in the BPNN are jointly optimized by the PSO algorithm with self-adaption inertial weight. This optimization algorithm achieved a recognition rate of 81.26% in the dynamic gesture database constructed in this paper, which is higher than other available algorithms. A preliminary usability evaluation in university classrooms shows that the teaching system in this paper can achieve a better interactive experience. In the second paper, Junsong Zhang, Zhu Shaoqiang, Kunxiang Liu, and Xiaoyu Liu, from National Engineering Research Center for E-learning, in Wuhan, China, propose a novel adversarial architecture for multiple sketch colorization which is a scribble-based, automatic and exemplar-based colorization method. The proposed framework has two stages, namely imitating stage, and shading stage. In the imitating stage, to address the challenge of lack of texture in the sketch, the authors train a grayscale generation network to accomplish a mapping task, namely generating a grayscale map with textured, grayscale, boundary information from the input sparse sketch. In the shading stage, the model can accurately colorize the objects in the gray image generated in the previous stage and generate high-quality colorized images. With the proposed model trained on their database, the experimental results show that their method can generate vivid colorized images and achieve a better performance than previous methods evaluated by FID metric. In the third paper, Assia MESSACI, Zenati Nadia, Belhocine Mahmoud, from CDTA, Algiers, Algeria and Otmane Samir, from Université Evry, Université Paris-Saclay, France propose the Zoom-fwd, which is an efficient 3D interaction technique. The proposed technique uses gesture recognition for different 3D interaction tasks like selection and manipulation. This new approach allows an efficient interaction with distant and occluded objects, while providing a precise selection, even when the environment is crowded. A user study is conducted to determine whether the proposed technique is more suitable when performing interaction tasks. The results show that the Zoom-fwd technique provides effective interaction with distant and occluded objects. The fourth paper, by Wen Zhou, Wenying Jiang, Biao Jie, and Weixin Bian, from Anhui Normal University, in Wuhu, China, present a multiagent evacuation framework for complex virtual fire scenarios, effectively used to simulate the procedure of multiagent evacuation to approximate the goal of fire drills in a least-cost manner. Specifically, the concept of a multihierarchy agent group model is proposed; that is, the evacuation of multiple agents is separated into leader-follower and freedom modes. Additionally, several complex actions of individual humans in actual fire drills are fully considered, and a multiaction agent schema is presented to characterize the associated real effects. In addition, generative adversarial imitation learning (GAIL) is adopted to obtain the evacuation path of the leader-agent by training numerous learning epochs. The results of extensive experiments show that the proposed method is feasible and realistically and reasonably shows the procedure of multiagent evacuation in complex fire emergency scenarios. The last paper by Jingjing Zhang, Jingsheng Lei, Shengying Yang, and Xinqi Yang, from Zhejiang University of Science and Technology, in Hangzhou, China propose the SIL-Net designed to discover semantic differences between two fine-grained categories via pairwise comparison. Specifically, SIL-Net first collecting contrastive information by learning the mutual feature of input image pair, and then compare it with individual features to generate corresponding semantic features. These features learn semantic differences from contextual comparison, this gives SIL-Net the ability to distinguish between two confusing images via pairwise interaction. After training, SIL-Net can adaptively learn feature priorities under the supervision of the margin ranking loss and converge quickly. SIL-Net performs well on two public vehicle benchmarks (Stanford Cars and CompCars), showing the suitability of SIL-Net to fine-grained vehicle recognition.

This issue contains five papers. In the first paper, Ege Tekgün, Muhtar Çağkan Uludağlı, Hüseyin Akcan, and Burak Erdeniz, from Izmir University of Economics in Turkey assess the influence of virtual avatar anthropomorphism and the synchronicity of the visuo-tactile stimulation on self-location using a virtual reality (VR) full-body illusion (FBI) experiment. During the experiment, half of the 36 participants observed a gender-matched full-body humanoid avatar from a first-person perspective (1PP) and the other half observed a less anthropomorphic full-body cubical avatar from 1PP while they were receiving synchronous and asynchronous visuo-tactile stimulation. Results show a significant main effect of the synchronicity of the visuo-tactile stimulation and avatar body type on self-location but no significant interaction was found between them. Moreover, the results of the self-report questionnaire provide additional evidence showing that participants who received synchronous visuo-tactile stimulation, experienced not only greater changes in the feeling of self-location, but also, increased ownership, and referral of touch. In the second paper, Ke Li, Qian Zhang, Jinyuan Jia, from Tongji University in Shanghai, China, and Hantao Zhao, from Southeast University in Nanjing, all in China discuss the technology of presenting building information modeling (BIM) with an online platform and the difficulty to display large-scale BIM scenes in a flawless manner on mobile browsers, due to network bandwidth and browser performance limitations. The authors propose CEBOW, a Cloud-Edge-Browser Online architecture for visualizing BIM components with online solutions. The method combines transmission scheduling, cache management, and optimal initial loading into a single system architecture. For network transmission testing, BIM scenes are used, and the results show that their method effectively reduces scene loading time and networking delay while improving the visualization effect of large-scale scenes. In the third paper, Yuzhu Dong and Eakta Jain, from University of Florida in Gainesville, and Sophie Jörg, from Clemson University, all in United States discuss how the importance of eyes for virtual characters stems from the intrinsic social cues. They emphasize that the eye animation impacts the perception of an avatar's internal emotional state. They present three large scale experiments that investigate the extent to which viewers can identify if an avatar is scared. The authors find that participants can identify a scared avatar with 75% accuracy using cues in the eyes including pupil size variation, gaze, and blinks. Because eye trackers return pupil diameter in addition to gaze, their experiments inform practitioners that animating the pupil correctly will add expressiveness to a virtual avatar with negligible additional cost. These findings also have implications for creating expressive eyes in intelligent conversational agents and social robots. The fourth paper, by Osman Güler, from TUSAŞ Şehit Hakan Gülşen Vocational and Technical Anatolian High School in Ankara and Serkan Savaş, from Çankırı Karatekin Üniversitesi, both in Turkey present a study showing that Interactive Boards (IBs) have the necessary hardware to run Stereoscopic 3D (S3D) training materials, but the panel has not got an S3D imaging feature. Therefore, only the Anaglyph S3D imaging method can be applied to IBs. Thus, an Anaglyph S3D training material was prepared for the interaction of the skeletal system and interactive 3D material design for IBs with its effects in education was investigated. A Likert-type scale was developed to measure the usability of the training material on IBs and the material was evaluated by 20 experts. The data were analyzed by the SPSS statistical program and the results were interpreted. According to the results, educational material seems to be positive in terms of image characteristics, content, navigation, and ease of use, font sizes were moderate for readability, the feedback process and the help menu were moderately effective. The last paper by Alexandra Sierra, Marie Postma, from Tilburg University in Netherlands and Menno Van Zaanen, from North-West University in Potchefstroom, South Africa investigate whether the uncanny valley effect, which has already been found for the human-like appearance of virtual characters, can also be found for animal-like appearances. They conducted an online study in which six different animal designs were evaluated in terms of the following properties: familiarity, commonality, naturalness, attractiveness, interestingness, and animateness. The study participants differed in age (under 10–60 years) and origin (Europe, Asia, North America, and South America). For the evaluation of the results, the authors ranked the animal-likeness of the character using both expert opinion and participant judgments. They also investigated the effect of movement and morbidity. The results confirm the existence of the uncanny valley effect for virtual animals, especially with respect to familiarity and commonality, for both still and moving images. No uncanny valley effect was detected for interestingness and animateness.

Abstract The use of social robots in healthcare systems or nursing homes to assist the elderly and their caregivers will be becoming common, where robots' understanding of engagement of the elderly is important. Traditional engagement estimation (EE) often requires expert involvement in a controlled dyadic interaction environment. In this article, we propose a supervised machine learning method to estimate the engagement state of the elderly in a multiparty human–robot interaction (HRI) scenario from the real‐world video recording as input. The method is built upon the basic concept of engagement in geriatric psychiatry and HRI video representations. It adapts pretrained models to extract behavior, affective, and visual signals to form the multi‐modal features. These features are then fed into a neural network made of a self‐attention mechanism and average pooling for individual learning, a graph attention network for group learning and a fully connected layer to estimate the engagement. We tested the proposed method using 43 wild multiparty elderly robot interaction (ERI) videos. The experimental results show that our method is capable of detecting the key participants and estimating the engagement state of the elderly effectively. Also our study demonstrates the signals from side‐participants in the main interaction group considerably contribute to the EE of the elderly in the multiparty ERI.

Sketch recognition relies on two types of information, namely, spatial contexts like the local structures in images and temporal contexts like the orders of strokes. Existing methods usually adopt convolutional neural networks (CNNs) to model spatial contexts, and recurrent neural networks (RNNs) for temporal contexts. However, most of them combine spatial and temporal features with late fusion or single-stage transformation, which is prone to losing the informative details in sketches. To tackle this problem, we propose a novel framework that aims at the multi-stage interactions and refinements of spatial and temporal features. Specifically, given a sketch represented by a stroke array, we first generate a temporal-enriched image (TEI), which is a pseudo-color image retaining the temporal order of strokes, to overcome the difficulty of CNNs in leveraging temporal information. We then construct a dual-branch network, in which a CNN branch and a RNN branch are adopted to process the stroke array and the TEI respectively. In the early stages of our network, considering the limited ability of RNNs in capturing spatial structures, we utilize multiple enhancement modules to enhance the stroke features with the TEI features. While in the last stage of our network, we propose a spatio-temporal enhancement module that refines stroke features and TEI features in a joint feature space. Furthermore, a bidirectional temporal-compatible unit that adaptively merges features in opposite temporal orders, is proposed to help RNNs tackle abrupt strokes. Comprehensive experimental results on QuickDraw and TU-Berlin demonstrate that the proposed method is a robust and efficient solution for sketch recognition.

CGI is one of the oldest international conferences in Computer Graphics in the world.It is the official conference of the Computer Graphics Society (CGS), a long-standing international computer graphics organization.CGI conference has been held annually in many different countries across the world and has gained a reputation as one of the key conferences for researchers and practitioners to share their achievements and discover the latest advances in Computer Graphics.With the change in the form of networking and intelligence in industry, manufacturing and all aspects of society, and the development of technology, we are aware of the increasingly obvious trend of evolution of intelligence in human society.Among them, metaverse is increasingly becoming a hot spot for research in various industries and has broad application prospects.It absorbs the results of the information revolution, the Internet revolution, the artificial intelligence revolution, and the virtual reality technology revolution including VR, AR, MR, and especially game engines, showing mankind the possibility of building a holographic digital world parallel to the traditional physical world.The core of the metaverse lies in the hosting of virtual assets and virtual identities.Unlike traditional games, users can experience different content, make different friends, create their own creations, and perform a series of virtual activities in the metaverse.With the popularity of smart terminals and the rise of applications such as e-commerce/short videos/games, "metaverse" has become an inevitable trend in the development of digital society.In a broad sense, the "metaverse" is a virtual space-time consisting of a series of augmented reality (AR), virtual reality (VR) and the Internet; in a narrow sense, the "metaverse" is a virtual world parallel to the real world.By wearing a helmet and headset device, one can enter a three-dimensional world constructed by computer simulation through a terminal connection".The new mode of "virtual reality" presentation and scene interaction for scene visualization will be more conducive to better visual effects and interactive operations in the digital world.The metaverse becomes the best track and new growth point for AI applications because of its huge imagination, close social attention and rich landing scenes, while AI and related arithmetic, big data and other technical fields are the technical base for the metaverse to become a kind of concrete expression in the future.Overall, with the further development of human technology and the improvement of hardware level, it becomes possible for humans to build a "meta" world.This year, CGI 2022 is still online as the pandemic prevents many researchers to come to Geneva.The conference CGI is organized from September 12 to September 16, 2022, by MIRALab at the Computer Research Centre (CUI) of the University of Geneva, in Switzerland.All presentations are online.In addition to the Visual Computer journal published by Springer, and the CAVW journal (Computer Animation and Virtual Worlds) published by Wiley, we have also included the twenty-three accepted papers in the VRIH journal (Virtual Reality and Intelligent Hardware journal published by Science Press).This special issue is composed of the six papers related to the topic of metaverse from these twenty-three accepted papers.

The uncanny valley hypothesis states that users might experience eerie when interacting with almost but not fully human-like artificial characters. The advancements in artificial intelligence, robotics, and computer graphics have led to life-like virtual humans and humanoid robots. It is necessary to revisit the hypothesis to check if they positively or negatively affect the current population, who are much more accustomed to the latest technologies. In this paper, we study and present a unique evaluation of the uncanny valley hypothesis by allowing participants to interact live with four different humanoid robots (of varying levels of humanlikeness). To evaluate the affinity of each robot, each participant needs to fill a survey questionnaire. Apart from this, we also use deep learning methods to quantify the participants’ emotional states using multi-modalcues, including visual, audio, and text, by recording the participant-robot interaction. The multi-modal analysis and surveys provide interesting results and insights into the uncanny valley hypothesis.

We present a unified and flexible framework to address the generalized problem of 3D motion synthesis that covers the tasks of motion prediction, completion, interpolation, and spatial-temporal recovery. Since these tasks have different input constraints and various fidelity and diversity requirements, most existing approaches only cater to a specific task or use different architectures to address various tasks. Here we propose a unified framework based on Conditional Variational Auto-Encoder (CVAE), where we treat any arbitrary input as a masked motion series. Notably, by considering this problem as a conditional generation process, we estimate a parametric distribution of the missing regions based on the input conditions, from which to sample and synthesize the full motion series. To further allow the flexibility of manipulating the motion style of the generated series, we design an Action-Adaptive Modulation (AAM) to propagate the given semantic guidance through the whole sequence. We also introduce a cross-attention mechanism to exploit distant relations among decoder and encoder features for better realism and global consistency. We conducted extensive experiments on Human 3.6M and CMU-Mocap. The results show that our method produces coherent and realistic results for various motion synthesis tasks, with the synthesized motions distinctly adapted by the given action labels.

Sketches are descriptive, high-level visual media in many systems and applications. However, current methods for sketch recognition are mainly based on large neural networks, which have millions of parameters and are too cumbersome to be deployed on edge devices. Besides, convolutional neural networks are not optimal, since many areas in sketches are blank and without any information. Hence, this paper aims at designing an efficient network that maintains the state-of-the-art accuracy. Our solution is a novel graph neural network that utilizes densely connected grouped convolutions on the graph representation of sketches. It allows us to extract and aggregate spatio-temporal features efficiently. Moreover, a compact spatial embedding module is introduced to explore spatial contexts, and consequently facilitates recognition. In this way, our network is small (about 10% parameters of ResNet- 18) and efficient (inference speed in 800 ~ 2,000 FPS), meanwhile has the competitive accuracy on the QuickDraw dataset.

Personality, moods and emotions play a vital role in real-life interactions, especially for team activities like sports. Diverse player personalities lead to distinct emotional reactions impacting game play. One of the major challenges while developing a realistic virtual environment is simulating human-like behaviors and imitations. We aim to improve the believability of virtual players by incorporating personality, moods and emotions for them. We propose a novel framework for a multi-player VR Volleyball game by integrating various behavioral changes during game play. This is achieved by simulating player reactions and rendering changes in the player’s body postures, which are dependent on game environment parameters and scenarios. We showcase visual analysis of individual players’ mood changes during game play to further corroborate the presented VR framework. Incorporating the proposed model, we exhibit the progression and variations in moods for players with different personalities when presented with varied game situations. We conducted a user study with 10 participants, which consisted of seven males and three females. We used two versions of the VR Volleyball game. One version of the game is without any behavioral changes, and the other version included the proposed behavioral model. Our results shows that behavioral changes setting had significantly higher ratings compared to the version with non-behavioral changes setting.

As the applications of intelligent agents (IAs) are gradually increasing in daily life, they are expected to have reasonable social intelligence to interact with people by appropriately interpreting human behavior and intention. This paper presents a method to estimate whether people have willingness to join in a conversation, which helps to endow IAs with the capability of detecting potential participants. The method is built on the CNN-LSTM network, which takes image features and social signals as input, making use of general information conveyed in images, semantic social cues proven by social psychology studies, and temporal information in the sequence of inputs. The network is designed to have a multi-branch structure with the flexibility of accommodating different types of inputs. We also discuss the signal transition in multiparty human-robot interaction scenarios. The method is evaluated on three datasets with social signals and/or images as inputs. The results show that the proposed method can infer human engagement intention well.

Abstract Virtual reality (VR) volleyball games can provide an immersive entertainment experience and facilitate users to get familiar with game rules. In real world, players often experience emotions from scores, strokes, passes, and the influence from their teammates or opponents, and so forth. However, seldom studies are carried out for this emotional experience in a virtual environment, as most VR volleyball games mainly concentrate on the game playing, such as body movements. In this article, we propose to enhance emotional experience by building emotional virtual characters in VR volleyball games. The virtual characters cannot only arouse their emotions but also express their facial expressions according to the game situation. Emotion patterns are learned from real‐world volleyball matches. The results demonstrate that our framework has great potential to enhance the user's emotional experience and engagement.

VRLAB

present a role-based attention model for reinforcement learning.The proposed model uses convolutional neural networks to generate soft attention maps, adding crucial role information in the task, forcing the agent to focus on important features, and distinguish task-related information.To validate the performance in complex problems, the proposed approach is evaluated in a challenging scenario, Football Academy in Google Research Football Environment, a newly released reinforcement learning environment with physics-based 3D simulator.The experimental results demonstrate that agents using role-based attention mechanism can perform better in football games.

This issue contains seven papers. In the first paper, Monan Wang and Jiaqi Cao, from Harbin University of Science and Technology in China have conducted in-depth research on collision detection, especially the collision detection of deformable objects, which is a bottleneck of virtual environment simulation. First, they briefly review the general problem of collision detection and how authors solve it. They then review the popular research methods and limitations of collision detection between deformable objects. Finally, they review the popular research methods and limitations of self-collision detection in deformable objects. They also discuss future directions of development. This review can be used as a reference for the application of collision detection in all directions. In the second paper, Priyadarshi Biplab Kumar, from NIT Hamirpur and Dayal R.Parhi and Mukesh Sethy, from National Institute of Technology Rourkela both in India, propose a firefly based computer vision integrated navigational analysis. It is intended to perform on humanoid robots for smooth movement by negotiation with obstacles present in complicated terrains. Here, the logic of the firefly algorithm has been used to design the controller by careful consideration of the navigational parameters. A computer vision-based method is integrated along with the developed controller. It is used to resolve some conflicting situations that may arise by encountering large sized obstacle or detection of an obstacle exactly in front of the robot where the robot becomes confused regarding the direction of turn. The developed navigational model has been tested in simulation environments using a NAO robot as humanoid platform. The simulation results are also verified through an experimental platform developed under research laboratory conditions. In the third paper, Oguzhan Tastan and Yusuf Sahillioglu, from METU in Ankara, Turkey propose a novel approach for reconstructing plausible 3D human body models from small number of 3D points which represent body parts. They leverage a database of 3D models of humans varying from each other by physical attributes such as age, gender, weight, and height. First, they divide the bodies in database into seven semantic regions. Then, they search the database for the best matching body part. For the matching criterion, they use the distance between novel point-based features of input points and body parts in the database. They then combine the matched parts from different bodies into one body. To evaluate their results objectively, they pick points from each part of the ground-truth human body models, then reconstruct them using their method and compare the resulting bodies with the corresponding ground-truths. The fourth paper, by Jong-Chul Yoon, from Kangwon National University, in Samcheok, and HyeongYeop Kang, from Kyung Hee University, Yongin, both in Korea, present a smartphone AR application, named the AR-E-Helper, which assists the learning of students in higher education lectures. Their goal is to provide an AR enhanced learning experience for students. To validate the effectiveness of the AR-E-Helper, they conducted an experiment that compares three classes: AR enhanced, smartphone enhanced, and non-technology enhanced classes. Through the experiment, they observed that their application was helpful in maintaining student's focus in class, promoting their interest, and increasing their satisfaction. Furthermore, they also found how to improve their application based on the observations that the application brought some downsides to the learning activities. In the fifth paper, Yun Sik Cho, Jie Won Kang, Jaekyung Jeon, Jong Chan Park and Jinmo Kim, from Hansung University, and Mingyu Kim, from Korea University, all in Seoul, Korea present a study in Virtual Reality for education. During this study, the researchers created a virtual environment for use in Southern Africa, where students could practise managing a young adult with a foreign object in the airway. The aim of the virtual environment was to determine whether a viable, “home-made” solution could be created which could also be expanded later to incorporate more scenarios. This was due to the expensive nature of existing systems for virtual clinical simulation. To determine whether the virtual environment is usable, two expert review panels assisted in testing the virtual environment. The first panel being Computer Science experts and the second Health Science experts. Each panel evaluated the environment and the scenario using heuristic evaluation and cognitive walkthroughs. In the sixth paper, Lin Zhuo, Zhen Liu, Tingting Liu, and Yanjie Chai, from Ningbo University and Chih-Chieh Hung, from National Chung Hsing University, Taichung, all in China present simple, highly immersive x-person asymmetric interactions that account for the experience type characteristics of asymmetric virtual environments, jointly experienced by virtual reality (VR) users and augmented reality (AR) users. The first-person interactions for VR users are performed using hand gestures, and they define a manipulation process that maps the gestures and object control scheme to provide intuitive interactions with the virtual environment and objects. The third-person interaction for AR users is designed to view the overall virtual scene and recognize and judge situations to allow for intuitive communication and interactions among the virtual environment, objects, and users based on a touch interface. The core goal of this process is to provide all users who participate in asymmetric virtual environments with satisfying experiences and an improved presence through individualized experience modes and roles. The last paper by Zhiyong Chen, Xiaowei Chen, Shihui Guo, Minghong Liao, from Xiamen University and Yong Ma, from Jiangxi Normal University, in Nanchang, all in China and Yipeng Qin, from Cardiff University, in UK propose a crowd emotional model to simulate the crowd movement in outdoor emergency situations. First, the crowd entropy and the movement difference are proposed to describe the emotional impact of the crowd scene on the agents. The perception of vision and hearing are considered, and the calculation formulas of the agent's emotional intensity and crowd emotional contagion are proposed. By calculating individual trajectories in the real video, the cumulative differences between the movements of the real crowd and the corresponding virtual crowd are analyzed. At last, a multi-parameter optimization method is implemented by the differential evolution algorithm. To verify the parameters in models, three videos which are generated from three real cases, including explosion attack, shooting incident, and crowd disturbance are selected for experimental verification.

This issue contains five papers. In the first paper, Kun Qian from King's College in London, UK, Meili Wang, from Northwest Agriculture and Forestry University, in Shannxi, China and Yaqing Cui, Bournemouth University, in Poole, UK propose a simulation ready model generation pipeline which can convert a non-manifold polygonal surface mesh into a degeneracy free surface while preserving the original model's surface parameterization attribute. Their pipeline includes two stages. The first stage is a voxelization and remesh based simulation ready model generation pipeline which can keep the shape of original 3D surface model meanwhile eliminate the non-manifold geometry. In the second stage, a cutting-based surface mesh parameterization transfer algorithm is proposed which can transfer the original surface parameterization to the simulation ready model. Detailed comparison with existing pipelines shows that the proposed pipeline can achieve surface parameterization preservation feature and is more suitable for improving the efficiency of virtual surgery production. In the second paper, Muhammad Usman and Petros Faloutsos, from York University in Toronto, Canada, Brandon Haworth, from University of Victoria in Canada and Mubbasir Kapadia, from Rutgers University, Piscataway in USA, propose a cross-browser service-based simulation analytics platform to analyze environment layouts with respect to occupancy and activity. Their platform allows users to access simulation services by uploading 3D environment models in numerous common formats, devise targeted simulation scenarios, run simulations, and instantly generate crowd-based analytics for their designs. The authors conducted a case study to showcase cross-domain applicability of their service-based platform, and a user study to evaluate the usability of this approach. In the third paper, Junxuan Bai, Rong Dai, Ju Dai, and Junjun Pan, from Beihang University in Beijing, and Peng Cheng Laboratory in Shenzhen, all in China propose a hybrid feature for emotional classification in dance performances. The hybrid feature is composed of an explicit feature and a deep feature. The explicit feature is calculated based on the Laban movement analysis, which considers the body, effort, shape, and space properties. The deep feature is obtained from latent representation through a 1D convolutional autoencoder. Eventually, the authors present an elaborate feature fusion network to attain the hybrid feature that is almost linearly separable. The abundant experiments demonstrate that their hybrid feature is superior to the separate features for the emotional classification in dance performances. The fourth paper, by Donya Ghafourzadeh, Sahel Fallahdoust, Cyrus Rahgoshay, André Beauchamp, Adeline Aubame, Eric Paquette, from Ecole de technologie supérieure, and Tiberiu Popa, from Concordia University, all in Montreal, Canada, present an approach to construct realistic 3D facial morphable models (3DMM) that allows an intuitive facial attribute editing workflow. They created a 3DMM by combining local part-based 3DMM for the eyes, nose, mouth, ears, and facial mask regions. Their local PCA-based approach uses a novel method to select the best eigenvectors from the local 3DMM to ensure that the combined 3DMM is expressive, while allowing accurate reconstruction. They provide different editing paradigms, all designed from the analysis of the data set. Their part-based 3DMM is compact, yet accurate, and compared to other 3DMM methods, it provides a new trade-off between local and global control. The results show that their part-based 3DMM approach has excellent generative properties and allows the user intuitive local control. The last paper by David Antonio Gomez Jauregui, from Ecole Supérieure des Technologies Industrielles Avancées, in Bidart, France, Tom Giraud, from University of Augsburg in Germany, Brice Isableu, from Aix-Marseille University, in Aix-en-Provence, France and Jean-Claude Martin, from LIMSI-CNRS, in Orsay, France propose a library of motion-captured movements that interviewers are most likely to display. They designed a fully automatic interactive virtual agent able to display these movements in response to the bodily movements of the user. Thirty-two participants presented themselves to this virtual agent during a simulated job interview. The authors explain the different hypotheses used (1) comparison between the performance of participants with human interviewers and the performance of participants with virtual interviewers, (2) comparison between mirror and random postural behaviors displayed by a female vs. a male virtual interviewer, (3) correlation between the participants' performance and their personality traits.

Compared with depth-based 3D hand pose estimation, it is more challenging to infer 3D hand pose from monocular RGB images, due to the substantial depth ambiguity and the difficulty of obtaining fully-annotated training data. Different from the existing learning-based monocular RGB-input approaches that require accurate 3D annotations for training, we propose to leverage the depth images that can be easily obtained from commodity RGB-D cameras during training, while during testing we take only RGB inputs for 3D joint predictions. In this way, we alleviate the burden of the costly 3D annotations in real-world dataset. Particularly, we propose a weakly-supervised method, adaptating from fully-annotated synthetic dataset to weakly-labeled real-world single RGB dataset with the aid of a depth regularizer, which serves as weak supervision for 3D pose prediction. To further exploit the physical structure of 3D hand pose, we present a novel CVAE-based statistical framework to embed the pose-specific subspace from RGB images, which can then be used to infer the 3D hand joint locations. Extensive experiments on benchmark datasets validate that our proposed approach outperforms baselines and state-of-the-art methods, which proves the effectiveness of the proposed depth regularizer and the CVAE-based framework.

Recent methods including CoViAR and DMC-Net provide a new paradigm for action recognition since they are directly targeted at compressed videos (e.g., MPEG4 files). It avoids the cumbersome decoding procedure of traditional methods, and leverages the pre-encoded motion vectors and residuals in compressed videos to complete recognition efficiently. However, motion vectors and residuals are noisy, sparse and highly correlated information, which cannot be effectively exploited by plain and separated networks. To tackle these issues, we propose a joint feature optimization and fusion framework that better utilizes motion vectors and residuals in the following three aspects. (i) We model the feature optimization problem as a reconstruction process that represents features by a set of bases, and propose a joint feature optimization module that extracts bases in the both modalities. (ii) A low-rank non-local attention module, which combines the non-local operation with the low-rank constraint, is proposed to tackle the noise and sparsity problem during the feature reconstruction process. (iii) A lightweight feature fusion module and a self-adaptive knowledge distillation method are introduced, which use motion vectors and residuals to generate predictions similar to those from networks with optical flows. With these proposed components embedded in a baseline network, the proposed network not only achieves the state-of-the-art performance on HMDB-51 and UCF-101, but also maintains its advantage in computational complexity.

Social Assistive Robotics is increasingly being used in care settings to provide psychosocial support and interventions for the elderly with cognitive impairments. Most of these social robots have provided timely stimuli to the elderly at home and in care centres, including keeping them active and boosting their mood. However, previous investigations have registered shortcomings in these robots, particularly in their ability to satisfy an essential human need: the need for companionship. Reports show that the elderly tend to lose interests in these social robots after the initial excitement as the novelty wears out and the monotonous familiarity becomes all too familiar. This paper presents our research facilitating conversations between a social humanoid robot, Nadine, and cognitively impaired elderly at a nursing home. We analysed the effectiveness of human-humanoid interactions between our robot and 14 elderly over 29 sessions. We used both objective tools (based on computer vision methods) and subjective tools (based on observational scales) to evaluate the recorded videos. Our findings showed that our subjects engaged positively with Nadine, suggesting that their interaction with the robot could improve their well-being by compensating for some of their emotional, cognitive, and psychosocial deficiencies. We detected emotions associated with cognitively impaired elderly during these interactions. This study could help understand the expectations of the elderly and the current limitations of Social Assistive Robots. Our research is aligned with all the ethical recommendations by the NTU Institutional Review Board.

This issue contains four papers. In the first paper, Xiao Zhang and Deling Yang, from Guangzhou Academy of Fine Arts in China, propose a study to explore how online museums distribute the postproduction of cultural knowledge related to artworks in a network society. This study has implications for how online museums will adapt to the future development of knowledge societies. The core innovation of this study is exploring distributed knowledge postproduction in online museums from the perspective of Mode 2 knowledge production and online distributed knowledge production. This study also examines distributed knowledge postproduction in online museums, which coconstructs knowledge and meaning. In the second paper, Bin Wang, Weibin Liu, and Weiwei Xing, from Beijing Jiaotong University, China, propose an automatic segmentation method based on geodesic by introducing Riemannian manifold. They convert Mo-cap data from Euler angles into quaternions, calculate the intrinsic mean of the motion sequence, hemispherize quaternions, and use logarithmic and exponential mapping to calculate geodesic distances instead of quaternions. The experimental results show that the algorithms can achieve automatic segmentation and have a better segmentation effect. In the third paper, Jure Demšar and Iztok Lebar Bajec, from University of Ljubljana, Slovenia and Will Blewitt, from Coventry University, UK, present a hybrid model for the simulation of herds of grazing sheep. The novel approach called hybrid modeling tries to mix the best of both worlds—precision of individual based models and speed of flow based ones. Through Bayesian data analysis, the authors show that they can encompass several aspects of real-world sheep behavior. Their hybrid model is also extremely efficient, capable of simulating herds of more than 1000 individuals in real-time without resorting to GPU execution. In the last paper, Wei Cao, Zhixin Yang, Xiaohua Ren, Luan Lyu, and Bob Zhang, from University of Macau, China; Yanci Zhang, from Sichuan University in Chengdu, China; and Enhua Wu, from University of Macau and Chinese Academy of Sciences in Beijing, China, introduce an improved approach to simulate nonorthotropic geometric models under large deformation. The improvements are mainly twofold. First, a frame-field is specified on a given undeformed object, that is, each point of the object is equipped with a frame. Second, they design the deformation properties along each axis in the local nonorthogonal coordinate to get a local constitutive model. To improve the stability, they introduce a time-varying method to simultaneously track the local coordinates reorientation by pushing forward the original frame-field to the deformed frame-field.

This issue contains three papers. In the first paper, Manoj Kumar Muni and Dayal R. Parhi, from the National Institute of Technology in Rourkela, India, and Priyadarshi Biplab Kumar, from the National Institute of Technology in Hamirpur, India, consider the use of gray wolf optimization controller (GWOC) as a multiobjective technique for multiple humanoid navigation. Upon activation of GWOC, the humanoids mimic the group hunting behavior of gray wolves and navigate toward the target in a collision-free manner in the presence of both static and dynamic hurdles. The wolves in the pack will either diverge for searching prey or converge together for attacking the prey following the best search agent. GWOC is able to keep the humanoid free from being trapped in local minima, whereas it facilitates it to head toward global minima. GWOC provides better results than other intelligent techniques because of its five characteristics: safe boundary, protection, following, hunting, and caring. In the second paper, Lianyao Wu, Wanggen Wan, Xiaoqing Yu, Chunkai Ye, and A.A.M. Muzahid, from Shanghai University, China, present a real-time augmented reality framework based on a semidense method with CPU. Specifically, the semidense method searches pixels with high gradients in each keyframe and estimates accurate depths by fusing matching pixels in other keyframes. The authors propose an outlier removal method that excludes 3D points outside the camera trajectory. By integrating this method, their framework preserves clean edges of the real environment. The experimental results on the data set show that their proposed framework has better surface reconstruction accuracy than other methods and their tracking thread runs in an acceptable speed when the semidense mapping thread runs backend. With the benefit of the robust camera and the aligned surface, virtual characters of their AR application enable realistic movement and collision. In the last paper, Jongmin Kim, from Kangwon National University in Chuncheon, Korea, Yeongho Seol, from Weta Digital Ltd in Wellington, New Zealand, and Hoemin Kim and Taesoo Kwon, from Hanyang University, Korea, present a novel interactive inverse kinematics (IK) framework that automatically and efficiently handles various types of collisions and spatial relationship. For the collision handling, they suggest a new type of linear constraint (half-space constraint) and a novel collision-response strategy based on gradual constraint accumulation. Specifically, the constraints that were used in the previous iterations continue to be used while more constraints are added to finally resolve the collisions. This approach also allows the user to edit human motion without any repetitive procedures and tuning the parameters. Regarding the spatial relationship, the authors provide another linear constraint that preserves complex spatial relationships between body parts. Their approach is novel in that both half-space constraints for collision handling and relative constraints for pose editing are embedded into an IK solver.

This issue contains three papers. In the first paper, M. Hassaballah, Abdelraheem M. Aly, and Ahmad Abdelnaim, from South Valley University Qena, Egypt, propose for the Interactive simulations of fluids flow to obtain the stable pressure from modifications in the source term of pressure Poisson equation (PPE). The particles disorders are solved using a shifting technique with the current treatment of source term in PPE. In addition, the dummy boundary particles are used for the rigid boundary treatment. Performance of the stabilized ISPH model is tested on various numerical simulations with largely distorted free surface including liquid sloshing problems, fluid–fluid, fluid–structure interactions and dam-break over horizontal and inclined plane. To extend the applicability of the stabilized ISPH model, the postprocess including visual realism with the highly rendering scheme is coupled. In the second paper, Xianxuan Lin, Wang Xun, Jinyu Li, Bailin Yang, Kaili Zhang, Tianxiang Wei, from Zhejiang Gongshang University in Hangzhou, China, and Frederick W. B. Li, from Durham University, UK, propose a color migration framework for home scene images. It picks the coloring from a template image and transforms such coloring to a home scene image through a simple interaction. Their framework comprises three main parts. First, they carry out an interactive segmentation to divide an image into local regions and extract their corresponding colors. Second, they generate a matching color table by sampling the template image according to the color structure of the original home scene image. Finally, they transform colors from the matching color table to the target home scene image with boundary transition maintained. Experiment results demonstrated their framework can produce aesthetically attractive home scene coloring. In the last paper, Xiaoyu Cui, Ruifan Cai, Xiangjun Tang, Xiaogang Jin, from Zhejiang University, in Hangzhou, China, and Zhigang Deng, from University of Houston, Texas, United States, propose a novel shape-constrained fireworks simulation method with rich textures in a helmet-mounted display virtual environment using sketched feature lines as input. Their approach first retrieves an object from a three-dimensional (3D) model database using a sketch-based 3D shape retrieval algorithm. Then, in order to approximate models with complex structures, they introduce a novel point sampling algorithm based on Gaussian curvatures, which stores not only the positions of the selected vertices but also the texture (UV) coordinates information for texture display. In addition, they introduce a multilevel explosion process so that the fireworks can dynamically form specific, visually pleasing shapes. Through their experiments, they demonstrate that their approach can produce better results than state-of-the-art approaches.

This special issue contains 28 full papers selected from the Computer Animation and Social Agents 2020 Conference (CASA2020). This conference was founded by the Computer Graphics Society in 1988 in Geneva and is the oldest conference on Computer Animation in the world. It has been held in many countries around the world and in recent years in Beijing, China (2018), Paris, France (2019) and this year in Bournemouth, United Kingdom. Because of the Covid-19 pandemic, this year, the conference will be held online through the Youtube Channel. The best paper award will be announced on the conference website after the conference. We would like to thank the authors for sharing their research findings by submitting papers to CASA2020. We are very grateful to the Program Committee members for reviewing the papers and to all the people who have contributed to the success of CASA2020 in Bournemouth. The conference is organized by Bournemouth University under the guidance of the Computer Graphics Society (CGS). Conference co-chairs Jian Jun Zhang (Bournemouth University, UK) Nadia Magnenat Thalmann (University of Geneva, Switzerland and Nanyang Technological University, Singapore) Program co-chairs Daniel Thalmann (EPFL, Switzerland) Xiaosong Yang (Bournemouth University, UK) Weiwei Xu (Zhejiang University, China) Publicity chair Jian Chang (Bournemouth University, UK) Local chair Feng Tian (Bournemouth University, UK) International program committee Nadine Aburumman, Brunel University, UK Norman Badler, University of Pennsylvania, USA Selim Balcisoy, Sabanci University, Turkey Loic Barthe, IRIT—Université de Toulouse, France Jan Bender, RWTH Aachen University, Germany Raphaëlle Chaine, LIRIS Université Lyon 1, France Jian Chang, Bournemouth University, UK Fred Charles, Bournemouth University, UK Parag Chaudhuri, Indian Institute of Technology, Bombay, India Marc Christie, INRIA, France Justin Dauwels, Nanyang Technological University, Singapore Shujie Deng, King's College London, UK Zhigang Deng, University of Houston, USA Etienne de Sevin, SANPSY University of Bordeaux, France Petros Faloutsos, York University, Canada Christos Gatzidis, Bournemouth University, UK Ugur Gudukbay, Bilkent University, Turkey Shihui Guo, Xiamen University, China Xiaohu Guo, The University of Texas at Dallas, USA James Hahn, George Washington University, USA Carlo Harvey, Birmingham City University, UK Gaoqi He, East China Normal University, China Ying He, Nanyang Technological University, Singapore Kemao Qian, Nanyang Technological University, Singapore Ruizhen Hu, Shenzhen University, China Jinyuan Jia, Tongji University, China Tao Jiang, University of Surrey, UK Xiaogang Jin, Zhejiang University, China Marcelo Kallmann, University of California, Merced, USA Prem Kalra, IIT Delhi, India Dongwann Kang, Seoul National University of Science and Technology, Korea Mubbasir Kapadia, Rutgers University, USA Min H. Kim, Korea Advanced Institute of Science and Technology, Korea Scott King, Texas A&M University—Corpus Christi, USA Taesoo Kwon, Hanyang University, China Sung-Hee Lee, Korea Advanced Institute of Science and Technology, Korea Wonsook Lee, University of Ottawa, Canada Tsai-Yen Li, National Chengchi University, Taiwan Guoliang Luo, East China Jiaotong University, China Chongyang Ma, Snap Inc., USA Anderson Maciel, Universidade Federal do Rio Grande do Sul, Brazil Nadia Magnenat Thalmann, University Of Geneva, Switzerland Shigeo Morishima, Waseda University, Japan Soraia Musse, Pontificia Universidade Catolica do Roi Grande do Sul, PUCRS, Brazil Rahul Narain, Indian Institute of Technology, Delhi, India Junjun Pan, Beihang University, China Nuria Pelechano, Universitat Politècnica de Catalunya, Spain Julien Pettre, INRIA, France Nicolas Pronost, Université Claude Bernard Lyon 1, France Kun Qian, King's College London, UK Craig Schroeder, University of California, Riverside, USA Ari Shapiro, Embody Digital, USA Hubert P. H. Shum, Northumbria University, UK Shinjiro Sueda, Texas A&M University, USA Daniel Thalmann, Ecole Polytechnique Fédérale de Lausanne, Switzerland Feng Tian, Bournemouth University, UK Yiying Tong, Michigan State University, USA Meili Wang, Northwest A&F University, China Zhao Wang, Zhejiang University, China Enhua Wu, University of Macau & ISCAS, China Zhongke Wu, Beijing Normal University, China Weiwei Xu, Zhejiang University, China Yachun Fan, Beijing Normal University, China Bailin Yang, Zhejiang Gongshang University, China Yin Yang, University of New Mexico, USA Xiaosong Yang, Bournemouth University, UK Yuting Ye, Oculus Research, USA Lihua You, Bournemouth University, UK Hongchuan Yu, Bournemouth University, UK Zerrin Yumak, Utrecht University, Netherlands Wenshu Zhang, Cardiff Metropolitan University Jian Zhang, Bournemouth University, UK Jianmin Zheng, Nanyang Technological University, Singapore

Despite great progress in 3D pose estimation from single-view images or videos, it remains a challenging task due to the substantial depth ambiguity and severe self-occlusions. Motivated by the effectiveness of incorporating spatial dependencies and temporal consistencies to alleviate these issues, we propose a novel graph-based method to tackle the problem of 3D human body and 3D hand pose estimation from a short sequence of 2D joint detections. Particularly, domain knowledge about the human hand (body) configurations is explicitly incorporated into the graph convolutional operations to meet the specific demand of the 3D pose estimation. Furthermore, we introduce a local-to-global network architecture, which is capable of learning multi-scale features for the graph-based representations. We evaluate the proposed method on challenging benchmark datasets for both 3D hand pose estimation and 3D body pose estimation. Experimental results show that our method achieves state-of-the-art performance on both tasks.

In this work, we address the challenging issue of scene segmentation. To increase the feature similarity of the same object while keeping the feature discrimination of different objects, we explore to propagate information throughout the image under the control of objects' boundaries. To this end, we first propose to learn the boundary as an additional semantic class to enable the network to be aware of the boundary layout. Then, we propose unidirectional acyclic graphs (UAGs) to model the function of undirected cyclic graphs (UCGs), which structurize the image via building graphic pixel-by-pixel connections, in an efficient and effective way. Furthermore, we propose a boundary-aware feature propagation (BFP) module to harvest and propagate the local features within their regions isolated by the learned boundaries in the UAG-structured image. The proposed BFP is capable of splitting the feature propagation into a set of semantic groups via building strong connections among the same segment region but weak connections between different segment regions. Without bells and whistles, our approach achieves new state-of-the-art segmentation performance on three challenging semantic segmentation datasets, i.e., PASCAL-Context, CamVid, and Cityscapes.

Hiring robots for the workplaces is a challenging task as robots have to cater to customer demands, follow organizational protocols and behave with social etiquette. In this study, we propose to have a humanoid social robot, Nadine, as a customer service agent in an open social work environment. The objective of this study is to analyze the effects of humanoid robots on customers in a work environment, and see if it can handle social scenarios. We propose to evaluate these objectives through two modes, namely: survey questionnaire and customer feedback. The survey questionnaires are analyzed based on the datapoints provided in the questionnaire. We propose a novel approach to analyze customer feedback data using sentic computing. Specifically, we employ aspect extraction and sentiment analysis to analyze the data. From our framework, we detect sentiment associated to the aspects that mainly concerned the customers during their interaction. This allows us to understand customers expectations and current limitations of robots as employees.

Human-robot interaction in corporate workplaces is a research area which remains unexplored. In this paper, we present the results and analysis of a social experiment we conducted by introducing a humanoid robot (Nadine) into a collaborative social workplace. The humanoid's primary task was to function as a receptionist and provide general assistance to the customers. Moreover, the employees who interacted with Nadine were given over a month to get used to her capabilities, after which, the feedback was collected from the staff on the grounds of influence on productivity, affect experienced during interaction and their views on social robots assisting with regular tasks. Our results show that the usage of social robots for assisting with normal day-to-day tasks is taken quite positively by the co-workers and that in the near future, more capable humanoid social robots can be used in workplaces for assisting with menial tasks. Finally, we posit that surveys such as ours could result in constructive opinions based on technological awareness, rather than opinions from media-driven fears about the threats of technology.

Abstract Team formation with realistic crowd simulation behavior is a challenge in computer graphics, multiagent control, and social simulation. In this study, we propose a framework of crowd formation via hierarchical planning, which includes cooperative‐task, coordinated‐behavior, and action‐control planning. In cooperative‐task planning, we improve the grid potential field to achieve global path planning for a team. In coordinated‐behavior planning, we propose a time–space table to arrange behavior scheduling for a movement. In action‐control planning, we combine the gaze‐movement angle model and fuzzy logic control to achieve agent action. Our method has several advantages. (1) The hierarchical architecture is guaranteed to match the human decision process from high to low intelligence. (2) The agent plans his behavior only with the local information of his neighbor; the global intelligence of the group emerges from these local interactions. (3) The time–space table fully utilizes three‐dimensional information. Our method is verified using crowds of various densities, from sparse to dense, employing quantitative performance measures. The approach is independent of the simulation model and can be extended to other crowd simulation tasks.

No abstract is available for this article.

This issue contains six papers.In the first paper, Nuttapon Vanakittistien, Attawith Sudsang, and Nuttapong Chentanez from Chulalongkorn University, Bangkok, Thailand, and NVIDIA, Zurich, Switzerland, propose a system for creating a hair model that matches a user's hairstyle from images.The model consists of guide hair strands and can be used in a real-time hair simulator.The goal differs from most previous works, which aim to create realistic high-resolution hair for offline applications or create mesh of the exterior of the hair volume for image manipulation.The primary aim is for a user to be able to put his/her hairstyle into a game or other real-time applications.By taking photos in eight views of the user's head using a smartphone camera and segmenting images with some easy-to-use tools, the player will obtain his/her own hair model in NVIDIA's HairWorks, which is a hair simulator used in many games.The authors show a number of results demonstrating the capabilities of their system in this paper.In the second paper, Priyadarshi Biplab Kumar, Saktiswarup Mohapatra, and Dayal R. Parhi from the National Institute of Technology Rourkela, Odisha, India, present an investigation dealing with the path planning of NAO humanoid robots.In the present work, a classical method of regression analysis and an artificial intelligence technique of fuzzy logic are implemented separately for the purpose of obstacle avoidance during the motion of humanoid NAOs towards respective targets.The simulation analysis of the proposed techniques is carried out using V-REP software.The experiments are performed in laboratory conditions with a proper environment for working on the humanoid NAOs.Finally, a comparison has been made between the simulation and experimental results.The results obtained from the simulation and experimental analyses are in good agreement with each other, which suggest that the proposed methodologies can be used as methods of robust control for the navigation of humanoids.In the third paper, Lin Wang and Soonhung Han from KAIST, Daejeon, Korea, conducted research related to the design for a realistic motion visualization system based on numerical simulation data using a commercial game engine (Unity 3D).In their system, the six-degree-of-freedom motion (Surge, Sway, Heave, Roll, Pitch, and Yaw) is simulated and visualized based on numerical analysis results of two hydrodynamics simulation softwares, which can illuminate the nuance between simulation results and experiment results and provide a "real-time" visual experience about motion in each direction.Meanwhile, comprehensive sea environment conditions, such as wind, rain, water, sound, and cloudiness, are also visualized in Unity 3D.

This special issue contains 26 full papers selected from the Computer Animation and Social Agents 2019 (CASA'2019) Conference. This conference was founded by the Computer Graphics Society in 1988 in Geneva and is the oldest conference on computer animation in the world. It has been held in various countries around the world, in recent years, in Seoul, South Korea (2017); Beijing, China (2018); and this year, in Paris, France. This year, the conference is organized jointly with ACM Intelligent Virtual Agents (ACM IVA 2019). They started the conference with two days of presentations of technical papers, followed by a one-day tutorial on “Modelling interaction between real humans and virtual humans or social robots” and “Capturing real-world data for characters, sometimes with machine learning,” and a workshop on “Crowd analysis and applications: Simulations meet video analytics.” Addressing this year's conference are two invited speakers: (1) Professor Marie-Paule Cani from Ecole Polytechnique, Paris, France, and (2) Dr. Thabo Beeler from Disney Research in Zürich, Switzerland. During the social dinner, the best paper award will be announced. We would like to thank the authors for having submitted a paper to CASA'2019, the program committee members, and all the people who have contributed to the success of CASA'2019 in Paris. The conference is organized by Centre National de la Recherche Scientifique (CNRS) and Sorbonne University, in cooperation with ACM-SIGGRAPH, under the guidance of the Computer Graphics Society (CGS).

VRLAB

Behavioral realism and realistic interactions are major criteria for improving social presence in virtual reality environments. We focus on multi-party VR applications where computer agents and avatars interact, share and collaborate with each other using objects. Our formulation employs realistic animations to simulate human-like behavioral motions of computer agents while they interact with avatars to enhance the sense of social presence in the VR environment. We exemplify our proposed model in a VR volleyball game setup. We model specific underlying interactions like gazing, collision detection and miscellaneous reactions (like how to pick a volleyball, how to transfer the ball to server) between computers players and avatars in the VR Volleyball game. We conduct a preliminary user survey to illustrate the significance of inclusion of realistic interactions for improving sense of social presence in a multi-party VR environment.

The current study concerns the identification of possible differences in perception between the virtual and the real world in terms of the effect on brain activity. For this reason, an EEG device was used to capture participants' brain activity in different brain areas during their exposure to different virtual and real environments. The environments considered in this study portray a classroom environment with a scenario suitable for teacher training and professional development. The first aim of the experiment is to investigate if exposure to a virtual environment can affect motor, cognitive or other function of the users, and the second aim is to test if the graphics content and nature of such an environment can influence the user experience. During the study, the optimum duration of exposure in a virtual environment was also assessed by measuring the time that the brain needs to perceive and adapt to the new state. Our results, consisting of EEG data analyzed in 10 Regions of Interest (ROIs) and responses from an Igroup Presence questionnaire, indicated a significant difference in each brain area, especially in the frontal and occipital region, when a participant was exposed to a non-realistic virtual environment, compared to a realistic one, highlighting the impact of the selected virtual environment design. The results of the experiment can play an important role in defining the characteristics of optimal virtual environments for virtual reality-based training applications.

Detecting small hand-held objects plays a critical role for human-robot interaction, because the hand-held objects often reveal the intention of the human, e.g., use a cell phone to make a call or use a cup to drink, thus helps the robots understand the human behavior and response accordingly. Existing solutions relying on wearable sensor to detect hand-held objects often comprise the user experiences thus may not be preferred. With the development of commodity RGB-D sensors, e.g., Microsoft Kinect II, RGB and depth information have been used for the understanding of human actions and recognizing objects. Motivated by the previous success, we propose to detect hand-held objects using RGB-D sensor. However, instead of performing object detection alone, we propose to leverage human body pose as the context to achieve robust hand-held object detection in RGB-D videos. Our system demonstrates a person can interact with a humanoid social robot with hand-held object such as a cell phone or a cup. Experimental evaluations validate the effectiveness of this proposed method.

This paper discusses automatic path planning for tower crane lifting in highly complex environments to be digitized using point cloud representation. A mathematical optimization technique is developed to identify the lifting path with GPU accelerated massively parallel genetic algorithm. A continuous collision detection method is designed for real time application of collision avoidance during the crane lifting process.

Virtual Reality(VR) is still an emerging technology in terms of recognizing its full potential in education and specifically in teacher education. A key issue of a VR-based approach for teacher training, is the level of presence along with the empathy inflicted on the trainees that will allow them to experience realistic and emotion-rich classroom situations in a virtual environment. This paper describes an experiment that aims to assess the influence of the graphical realism of a virtual classroom to the levels of presence and development of empathy skills for trainee teachers. Moreover, a second objective is to investigate whether there are significant differences between training in a VR classroom and a real physical classroom and how this affects the trainee teacher. The overall conclusion of the experiment is that the design of the VR classroom environment influenced the levels of immersion and presence. Moreover, according to the results there are serious indications that the VR system provided users the immersion necessary for the development of embodied thinking skills and thus of empathy in relation to multiculturalism.

When building robotic hands, researchers are always face with two main issues of how to make robotic hands look human-like and how to make robotic hands function like real hands. Most existing solutions solve these issues by manually modelling the robotic hand [10-18]. However, the design processes are long, and it is difficult to duplicate the geometry shape of a human hand. To solve these two issues, this paper presents a simple and effective method that combines 3D printing and digitization techniques to create a 3D printable cable-driven robotic hand from scanning a physical hand. The method involves segmenting the 3D scanned hand model, adding joints, and converting it into a 3D printable model. Comparing to other robotic solutions, our solution retains more than 90% geometry information of a human hand1, which is attained from 3D scanning. Our modelling progress takes around 15 minutes that include 10 minutes of 3D scanning and five minutes for changing the scanned model to an articulated model by running our algorithm. Compared to other articulated modelling solutions [19, 20], our solution is compatible with an actuation system which provides our robotic hand with the ability to mimic different gestures. We have also developed a way of representing hand skeletons based on the hand anthropometric. As a proof of concept, we demonstrate our robotic hand's performance in the grasping experiments.

Speech disorders are among the salient characteristics of negative symptoms of schizophrenia. Such impairments are often exhibited through disorganized speech, inappropriate affective prosody, and poverty of speech. The current method of detecting such symptoms requires the expertise of a trained clinician, which may be prohibitive due to cost, stigma or high patient-to-clinician ratio. An objective method to extract non-verbal and verbal speech-related cues can help to automate and simplify the assessment method of severity of speech-related symptoms of schizophrenia. In this paper, a novel automated method is presented which uses speech content from schizophrenic patients to predict the clinician-assigned subjective ratings of their negative symptoms. Specifically, the interviews of 50 schizophrenia patients were recorded and features related to acoustics, linguistics and non-verbal conversation were extracted. The subjective ratings can be accurately predicted from the objective features with an accuracy of 64-82% using machine learning algorithms with leave-one-out cross-validation. Our findings support the utility of automated speech analysis to aid clinician diagnosis, monitoring and understanding of schizophrenia.

Negative symptoms of schizophrenia are often associated with the blunting of emotional affect which creates a serious impediment in the daily functioning of the patients. Affective prosody is almost always adversely impacted in such cases, and is known to exhibit itself through the low-level acoustic signals of prosody. To automate and simplify the process of assessment of severity of emotion related symptoms of schizophrenia, we utilized these low-level acoustic signals to predict the expert subjective ratings assigned by a trained psychologist during an interview with the patient. Specifically, we extract acoustic features related to emotion using the openSMILE toolkit from the audio recordings of the interviews. We analysed the interviews of 78 paid participants (52 patients and 26 healthy controls) in this study. The subjective ratings could be accurately predicted from the objective openSMILE acoustic signals with an accuracy of 61-85% using machine-learning algorithms with leave-one-out cross-validation technique. Furthermore, these objective measures can be reliably utilized to distinguish between the patient and healthy groups, as supervised learning methods can classify the two groups with 79-86% accuracy.

Learning couple dance such as Salsa is a challenge for modern human as it requires to assimilate and understand correctly all the required parameters. In this paper, we propose a set of music-related motion features (MMF) allowing to describe, analyse and classify salsa dancer couple in their respective learning state (beginner, intermediate and expert). These dance qualities have been proposed from a systematic review of papers cross linked with interviews from teacher and professionals in the field of social dance. We investigated how to extract these MMF from musical data and 3D movements of dancers in order to propose a new algorithm to compute them. For the presented study, a motion capture database (SALSA) has been recorded of 26 different couples with varying skill levels dancing on 10 different tempos (260 clips). Each recorded clips contains a basic steps sequence and an extended improvisation sequence during two minutes in total at 120 frame per second. We finally use our proposed algorithm to analyse and classify these 26 couples in three learning levels, which validates some proposed music-related motion features and give insights on others.

This issue contains six papers.In the first paper, Kim J. L. Nevelsteen from Stockholm University proposes to sample technologies using grounded theory and obtained a definition for a "virtual world" that is directly applicable to technology.The obtained definition is compared with related work and used to classify advanced technologies, such as a pseudo-persistent video game, a MANet, a virtual and mixed reality, and the Metaverse.The results of this article include: a breakdown of properties that set apart various technologies; a definition that is validated by comparing it with other definitions; an ontology showing the relation of different complimentary terms and acronyms; and, the usage of pseudo-persistence to categories of those technologies that only mimic persistence.

This issue contains five papers. In the first paper, Bartlomiej Waszak, from Quebec City, Canada, proposes a novel approach to limbless movement simulation. The author built a simulation framework using position-based dynamics. He describes the body configuration of snakes using different types of distance constraints. The limbless movement is based on the formulation of a friction constraint to model the behavior of a snake's scales. In this approach, it is easy to solve collisions between objects and self-collisions for simulated snakes. The model includes a dynamic geometrical environment colliding with simulated animals. Detailed patterns are presented for four main types of limbless movement: serpentine, rectilinear, concertina, and sidewinding. In the second paper, Min Jiang, Richard Southern, and Jian J Zhang, from Bournemouth University, UK, present a novel and unified particle-based method for real-time dissolution simulation, which is fast, predictable, independent of sampling resolution, and visually plausible. The dissolution model is derived from collision theory and integrated into a smoothed particle hydrodynamics (SPH) fluid solver. Dissolution occurs when a solute is submerged in solvent. Physical laws govern the local excitation of solute particles based on kinetic energy. Solute separation during dissolution is handled using a new GPU-based region growing method. The use of SPH sampling for both solute and solvent guarantees a predictable and smooth dissolution process and provides user control of the volume change during the phase transition. In the third paper. Ashish Dhiman, Dhaval Solanki, and Uttama Lahiri, from The Indian Institute of Technology Gandhinagar in Gujarat, India; Ashu Bhasin, from All India Institute of Medical Sciences, in Delhi, India; and Abhijit Das, from AMRI Institute of Neuroscience, in Kolkata, India, describe the design of a novel, multimodal, virtual-reality-based, and performance-sensitive exercise platform that can intelligently adapt its task presentation to one's performance. Here, the authors aim to address unilateral shoulder abduction and adduction that are essential for the performance of daily living activities. They designed an experimental study in which six chronic stroke survivors participated. While they interacted with the virtual-reality-based tasks, the authors recorded the physiological signals in a synchronized manner. In the fourth paper, Xiaoyong Zhang and Shiguang Liu, from Tianjin University, China, propose a novel interactive SPH fluid control framework with turbulent details. The authors run SPH fluid simulation on CUDA and greatly improve the efficiency of fluid control. The control particle with curvature framework was adapted in this paper. The authors specially designed spring forces to make the fluid match a fast moving control target. Moreover, fine fluid details were preserved by separately calculating the under-control fluid turbulence and the free fluid turbulence. This improved SPH fluid control can run in real time, which can enhance the visual quality of fluid animation as well. In the fifth paper, Dayal R.Parhi, Chinmaya Sahu, and Priyadarshi Biplab Kumar, from The National Institute of Technology Rourkela in Orissa, India, present a navigation strategy for humanoid robots using a hybridized technique consisting of adaptive particle swarm optimization and adaptive ant colony optimization. Here, the governing parameters of the adaptive ant colony optimization technique are optimized by using adaptive particle swarm optimization method. These optimized parameters are subsequently used by the adaptive particle swarm optimization technique to get the final turning angle by which the humanoid navigates in a cluttered environment. Here, navigation is performed in both static and dynamic environments. To avoid the intercollision among the humanoids, a Petri net controller has been designed and implemented along with the proposed hybridized method.

This issue contains six papers.In the first paper, Christos Kyrlitsias and Despina Michael-Grigoriou, from the Cyprus University of Technology in Limassol, Cyprus, investigate the conformity to virtual humans in an immersive virtual environment using two experiments.In the first experiment, they study whether agents have social influence on the participants by conducting the Asch conformity experiment.In the second experiment, they use a similar method to study how the factors "agency" and "behavioral realism" affect social conformity.The results of the experiment show that conformity can be caused by virtual humans in immersive virtual environments.

Negative symptoms of schizophrenia significantly affect the daily functioning of patients, especially movement and expressive gestures. The diagnosis of such symptoms is often difficult and require the expertise of a trained clinician. Apart from these subjective methods, there is little research on developing objective methods to quantify the symptoms. Therefore, we explore body movement signals as objective measures of negative symptoms. Specifically, we extract the signals from video recordings of patients being interviewed. We analysed the interviews of 69 paid participants (46 patients and 23 healthy controls) in this study. Correlation between movement signals (linear and angular speeds of upper limbs and head, acceleration and gesture angles) and subjective ratings (assigned during same interview) from the NSA-16 scale were calculated. As hypothesized, the movement signals correlated strongly with the movement impairment aspect of the NSA-16 questionnaire. Also, not quite surprisingly, strong correlations were obtained between the movement signals and speech items of NSA-16, indicating lack of associated gestures in patients during speech. These subjective ratings could also be reasonably predicted from the objective signals with an accuracy of 61-78% using machine-learning algorithms with leave-one-out cross-validation technique. Furthermore, these objective measures can be reliably utilized to distinguish between the patient and healthy groups, as supervised learning methods can classify the two groups with 74-87% accuracy.

course Share on Real humans with virtual humans and social robots interactions (HCI) Authors: Daniel Thalmann EPFL, Switzerland EPFL, SwitzerlandView Profile , Nadia Magnenat Thalmann NTU, Singapore NTU, SingaporeView Profile , Manoj Ramanathan NTU, Singapore NTU, SingaporeView Profile Authors Info & Claims SA '17: SIGGRAPH Asia 2017 CoursesNovember 2017 Article No.: 15Pages 1–221https://doi.org/10.1145/3134472.3134513Published:27 November 2017Publication History 0citation237DownloadsMetricsTotal Citations0Total Downloads237Last 12 Months26Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This issue contains five papers. In the first paper, Angel Zaldivar Pino, Manuel Gonzalez Bedia, and Francisco J. Seron, from Universidad Zaragoza, Spain, present a simulation framework in which the agents are endowed with a sensing device, an oscillator network as controller, and actuators to interact with the environment. The perception device is designed as an optic array emulating the principles of the animal retina, which assimilates stimuli resembling optic flow to be captured from the environment. The controller modulates informational variables to action variables in a sensory motor flow. Their approach is based on the Kuramoto model that describes mathematically a network of coupled phase oscillators and the use of evolutionary algorithms, which is proved to be capable of synthesizing minimal synchronization strategies based on the dynamical coupling between agents and environment. In the second paper, Congkun Chen, Yun Sheng, Fang Li, Guixu Zhang, from East China Normal University, Shanghai, China, and Hassan Ugail, from University of Bradford, UK, propose an extension of the use of the partial differential equation (PDE) method to head visualization with CT data and show how the two primary medical visualization means, surface reconstruction, and volume rendering can be integrated into one single framework through PDEs. Their scheme first performs head segmentation from CT slices using a variational approach; with the extracted boundary conditions, head surface reconstruction is then executed. Since only a few slices are used, their method can perform head surface reconstruction more efficiently in both computational time and storage cost than the widely used marching cubes algorithm. By elaborately introducing a third parameter ω to the PDE method, a solid head can be created, based on which the head volume is subsequently rendered with 3-D texture mapping. In the third paper, Francis Laclé, Utrecht University, the Netherlands, and Nicolas Pronost, Université Claude Bernard, Lyon, France, focus on combining, by a controllable enhancement process, a functional and biomechanical model of musculotendon units with its high-resolution geometrical counterpart. The method was developed in order to be invariant to spatial and polygonal configurations and to be scalable in both longitudinal and latitudinal directions. Results with 48 musculotendon units for the lower body show a drop of 84% with respect to the number of vertices when compared to the high-resolution model, while maintaining the functional information. A real-time simulation experiment resulted in a runtime of 135 Hz. Sergio Orts-Escolano, from University of Alicante, Spain, Johannes Bastiaan Boom, Xin Xin Ning, Steven McDonagh, Peter Sandilands, and Robert B. Fisher, from University of Edinburgh, UK, propose in the fourth paper, the first hybrid CPU-GPU based method for estimating a point light source position in a scene recorded by an RGB-D camera. The image and depth information from the Kinect is enough to estimate a light position in a scene, which allows for the rendering of synthetic objects into a scene that appear realistic enough for augmented reality purposes. This method does not require a light probe or other physical device. To make this method suitable for augmented reality, the authors developed a hybrid implementation that performs light estimation in less than 1 s. By rendering synthetic objects into the recorded scene, they illustrate that this accuracy is good enough for the rendered objects to look realistic. Finally, Jorge Eduardo Ramirez Flores and Antonio Susin Sanchez, from Universitat Politecnica de Catalunya, Barcelona, Spain, present an approach based on mesh segmentation for skinning and skeleton-driven computer animation. They propose a novel and fast method, based in watershed segmentation to deal with characters in T-Pose and arbitrary poses; they also introduce a simple weight assign algorithm based in the rigid skinning obtained with the segmentation algorithm for the linear blend skinning (LBS) deformation method. They ultimately propose a modified version of the LBS that avoids the loss of volume in twist rotations using the segmentation stage output values.

This issue contains six papers. In the first paper, Jong-Hyun Kim and Chang-Hun Kim, Korea University, Seoul, and Jung Lee, Hallym University, Gangwon-do, Republic of Korea, present a new method for the fast simulation of interactions between fluids and solids by incorporating particle-based water flow into an adaptive signed distance field (ADF). This paper addresses the problem of “tunneling” artifact, in which particles with high velocities skip across the layer of leaf nodes by (i) considering particles only on the leaf nodes in the adaptive structure to improve the processing time required for the water–solid coupling and (ii) considering the water flow to avoid the tunneling artifact by incorporating particle motion into the tree structure of the ADF. In the second paper, Rodrigo Guillermo Baravalle and Leonardo Scandolo, Universidad Nacional de Rosario, Argentina; Claudio Delrieux, Universidad Nacional del Sur, Buenos Aires, Argentina; Cristian García Bauza, Universidad Nacional del Centro de la Provincia de Buenos Aires, Argentina; and Elmar Eisemann, Delft University of Technology, The Netherlands, propose an algorithm for the procedural generation of porous materials based on a simulation of the growth of self-avoiding bubbles inside a volume, by means of dynamical systems. This simulation generates 3D textures that adequately represent porous materials, which are then rendered by means of a GPU-based direct volume rendering method. The patterns induced by the bubbles can be intuitively controlled. The bubbles adapt to any given shape and have convincing global and local fluid-like patterns as seen in bread and sponges. In the third paper, Andre Possani-Espinosa, J. Octavio Gutierrez-Garcia, and Isaac Vargas Gordillo, Instituto Tecnológico Autónomo de México, lay the foundations for the design of believable virtual drivers by proposing a methodology for profiling players using the open racing car simulator. Data collected from 125 players about their driving behaviors and personality traits give insights into how personality traits should model the behavior of believable virtual drivers. The data analysis was conducted using a correlation analysis and the J48 decision tree algorithm. In addition, this work also (i) gives preliminary insights into the relationship between the driving behavior and personality of racing game players and actual car drivers and (ii) presents evidence of the relevance of gender as a predictor of personality traits of racing game players. Monssef Alsweis, Universitat Konstanz, Germany, suggests, in the fourth paper, a procedural biologically motivated method to simulate the development of leaf contours and the generation of different levels of leaf venation systems. Visually realistic development is described by a growth function RERG that reacts to hormone (Auxin) sources embedded in the leaf blade. The shape of the leaf is determined by a set of feature points at the leaf contour. The contour is extracted from images utilizing a Curvature Scale Space Corner Detection Algorithm. Auxin transport is described by an initial Auxin flux from a source to a sink that is gradually channelized into cells with high levels of highly polarized transporters. The proposed model simulates a wide range of leaf forms, from simple shapes to lobed leaves. In the fifth paper, Kun Qian, Xiaosong Yang, and Jian J Zhang, Bournemouth University, UK, and Junxuan Bai and Junjun Pan, Beihang University, China, propose a set of tailored key technologies for laparoscopic surgery simulation, ranging from the simulation of soft tissues with different properties, the interactions between surgical tools and soft tissues, and the rendering of complex anatomical environment. Compared with the current methods, their tailored algorithms aimed at improving the performance from accuracy, stability, and efficiency perspectives. The authors also abstract and design a set of intuitive parameters that can provide developers with high flexibility to develop their own simulators. Finally, Dele Cui, Yun Sheng, and Guixu Zhang, East China Normal University, Shanghai, China, present an image-based method to simulate the traditional embroidery art. The method combines stroke-based rendering techniques with the Phong lighting model to create picturesque embroidery-like images. They first build a 3D stitch model and derive some most commonly used stitch patterns from it. Then they preprocess the input image by segmenting it into regions, from which the parameters to specify stitch patterns are obtained. Finally, they apply stitches back onto the desired regions and render them under a virtual light source. Experimental results show that their method, different from the existing schemes, is capable of performing fine embroidery simulations with the effects of lighting and shading based on an input image.

This issue contains six papers. In the first paper, Hui Liang, Jian Chang, and Shujie Deng, from the National Centre for Computer Animation in Bournemouth, UK; Can Chen from Changzhou University, China; Ruo-feng Tong, from Zhejiang University, Hangzhou, China; and Jian J. Zhang, from Bournemouth University, UK, design an immersive storytelling environment that allows multiple players to use naturally interactive hand gestures to manipulate virtual puppetry for assisting narration. A set of multimodal interaction techniques is presented for a hybrid user interface that integrates existing 3D visualization and interaction devices including head-mounted displays and depth motion sensor. In the second paper, Sybren A. Stüvel, Frank van der Stappen, and Arjan Egges, from Universiteit Utrecht, The Netherlands, present an investigation into the accuracy of human observers with regard to the recognition of collisions between virtual characters. They show the result of two user studies, where participants classify scenarios as “colliding” or “not colliding”; a pilot study investigates the perception of static images, whereas the main study expands on this by employing animated videos. In the pilot experiment, they investigated the effect of two variables on the ability to recognise collisions: distance between the character meshes, and visibility of the inter-character gap. In the main experiment, they investigate the angle between the character paths and the severity of the (near) collision. In the third paper, Marios Andreas Kyriakou, from the University of Cyprus; Xueni Pan, from the University of London, UK; and Yiorgos Lambros Chrysanthou, from the University of Cyprus examine attributes of virtual human behavior that may increase the plausibility of a simulated crowd and affect the user's experience in Virtual Reality (VR). Purpose-developed experiments in both Immersive and semi-Immersive VR systems queried the impact of collision and basic interaction between real users and the virtual crowd and their effect on the apparent realism and ease of navigation within VR. Participants' behavior and subjective measurements indicated that facilitating collision avoidance between the user and the virtual crowd makes the virtual characters, the environment, and the whole VR system appear more realistic and lifelike. In the fourth paper, Mihai Polceanu and Cedric Buche, from Florida International University, and Lab-STICC - ENIB, CERV, Plouzané, France, introduce a study of existing approaches that explicitly use mental simulation. Current implementations of the mental simulation paradigm, taken together, computationally address many aspects suggested by cognitive science research. Agents are able to find solutions to nontrivial scenarios in virtual or physical environments. Existing systems also learn new behavior by imitation of others similar to them and model the behavior of different others with the help of specialized models, culminating with the collaboration between agents and humans. Approaches that use self models are able to mentally simulate interaction and to learn about their own physical properties. In the fifth paper, Yuxing Qiu, Lipeng Yang, Shuai Li, and Qing Xia, from Beihang University, Beijing, China; Hong Qin, from Stony Brook University, USA; and Aimin Hao, from Beihang University, Beijing, China, advocate a method for the modeling and enhancement of scale-sensitive fluid details. The core of their method is the coupling of multilayer depth regression analysis and FLIP fluid simulation. First, they capture the depth buffer of the fluid surface from the top of scene. Second, they employ depth peeling to decompose the target fluid volume into multiple layers and conduct time–space analysis over surface layers. Third, they propose a logistic regression-based model to pinpoint the interacting regions, wherein multiple detail-relevant factors are taken into account. Finally, details are enhanced by animating extra diffuse materials and augmenting the air–fluid mixing phenomenon. Abdullah Bulbul, from Yildirim Beyazit Universitesi, Ankara, Turkey, and Rozenn Dahyot, from Trinity College, Dublin, Ireland, propose to automatically populate geo-located virtual cities by harvesting and analyzing online contents shared on social networks and websites. They show how pose and motion paths of agents can be realistically rendered using information gathered from social media. 3D cities are automatically generated using open source information available online. To provide a final rendering of both static and dynamic urban scenes, they use Unreal game engine.

from the University of Zaragoza, Spain, propose a "dynamic agent" endowed with a sensing device, a controller, and actuators to interact with the environment.The control architecture is based on ordinary differential equations with the function of modulating the stimulus signals to action signals under a sensory-motor flow.The parameter values are defined in an evolutionary process depending on the task to be performed.A series of experiments are presented to illustrate certain qualities of their model such as the adaptability to change, a highly intuitive and flexible design methodology, and a high degree of individual autonomy.

Abstract Positioning virtual garments onto a 3D avatar is known to be a time‐consuming process. This task is usually done manually if the proportion of the garment needs to be kept during the process for a fitting purpose. The positioning of the garment is currently the main limitation to propose a fully automatic virtual try‐on application. This paper presents an automatic method based on surface energy minimisation that allows fully automatically positioning of a 3D virtual cloth without deformation of the patterns. The minimum of surface energy is reached by animating the humanoid avatar. Humanoid joints are manipulated to place a 3D avatar into the virtual garment. The proposed method, as well as its implementation, is described in this paper. image

With increasing resolutions and number of acquisitions, medical imaging more and more requires computer support for interpretation as currently not all imaging data is fully used. In our work, we show how multi-channel images can be used for robust air masking and reliable muscle tissue detection in the human lower limb. We exploit additional channels that are usually discarded in clinical routine. We use the common mDixon acquisition protocol for MR imaging. A series of thresholding, morphological, and connectivity operations is used for processing. We demonstrate our fully automated approach on four subjects and present a comparison with manual labeling. We discuss how this work is used for advanced and intuitive visualization, the quantification of tissue types, pose estimation, initialization of further segmentation methods, and how it could be used in clinical environments.

Low-rank matrix approximation (LRMA) is a powerful technique for signal processing and pattern analysis. However, its potential for data compression has not yet been fully investigated. In this paper, we propose sparse LRMA (SLRMA), an effective computational tool for data compression. SLRMA extends conventional LRMA by exploring both the intra and inter coherence of data samples simultaneously. With the aid of prescribed orthogonal transforms (e.g., discrete cosine/wavelet transform and graph transform), SLRMA decomposes a matrix into a product of two smaller matrices, where one matrix is made up of extremely sparse and orthogonal column vectors and the other consists of the transform coefficients. Technically, we formulate SLRMA as a constrained optimization problem, i.e., minimizing the approximation error in the least-squares sense regularized by the $\ell _{0}$ -norm and orthogonality, and solve it using the inexact augmented Lagrangian multiplier method. Through extensive tests on real-world data, such as 2D image sets and 3D dynamic meshes, we observe that: 1) SLRMA empirically converges well; 2) SLRMA can produce approximation error comparable to LRMA but in a much sparse form; and 3) SLRMA-based compression schemes significantly outperform the state of the art in terms of rate-distortion performance.

Sound-event classification often utilizes time-frequency analysis, which produces an image-like spectrogram. Recent approaches such as spectrogram image features and subband power distribution image features extract the image local statistics such as mean and variance from the spectrogram. They have demonstrated good performance. However, we argue that such simple image statistics cannot well capture the complex texture details of the spectrogram. Thus, we propose to extract the local binary pattern (LBP) from the logarithm of the Gammatone-like spectrogram. However, the LBP feature is sensitive to noise. After analyzing the spectrograms of sound events and the audio noise, we find that the magnitude of pixel differences, which is discarded by the LBP feature, carries important information for sound-event classification. We thus propose a multichannel LBP feature via pixel difference quantization to improve the robustness to the audio noise. In view of the differences between spectrograms and natural images, and the reliability issues of LBP features, we propose two projection-based LBP features to better capture the texture information of the spectrogram. To validate the proposed multichannel projection-based LBP features for robot hearing, we have built a new sound-event classification database, the NTU-SEC database, in the context of social interaction between human and robot. It is publicly available to promote research on sound-event classification in a social context. The proposed approaches are compared with the state of the art on the RWCP database and the NTU-SEC database. They consistently demonstrate superior performance under various noise conditions.

We present a novel dense crowd simulation method. In real crowds of high density, people manoeuvring the crowd need to twist their torso to pass between others. Our proposed method does not use the traditional disc-shaped agent, but instead employs capsule-shaped agents, which enables us to plan such torso orientations. Contrary to other crowd simulation systems, which often focus on the movement of the entire crowd, our method distinguishes between active agents that try to manoeuvre through the crowd, and passive agents that have no incentive to move. We introduce the concept of a focus point to influence crowd agent orientation. Recorded data from real human crowds are used for validation, which shows that our proposed model produces equivalent paths for 85 percent of the validation set. Furthermore, we present a character animation technique that uses the results from our crowd model to generate torso-twisting and side-stepping characters.

In this paper, we Secdescribe an agency model for generative populations of humanoid characters, based upon temporal variation of affective states. We have built on an existing agent framework from Sequeira et al. [18], and adapted it to be susceptible to temperamental and emotive states in the context of cooperative and non-cooperative interactions based on trading activity. More specifically, this model operates within two existing frameworks: a) intrinsically motivated reinforcement learning, structured upon affective appraisals in the relationship of the agents with their environment [20,18]; b) a multi-temporal representation of individual psychology, common in the field of affective computing, structuring individual psychology as a tripartite relationship: emotions-moods-personality [8,16]. Results show a populations of agents that express their individuality and autonomy with a high level of heterogeneous and spontaneous behaviors, while simultaneously adapting and overcoming their perceptual limitations.

This paper presents an immersive volleyball game, where a player plays not only against virtual opponents but also with support on his/her side of virtual teammates. This volleyball game has been implemented for several 3D displays such as a stereoscopic display, an autostereoscopic display, Oculus Rift glasses, and a 320o Immersive Room. In this paper, we also propose a user study of the relations between virtual humans and the sense of presence in the different 3D displays. We particularly study how surrounding virtual humans affect the sense of presence. Results show that users more significantly perceived spatial presence of virtual environment and social presence of virtual humans with the Oculus Rift and the Immersive Room.

In the coming era of social companions, many researches have been pursuing natural dialog interactions and long-term relations between social companions and users. With respect to the quick decrease of user interests after the first few interactions, various emotion and memory models are developed and integrated with social companions for better user engagement. This paper reviews related works in the effort of combining memory and emotion with natural language dialog on social companions. We separate these works into three categories: (1) Affective system with dialog, (2) Task-driven memory with dialog, (3) Chat-driven memory with dialog. In addition, we discussed limitations and challenging issues to be solved. Finally, we also introduced our framework of social companions.

Negative symptoms in schizophrenia are associated with significant burden and functional impairment, especially speech production. In clinical practice today, there are no robust treatments for negative symptoms and one obstacle surrounding its research is the lack of an objective measure. To this end, we explore non-verbal speech cues as objective measures. Specifically, we extract these cues while schizophrenic patients are interviewed by psychologists. We have analyzed interviews of 15 patients who were enrolled in an observational study on the effectiveness of Cognitive Remediation Therapy (CRT). The subject (undergoing CRT) and control group (not undergoing CRT) contains 8 and 7 individuals respectively. The patients were recorded during three sessions while being evaluated for negative symptoms over a 12-week follow-up period. In order to validate the non-verbal speech cues, we computed their correlation with the Negative Symptom Assessment (NSA-16). Our results suggest a strong correlation between certain measures of the two rating sets. Supervised prediction of the subjective ratings from the non-verbal speech features with leave-one-person-out cross-validation has reasonable accuracy of 53-80%. Furthermore, the non-verbal cues can be used to reliably distinguish between the subjects and controls, as supervised learning methods can classify the two groups with 80-93% accuracy.

Due to the growing needs of motion capture (mocap) in movie, video games , sports, etc., it is highly desired to compress mocap data for efficient storage and transmission. Unfortunately, the existing compression methods have either high latency or poor compression performance , making them less appealing for time-critical applications and/or network with limited bandwidth . This paper presents two efficient methods to compress mocap data with low latency. The first method processes the data in a frame-by-frame manner so that it is ideal for mocap data streaming. The second one is clip-oriented and provides a flexible trade-off between latency and compression performance . It can achieve higher compression performance while keeping the latency fairly low and controllable. Observing that mocap data exhibits some unique spatial characteristics , we learn an orthogonal transform to reduce the spatial redundancy . We formulate the learning problem as the least square of reconstruction error regularized by orthogonality and sparsity , and solve it via alternating iteration. We also adopt a predictive coding and temporal DCT for temporal decorrelation in the frame- and clip-oriented methods, respectively. Experimental results show that the proposed methods can produce higher compression performance at lower computational cost and latency than the state-of-the-art methods. Moreover, our methods are general and applicable to various types of mocap data.

This issue contains four papers. In the first paper, Mingfei Wang, Jinyuan Jia, Ning Xie, and Chenxi Zhang, from Tongji University in Shanghai, China, aim to solve the challenging problems of choosing the data supplier for data-dispatching services in distributed virtual environments based on peer-to-peer networks. They propose a dynamic node-organizing mechanism that aims by applying the avatar's behavioral characteristics to the neighbor maintenance mechanism and scene data transmission. They have conducted extensive simulation experiments that simulate avatar behaviors in a popular online game. The results show that their proposed mechanism achieved a substantial alleviation of neighbor churn and reduced information exchange, which improves the transmission efficiency in DVEs. In the second paper, Masaki Oshita, from the Kyushu Institute of Technology, Japan, proposes an interactive character motion control interface that uses hands. Using their hands and fingers, the user can control a large number of degrees of freedom (DOFs) at the same time. The author applied principal component analysis (PCA) to a set of sample poses and assigned the extracted principal components to each DOF of the hands (such as the hand positions and finger bending/extending angles). The author developed methods for computing the feature vector, for applying PCA, and for pose and action synthesis. In addition, he introduced a pose transition method for performing a step motion when necessary to prevent foot sliding. He presents his experimental results and demonstrates the effectiveness of his interface. In the third paper, Wenwu Yang, Xun Wang, Wangbin Kou, Bailin Yang, and Guozheng Wang, from Zhejiang Gongshang University, Hangzhou, China, present a topology-aware method based on moving least squares (MLS) deformation approach for 2D characters. First, a Laplace equation is solved to obtain a set of weights, which are called as harmonic weights. Then, the MLS deformation is performed by using the harmonic weights as the deformation influence of the user-specified controls. Finally, the possible distortion in the traditional MLS deformation can be effectively avoided, since the harmonic weights spread the deformation of the controls in a localized and topology-aware way. In addition, a simple but effective area-preserving variant of MLS deformation is proposed, which is suitable for the editing of incompressible objects. Mohammadali Hajizadeh and Hossein Ebrahimnezhad, from Sahand University of Technology, Islamic Republic of Iran, propose a key frame-based technique for 3D dynamic mesh compression. First, key frames are extracted from the animated sequence. Extracted key frames are then linearly combined using blending weights to predict the vertex locations of the other frames. These blending weights play a key role in the proposed algorithm since the prediction performance and the required number of key frames are greatly dependent on these weights. They present a novel method in order to compute the optimum blending weight, which makes it possible to predict location of the vertices of the non-key frames with the minimum number of key frames. The residual prediction errors are finally quantized and encoded using Huffman coding and another heuristic method. Experimental results on different test sequences with various sizes, topologies, and geometries demonstrate the privileged performance of the proposed method compared to the previous techniques.

Abstract Understanding emotional human behavior in its multimodal and continuous aspect is necessary for studying human machine interaction and creating constituent social agents. As a first step, we propose a system for continuous emotional behavior recognition expressed by people during communication based on their gesture and their whole body dynamical motion. The features used to classify the motion are inspired by the Laban Movement Analysis entities and are mapped onto the well‐known Russell Circumplex Model . We choose a specific case study that corresponds to an ideal case of multimodal behavior that emphasizes the body motion expression: theater performance. Using a trained neural network and annotated data, our system is able to describe the motion behavior as trajectories on the Russell Circumplex Model diagram during theater performances over time. This work contributes to the understanding of human behavior and expression and is a first step through a complete continuous emotion recognition system whose next step will be adding facial expressions. Copyright © 2016 John Wiley & Sons, Ltd.

A human-computer interface (namely Facial position and expression Mouse system, FM) for the persons with tetraplegia based on a monocular infrared depth camera is presented in this paper. The nose position along with the mouth status (close/open) is detected by the proposed algorithm to control and navigate the cursor as computer user input. The algorithm is based on an improved Randomized Decision Tree, which is capable of detecting the facial information efficiently and accurately. A more comfortable user experience is achieved by mapping the nose motion to the cursor motion via a nonlinear function. The infrared depth camera enables the system to be independent of illumination and color changes both from the background and on human face, which is a critical advantage over RGB camera-based options. Extensive experimental results show that the proposed system outperforms existing assistive technologies in terms of quantitative and qualitative assessments.

Localizing, identifying, and extracting humans with consistent appearance jointly from a personal photo stream is an important problem and has wide applications. The strong variations in foreground and background and irregularly occurring foreground humans make this realistic problem challenging. Inspired by advancements in object detection, scene understanding, and image cosegmentation, we explore explicit constraints to label and segment human objects rather than other nonhuman objects and “stuff.” We refer to such a problem as multiple human identification and cosegmentation (MHIC). To identify specific human subjects, we propose an efficient human instance detector by combining an extended color line model with a poselet-based human detector. Moreover, to capture high-level human shape information, a novel soft shape cue is proposed. It is initialized by the human detector, then further enhanced through a generalized geodesic distance transform, and finally refined with a joint bilateral filter. We also propose to capture the rich feature context around each pixel by using an adaptive cross-region data structure, which gives a higher discriminative power than a single pixel-based estimation. The high-level object cues from the detector and the shape are then integrated with the low-level pixel cues and midlevel contour cues into a principled conditional random field (CRF) framework, which can be efficiently solved by using fast graph cut algorithms. We evaluate our method over a newly created NTU-MHIC human dataset, which contains 351 images with manually annotated groundtruth segmentation. Both visual and quantitative results demonstrate that our method achieves state-of-the-art performance for the MHIC task.

This paper presents a simple yet effective algorithm for compressing human motion capture (mocap) data. With a reordering-based discrete wavelet transform and the standard discrete cosine transform, our method can effectively reduce the spatial and temporal correlation in mocap data. Our method is conceptually simple and easy to implement. Experimental results show that our method can achieve better compression performance with lower latency, compared to the state-of-the-art methods.

We present a demonstration of a multi-modal 3D capturing platform coupled to a motion comparison system. This work is focused on the preservation of Traditional Sports and Games, namely the Gaelic sports from Ireland and Basque sports from France and Spain. Users can learn, compare and compete in the performance of sporting gestures and compare themselves to real athletes. Our online gesture database provides a way to preserve and display a wide range of sporting gestures. The capturing devices utilised are Kinect 2 sensors and wearable inertial sensors, where the number required varies based on the requested scenario. The fusion of these two capture modalities, coupled to our inverse kinematic algorithm, allow us to synthesize a fluid and reliable 3D model of the user gestures over time. Our novel comparison algorithms provide the user with a performance score and a set of comparison curves (i.e. joint angles and angular velocities), providing a precise and valuable feedback for coaches and players.

Wearable devices such as Microsoft Hololens and Google glass are highly popular in recent years. As traditional input hardware is difficult to use on such platforms, vision-based hand pose tracking and gesture control techniques are more suitable alternatives. This demo shows the possibility to interact with 3D contents with bare hands on wearable devices by two Augmented Reality applications, including virtual teapot manipulation and fountain animation in hand. Technically, we use a head-mounted depth camera to capture the RGB-D images from egocentric view, and adopt the random forest to regress for the palm pose and classify the hand gesture simultaneously via a spatial-voting framework. The predicted pose and gesture are used to render the 3D virtual objects, which are overlaid onto the hand region in input RGB images with camera calibration parameters for seamless virtual and real scene synthesis.

Abstract The solid boundary handling has been a research focus in physically based fluid animation. In this paper, we propose a novel stable and fast particle method to couple predictive–corrective incompressible smoothed particle hydrodynamics and geometric lattice shape matching (LSM), which animates the visually realistic interaction of fluids and deformable solids allowing larger time steps or velocity differences. By combining the boundary particles sampled from solids with a momentum‐conserving velocity‐position correction scheme, our approach can alleviate the particle deficiency issues and prevent the penetration artefacts at the fluid–solid interfaces simultaneously. We further simulate the stable deformation and melting of solid objects coupled to smoothed particle hydrodynamics fluids based on a highly extended LSM model. In order to improve the time performance of each time step, we entirely implement the unified particle framework on GPUs using compute unified device architecture. The advantages of our two‐way fluid–solid coupling method in computer animation are demonstrated via several virtual scenarios.

Abstract Example‐based material allows simulating complex material behaviors in an art‐directed way. This paper presents a method for fast subspace integration for example‐based elastic material, which is suitable for real‐time simulation in computer graphics. At the core of the method is the formulation of a new potential using example‐based Green strain tensors. By using this potential, the deformation can be attracted towards the example‐based deformation feature space, the example weights can be explicitly obtained and the internal force can be decomposed into the conventional one and an additional one induced by the examples. The real‐time subspace integration is then developed with subspace integration costs independent of geometric complexity, and both the reduced conventional internal force and additional one being cubic polynomials in reduced coordinates. Experiments demonstrate that our method can achieve real‐time simulation while providing comparable quality with the prior art.

first paper, Changbo Wang, Qiang Zhang, and Fanlong Kong, from East China Normal University in Shanghai, China, present a highly effective method of modeling and animating debris flows with adaptive grid.The debris flow is modeled as Bingham plastic fluid with view-dependent adaptive grid to model the flow volume, and the boundless grids can cover the large scale region of debris flow.Then the mixed grids are built for confluent flows, and two-way coupling interaction between flows and environment is considered.After extracting the debris flow surface, adaptive surface tension combining wave particles equation is used to enhance the details, and sprays are generated by particles considering the interaction between two fluid volumes.

and Nathan Sturtevant from the

Abstract How to endow the virtual characters with personalized behavior patterns remains a challenging problem. Instead of heuristically designing behaviors for certain personalities, this paper bridges the gap between personalities and behaviors using a medium concept, mood, to make the behaviors of the characters consistent enough to convey their personalities, while flexible enough to make appropriate response in various situations. We propose a personality‐characterized mood dynamics model, in which the emotion weights are computed as a solution of a convex optimization problem that is constructed to make the overall mood approaches the personality after sufficient interactions. The convergence of the algorithm is demonstrated by numerical simulations. The implementation of the personality‐characterized mood dynamics model enables an emotion‐oriented virtual human, Sophie, to show personalized behaviors in the emotional interactions with users. Copyright © 2015 John Wiley & Sons, Ltd.

This paper presents keyframe-based geometry video (KGV), a novel framework for compressing 3-D human motion data by using geometry videos. Given a motion data encoded in a geometry video (GV) format, our method extracts the keyframes and produces a reconstruction matrix. Then it applies the video compression technique (e.g., H.264/Advanced Video Coding) to the reordered keyframes, which can significantly reduce the spatial and temporal redundancy in the KGV. We develop a rate distortion-based optimization algorithm to determine the parameters (i.e., the number of keyframes and quantization parameter) leading to optimal performance. Experimental results show that the proposed KGV framework significantly outperforms the existing GV techniques in terms of both the rate distortion performance and visual quality. Besides, the computational cost of the KGV is rather low at the decoder, making it highly desirable for power-constrained devices. Last but not least, our method can be easily extended to progressive compression with heterogeneous communication network.

Facing and pointing toward moving targets is a usual and natural behavior in daily life. Social robots should be able to display such coordinated behaviors in order to interact naturally with people. For instance, a robot should be able to point and look at specific objects. This is why, a scheme to generate coordinated head-arm motion for a humanoid robot with two degrees-of-freedom for the head and seven for each arm is proposed in this paper. Specifically, a virtual plane approach is employed to generate the analytical solution of the head motion. A quadratic program (QP)-based method is exploited to formulate the coordinated dual-arm motion. To obtain the optimal solution, a simplified recurrent neural network is used to solve the QP problem. The effectiveness of the proposed scheme is demonstrated using both computer simulation and physical experiments.

The elderly population is increasing rapidly all over the world. One major risk for elderly people is fall accidents, especially for those living alone. In this paper, we propose a robust fall detection approach by analyzing the tracked key joints of the human body using a single depth camera. Compared to the rivals that rely on the RGB inputs, the proposed scheme is independent of illumination of the lights and can work even in a dark room. In our scheme, a pose-invariant randomized decision tree algorithm is proposed for the key joint extraction, which requires low computational cost during the training and test. Then, the support vector machine classifier is employed to determine whether a fall motion occurs, whose input is the 3-D trajectory of the head joint. The experimental results demonstrate that the proposed fall detection method is more accurate and robust compared with the state-of-the-art methods.

Human motion capture (mocap) is a widely used technique for digitalizing human movements. With growing usage, compressing mocap data has received increasing attention, since compact data size enables efficient storage and transmission. Our analysis shows that mocap data have some unique characteristics that distinguish themselves from images and videos. Therefore, directly borrowing image or video compression techniques, such as discrete cosine transform, does not work well. In this paper, we propose a novel mocap-tailored transform coding algorithm that takes advantage of these features. Our algorithm segments the input mocap sequences into clips, which are represented in 2D matrices. Then it computes a set of data-dependent orthogonal bases to transform the matrices to frequency domain, in which the transform coefficients have significantly less dependency. Finally, the compression is obtained by entropy coding of the quantized coefficients and the bases. Our method has low computational cost and can be easily extended to compress mocap databases. It also requires neither training nor complicated parameter setting. Experimental results demonstrate that the proposed scheme significantly outperforms state-of-the-art algorithms in terms of compression performance and speed.

With increasing resolutions and number of acquisitions, medical imaging more and more requires computer support for interpretation as currently not all imaging data is fully used. In our work we show how multi-channel images can be used for robust air masking and reliable muscle tissue detection in human subjects. We exploit additional channels that are usually discarded in clinical routine. We use the common mDixon acquisition protocol for MR imaging. A series of thresholding, morphological and connectivity operations is used for processing. We demonstrate our approach on two subjects and present a comparison with manual labeling. We discuss how this work is used for advanced and intuitive visualization, the quantification of tissue types, pose estimation, initialization of further segmentation methods and how it could be used in clinical environments.

In this paper, we propose a practical, elegant and effective scheme for compact mocap data representation. Guided by our analysis of the unique properties of mocap data, the input mocap sequence is optimally segmented into a set of subsequences. Then, we project the subsequences onto a pair of computational orthogonal matrices to explore strong low-rank characteristic within and among the subsequences. The experimental results show that the proposed scheme is much more effective for reducing the data size, compared with the existing techniques.

Localizing, identifying and extracting human groups with consistent appearance jointly from a personal photo stream is an important problem and has wide applications. Inspired by recent advances in object detection, scene understanding and image cosegmentation, in this paper we explore explicit constraints to label and segment human objects rather than other non-human objects and “stuff”. We propose a novel soft human shape cue, which is initialized by color line poselet-based human part detection, further processed through a generalized geodesic distance transform, and refined finally with a joint bilateral filter. Such a high-level object cue is then integrated with other low-level unary and pairwise terms into a principled conditional random field framework, which can be efficiently solved by fast graph cut algorithms. We evaluate our algorithm over the FlickrMFC human dataset, and show that it achieves state-of-the-art performance for this challenging task.

Motion capture (mocap) technology is widely used in various applications. The acquired mocap data usually has missing data due to occlusions or ambiguities. Therefore, restoring the missing entries of the mocap data is a fundamental issue in mocap data analysis. Based on jointly low-rank matrix completion, this paper presents a practical and highly efficient algorithm for restoring the missing mocap data. Taking advantage of the unique properties of mocap data (i.e, strong correlation among the data), we represent the corrupted data as two types of matrices, where both the local and global characteristics are taken into consideration. Then we formulate the problem as a convex optimization problem, where the missing data is recovered by solving the two matrices using the alternating direction method of multipliers algorithm. Experimental results demonstrate that the proposed scheme significantly outperforms the state-of-the-art algorithms in terms of both the quality and computational cost.

Compression of 3D time-varying meshes (TVMs) plays a critical role in the storage and transmission of 3D contents. In this paper, we propose a novel framework for compressing 3D TVMs. In our framework, 3D TVMs are parameterized and represented by the geometry videos (GVs) through polycube parameterization. By considering the low-rank characteristic of dynamic meshes, we decompose GVs into a sequence with small frames namely EigenGV and the computed reconstruction matrix. We further apply 2D video encoder to eliminate spatial and temporal redundancy among the EigenGV. Experimental results demonstrate that the proposed method significantly outperforms the existing compression schemes in terms of both the rate distortion performance and visual quality. Besides, the proposed method naturally achieves progressive form, which is very suitable for error prone channel transmission.

This paper proposes a human computer interface using a single depth camera for quadriplegic people. The nose position is employed to control the cursor along with the commands provided by mouth's status. The detection of nose position and mouth's status is based on randomized decision tree algorithm.The experimental results show that the proposed interface is comfortable, easy to use, robust, and outperforms the existing assistive technology.

Telepresence refers to the possibility of feeling present in a remote location through the use of technology. This can be achieved by immersing a user to a place reconstructed in 3D. The reconstructed place can be captured from the real world or can be completely virtual. Another way to realize telepresence is by using robots and virtual avatars that act as proxies for real people. In case a human-mediated interaction is not needed or not possible, the virtual human and the robot can rely on artificial intelligence to act and interact autonomously. In this paper, these forms of telepresence are discussed, how they are related and different from each other and how autonomy takes place in telepresence. The paper concludes with an overview of the ongoing research on autonomous virtual humans and social robots conducted in the BeingThere centre.

Localizing, recognizing, and segmenting multiple foreground objects jointly from a general user's photo stream that records a specific event is an important task with many useful applications. As argued in recent Multiple Foreground Cosegmentation (MFC) work by Kim and Xing, this task is very challenging in that it contrasts substantially from the classical cosegmentation problem, and aims to parse a set of realistic event photos but each containing irregularly occurring multiple foregrounds with high appearance and scene configuration variations. Inspired by the impressive advance in scene understanding and object recognition, this paper casts the multiple foreground recognition and cosegmentation (MFRC) problem within a conditional random fields (CRFs) framework in a principled manner. We capitalize centrally on the key objective that MFRC is to segment out and annotate foreground objects or “things” rather than “stuff”. To this end, we exploit a few complementary objectness cues (e.g. contours, object detectors and layout) and propose novel and efficient methods to capture object-level information. Integrating object potentials as soft constraints (e.g. robust higher-order potentials defined over detected object regions) with low-level unary and pairwise terms holistically, we solve the MFRC task with a probabilistic CRF model. The inference for such a CRF model is performed efficiently with graph cut based move making algorithms. With a minimal amount of user annotations on just a few example photos, the proposed approach produces spatially coherent, boundary-aligned segmentation results with correct and consistent object labeling. Experiments on the FlickrMFC dataset justify that our method achieves state-of-the-art performance.

demonstrate a pipeline where humanoid 3D models can be incorporated within seconds into an animation system and infused with a wide range of capabilities, such as locomotion, object manipulation, gazing, speech synthesis, and lip syncing.They offer a set of heuristics that can associate arbitrary joint names with canonical ones and describe a fast retargeting algorithm that enables them to instill a set of behaviors onto an arbitrary humanoid skeleton on-the-fly.In the second paper, Mika Tapani Rantanen, and Martti Juhola, from the University of Tampere, Finland, examine how a path-planning problem can be solved in changing environments using probabilistic roadmap planners.They show that probabilistic roadmaps can be built in such a way that they work with dynamic obstacles; but as their experiments show, the roadmap graph must be built carefully.They compare three different methods that are used to connect the nodes of the roadmap graph.One of these is a distance-based method, which they present in this paper.

ABSTRACT Simulating a crowded scene like a busy shopping street requires tight packing of virtual characters. In such cases, collisions are likely to occur, and the choice in collision detection shape will influence how characters are allowed to intermingle. Full collision detection is too expensive for crowds, so simplifications are needed. The most common simplification, the fixed‐width, pose‐independent cylinder, does not allow intermingling of characters, as it will either cause too much empty space between characters or undetected penetrations. As a possible solution to this problem, we introduce the bounding cylinder hierarchy (BCH), a bounding volume hierarchy that uses vertical cylinders as bounding shapes. Because the BCH is a generalization of the single cylinder, we expect that this representation can be easily integrated with existing crowd simulation systems. We compare our BCH with commonly used collision shapes, namely the single cylinder and oriented bounding box tree, in terms of query time, construction time, and represented volume. To get an indication of possible crowd densities, we investigate how close characters can be before collision is detected and finally propose a critical maximum depth for the BCH. Copyright © 2014 John Wiley & Sons, Ltd.

Motion capture (mocap) technology is widely used in movie and game industries. Compact representation of the mocap data is critical to efficient storage and transmission. In this letter, we propose a novel tensor decomposition based scheme for compact and progressive representation of the mocap data. Our method segments and stacks the mocap sequence locally, and generates a 3rd-order tensor, which has strong correlation within and across slices of the tensor. Then, our method iteratively applies tensor decomposition in a multi-layer structure to explore the correlation characteristic. Experimental results demonstrate that the proposed scheme significantly outperforms existing algorithms in terms of scalability and storage requirement.

The rapid recent development of 3-DTV technology has led to an increase in studies on mesh-based 3-D scene representation. Compressing 3-D time-varying meshes is critical for the storage and transmission of 3-D contents. This paper proposes a highly efficient framework for compressing time-varying 3-D facial expressions. We use the near-isometric property of human facial expressions to parameterize the 3-D dynamic faces into an expression-invariant 2-D canonical domain that will naturally generate 2-D geometry videos (GVs). Considering the intrinsic properties of GVs, we apply low-rank and sparse matrix decomposition (LRSMD) separately to three dimensions of GVs (namely, \(X, Y,\) and \(Z\) ). Based on our high precision rate and distortion models for GVs, we further compress the components from LRSMD using a video encoder in which bitrates of all components are assigned optimally according to the target bitrate. Experimental results show that the proposed scheme can significantly improve compression performance in terms of rate-distortion performance and visual quality compared with the state-of-the-art algorithms.

Motion capture is widely used in sports, entertainment and medical applications. An important issue is to recover motion capture data that has been corrupted by noise and missing data entries during acquisition. In this paper, we propose a new method to recover corrupted motion capture data through trajectory-based sparse representation. The data is firstly represented as trajectories with fixed length and high correlation. Then, based on the sparse representation theory, the original trajectories can be recovered by solving the sparse representation of the incomplete trajectories through the OMP algorithm using a dictionary learned by K-SVD. Experimental results show that the proposed algorithm achieves much better performance, especially when significant portions of data is missing, than the existing algorithms.

Given any image input by users, how to automatically cutout the object-of-interest is a challenging problem due to lack of information of the object-of-interest and the background. Saliency detection techniques are able to provide some rough information about object-of-interest since they highlight high-contrast or high attention regions or pixels. However, the generated saliency map is often noisy and directly applying it for segmentation often leads to erroneous results. Motivated by the recent progress on image co-segmentation and internet image retrieval techniques, in this paper, we propose to use the user input image for segmentation as a query image to Google Images and then employ the top returned Google images to build up the knowledge about the object-of-interest in the user input image. Particularly, we develop a lightweight algorithm to learn the knowledge of the object-of-interest in the retrieved images to enhance the saliency map of the input image. Then, the enhanced saliency map is used to initialize the graph-cut to extract the object-of-interest. Experiments with the Mcgill dataset and multiple challenge cases demonstrate the effectiveness of our method in terms of producing a clean cutout.

Micro-blogging services, such as Twitter, and location-based social network applications have generated short text messages associated with geographic information, posting time, and user ids. The availability of such data received from users offers a good opportunity to study the user's spatial-temporal behavior and preference. In this paper, we propose a probabilistic model W4 (short for Who+Where+When+What) to exploit such data to discover individual users' mobility behaviors from spatial, temporal and activity aspects. To the best of our knowledge, our work offers the first solution to jointly model individual user's mobility behavior from the three aspects. Our model has a variety of applications, such as user profiling and location prediction; it can be employed to answer questions such as ``Can we infer the location of a user given a tweet posted by the user and the posting time?" Experimental results on two real-world datasets show that the proposed model is effective in discovering users' spatial-temporal topics, and outperforms state-of-the-art baselines significantly for the task of location prediction for tweets.

The availability of user check-in data in large volume from the rapid growing location based social networks (LBSNs) enables many important location-aware services to users. Point-of-interest (POI) recommendation is one of such services, which is to recommend places where users have not visited before. Several techniques have been recently proposed for the recommendation service. However, no existing work has considered the temporal information for POI recommendations in LBSNs. We believe that time plays an important role in POI recommendations because most users tend to visit different places at different time in a day, \eg visiting a restaurant at noon and visiting a bar at night. In this paper, we define a new problem, namely, the time-aware POI recommendation, to recommend POIs for a given user at a specified time in a day. To solve the problem, we develop a collaborative recommendation model that is able to incorporate temporal information. Moreover, based on the observation that users tend to visit nearby POIs, we further enhance the recommendation model by considering geographical information. Our experimental results on two real-world datasets show that the proposed approach outperforms the state-of-the-art POI recommendation methods substantially.

Compression of mesh-based 3-D models has been an important issue, which ensures efficient storage and transmission. In this paper, we present a very effective compression scheme specifically for expression variation 3-D face models. Firstly, 3-D models are mapped into 2-D parametric domain and corresponded by expression-invariant parameterizaton, leading to 2-D image format representation namely geometry images, which simplifies the 3-D model compression into 2-D image compression. Then, sparse representation with learned dictionaries via K-SVD is applied to each patch from sliced GI so that only few coefficients and their indices are needed to be encoded, leading to low datasize. Experimental results demonstrate that the proposed scheme provides significant improvement in terms of compression performance, especially at low bitrate, compared with existing algorithms.

Through a multitude of different devices, such as phones, tablets, desktop systems etc., we are able to exchange data across the world, independently of location, time and the device used. Almost by default applications are extended with networking capabilities, either deployed locally on the client device and connecting to a server or, as the trend is now, fully hosted on the Internet (servers) as a service (cloud services). However many 3D applications are still restricted to a single platform, as it is costly in terms of developing, maintaining, adapting and providing support for multiple platforms (software as well for hardware dependencies). Therefore applications that we see now available on a variety of devices are either single-platform, single-user, non-real-time collaborative or graphically not demanding. By using an adaptive remote rendering approach it is feasible to take advantage of these new devices and provide means for old and new 3D oriented applications to be used in collaborative environments. In this paper, we look at the conversion of a single user 3D application into a multi-user service. Analyse the requirements needed for adapting the software for being integrated into the "Herd framework". Offering remote rendering to end devices, a single instance accessible to multiple users and in order to optimize each instance of the application for different devices the user interface representation is handled in a dynamically using a device profile, as well for handling different input techniques.

Research on interactive virtual characters and social robots focuses mainly on one-to-one interactions and multi-party interactions concept are rather less explored. As we are developing these characters to be helpful to us in our daily lives as guides, companions, assistants or receptionists, they should be aware of the existence of multiple people and address their requirements in a natural way and act according to the social rules and norms. In contrast with previous work, we are interested in multi-party and multi-modal interactions between 3D virtual characters, real humans and social robots. This means that any of these participants can interact with each other. In this paper we present our on-going work, provide a discussion on multi-party interaction, describe the overall system architecture and mention our future work.

This issue is a special issue with selected papers from Motion in Games (MIG) 2012, which was held during November 15–17 in Rennes, France. Five papers were selected by a review committee composed of Paul Kry, McGill University; Rachel McDonnell, Trinity College Dublin; and Arjan Egges, Utrecht University. The review committee took into account not only the manuscripts but also the presentations and the potential for impact on the motion in games area that is being nurtured by MIG. This special issue contains four out of the five selected papers. The fifth paper will appear in the next issue of Computer Animation & Virtual Worlds. The first paper on this issue is from Peter Sandilands, Myung Geol Choi and Taku Komura, from the University of Edinburgh, UK. The authors propose a technique for action motion capture that allows them to capture an object's motion and geometry alongside a character's movement and local environment, using a magnetic motion capture system and a RGB-D sensor. Traditional methods of actor motion capture do not give any information about the spatial relationship between objects you may interact with, or are limited to large props and motions that are not occluded during capture. The proposed method not only gives greater information when placing a character in the scene, but enables the authors to digitally recreate the scene in motion without significant animator work after capture. The second paper by Junghyun Ahn, Stephane Gobron, Daniel Thalmann, and Ronan Boulic, from Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland, and NTU, Singapore, addresses emotional expressivity for embodied conversational agents by considering asymmetric facial expressions. The asymmetry of facial expressions helps to convey complex emotional feelings such as conflicting and/or hidden emotions due to social conventions. The proposed linear model can automatically drive a large number of autonomous virtual humans, or support the interactive design of complex facial expressions over time. The approach produces facial expressions for most of the emotional spectrum and it can also achieve more complex ambivalent feelings when differing emotions are applied on the left and right sides of the face. The third paper by Robert Backman and Marcelo Kallmann from the University of California, Merced, presents a system that allows non-programmers to create generic controllers for physically-simulated characters. The core of the proposed system is based on a directed acyclic graph of trajectory transformations, which can be modified by feedback terms and serve as reference motions tracked by the physically simulated character. The authors introduce tools to enable the automatic creation of robust and parameterized controllers suitable for running in real-time applications, such as in computer games. The entire process is accomplished by means of a graphical user interface. The paper demonstrates how the system can be intuitively used to design a simbicon-like walking controller and a parameterized jump controller to be used in real-time simulations. The last paper of this issue is by Jongmin Kim, Yeongho Seol and Jehee Lee, from the Seoul National University, Korea. The authors describe a real-time performance animation system that reproduces full-body character animation based on sparse 3D motion sensors on a performer. Producing faithful character animation from this setting is a mathematically ill-posed problem because input data from the sensors are not sufficient to determine the full degrees of freedom of a character. Given the input data from 3D motion sensors, similar poses are selected from a motion database and a local model is built for transforming on-line the low-dimensional input signal into a high-dimensional character pose. A regression method based on kernel CCA (Canonical Correlation Analysis) is employed and it effectively handles a wide variety of motions. Examples show that various human motions are naturally reproduced by the proposed method.

This issue contains 5 regular papers. Yuanfeng Zhu, Ajay Sundar Ramakrishnan, Bernd Hamann, and Michael Neff, from University of California in Davis, USA, present a new system for automatically generating three dimensional animations of piano performance, given an input midi music file. A graph theory-based motion planning method is used to decide which set of fingers should strike the piano keys for each chord. Anticipating the progression of the music, the positions of unused fingers are calculated to make possible efficient fingering of future notes. Initial key poses of the hands, including those for complex piano techniques such as crossovers and arpeggio, are determined based on the finger positions and piano theory. An optimization method is used to refine these poses, producing a natural and minimal energy pose sequence. The second paper by Mathieu Perriollat, from VI-Technology in Grenoble, France and Adrien Bartoli, from Université d'Auvergne in Clermont Ferrand, France, deals with developable surface reconstruction from real observations. Most of the existing developable surface parameterizations do not handle boundaries or are driven by overly large parameter sets. The main contribution of this paper is a generative model of bounded developable surfaces that solves these two issues. The model is governed by intuitive parameters whose number depends on the actual deformation and includes the `flat shape boundary'. The authors also propose a 3D reconstruction method well adapted to the use of key point matches over multiple images. The third paper by Fabien Tence from Virtualys in Brest, France, and Laurent Gaubert, Julien Soler, Pierre De Loor, and Cédric Buche, from Université de Bretagne Occidentale in Plouzané, France, propose a method for video games to create programs whose behaviors cannot be told apart from players when observed playing the game. To achieve this goal, the authors choose models using Markov chains to generate the behaviors by imitation. They propose a new model, called CHAMELEON, to enhance expressiveness and the associated imitation learning algorithm. They first organize the sensors and motors by semantic refinement and add a focus mechanism in order to improve the believability. Then, they integrate an algorithm to learn the topology of the environment which tries to best represent the use of the environment by the players. In the fourth paper, MingQi Yu and HongYan Quan, from China Normal University in Shanghai, China, put forward a hierarchical method of fluid surface modeling in natural landscapes. The proposed method produces a visually plausible surface geometry with the texture from a single video image recorded by a standard video device. In contrast with the conventional physically-based fluid simulation, the new method computes preliminary results using empirical method and adopts Stokes wave model to obtain the reconstruction result. The authors illustrate the working of system with a wide range of possible scene, and a qualitative evaluation of their method is provided to verify the quality of the surface geometry. The experiment shows that the method can meet the requirement of real-time performance and the reality of the fluid. The last paper by Oktar Ozgen, Marcelo Kallmann, Carlos Coimbra, from University of California at Merced, USA, and Selcuk Sumengen and Selim Balcisoy, from Sabanci University, Istanbul, Turkey, describes a new method based on the use of fractional differentiation in order to improve the efficiency of simulations based on Smoothed Particle Hydrodynamics (SPH). The proposed approach is based on the observation that the effects requiring a high concentration of particles are most often produced from colliding flows, and therefore by achieving a better modeling of this behavior with the use of fractional derivatives similar high-quality results can be achieved with a lower number of particles. As a result, the new method can be used to reduce the resolution without significant loss of quality, or to improve the quality of the simulation in the current chosen resolution.

This issue contains five regular papers. Ahmad Abdul Karim, Thibaut Gaudin, Alexandre Meyer, Axel Buendia and Saïda Bouakaz, from Université Lyon 1 and Spir.Ops in France, present a fully procedural method capable of generating in real time a wide range of locomotion for multi-legged characters in a dynamic environment, without using any motion data. Their system consists of several independent blocks: a character controller, a gait/tempo manager, a 3D path constructor and a footprints planner. The four modules work cooperatively to calculate in real time the footprints and the 3D trajectories of the feet and the pelvis. The system can animate dozens of creatures using dedicated level of detail techniques and is totally controllable allowing the user to design a multitude of locomotion styles through a user-friendly interface. In the second paper, Amit Kumar and Aparajita Ojha, from PDPM Indian Institute of Information Technology in Jabalpur, in India, propose a new approach to natural path planning by adding wavelet noise to a path generated using subdivision-based corridor map method. Because wavelet noise is almost perfectly band-limited and provides good details with minimal aliasing effects, the resulting path becomes smoother and more natural. Moreover, by appropriately choosing the levels of down/up sampling in the wavelet noise generation algorithm, frequency of wavelet noise can be adjusted. This serves as an effective tool in bringing variations in path as per the requirement. The third paper by Yanzhen Wang, Yueshan Xiong, Kai Xu and Dong Liu, from National University of Defense Technology in Changsha, Hunan, in China, describes a surgical procedure simulation system for the training of arthroscopic ACL reconstruction involving operations such as puncturing, probing, incision and drilling. In this system, the authors employ a linear elastic finite element method and position-based dynamics for deformable modelling. Simplified vertex duplicating method and an implementation of real-time Boolean operations are proposed for the topological change of tissue models involved in the incision simulation and tunnel construction. Two specially designed force-feedback models are introduced for the haptic rendering of probing and drilling operations. In the fourth paper by Siddharth Hegde, Christos Gatzidis and Feng Tian, from Bournemouth University, in UK, look at the different methods presented over the past few decades that attempted to recreate digital paintings. Whereas previous surveys concentrate on the broader subject of non-photorealistic rendering, the focus of this paper is firmly placed on painterly rendering techniques. The authors compare different methods used to produce different output painting styles such as abstract, colour pencil, watercolour, oriental, oil and pastel. Although some methods demand a high level of interaction using a skilled artist, others require simple parameters provided by a user with little or no artistic experience. The last paper by Llyr ap Cenydd and Bill Teahan, from Bangor University, in UK, describes a system for dynamically animating the locomotive behaviour of arthropods in real time, facilitating realistic and autonomous traversal across an arbitrary environment. By combining a de-centralized reactive behavioural model with a hybrid approach to motion that utilizes the comparative advantages of physical simulation and kinematic control, the system is capable of automatically generating complex organic motion over a wide range of surface features, independent of structural complexity. The reactive embodiment of the creature, combined with the physical simulation of the virtual world enables emergent behaviours to form that are entirely based on circumstance, including rigid-body interaction, grip recovery and adaptive wall climbing.

ABSTRACT Our goal is to create socially responsible agents, either robots or virtual humans. In this paper, we present an integration of emotions, attachment, and learning in emotional decision‐making to achieve this goal. Based on emerging psychological theories, we aim at building human‐like emotional decision‐making, where emotions play a central role in selecting the next action to be performed by the agent. Here, we present our own approach for emotion appraisal where we use emotional attachment as an important impulse for determining the intensities of emotions. Emotions in their turn are used to calculate the emotional attachment toward the users and for learning to predict future consequences. We report on the results of a simulation evaluation where we assess the influence of emotions, attachment, and learning on decision‐making. It is our strong belief that by giving an agent the ability to have emotions and to feel empathy and emotional attachment toward others, we will ultimately give this agent the ability to learn and improve its social behavior skills through interactions with the users and through user feedback. Copyright © 2013 John Wiley & Sons, Ltd.

With the extensive applications of 3-D multimedia technology, 3-D content compression has been an important issue, which ensures its smooth transmission on the network with constrained bandwidth. In this letter, we propose a new compression framework for dynamic 3-D facial expressions. Taking advantage of the near-isometric property of human facial expressions, we parameterize the dynamic 3-D faces into an expression-invariant canonical domain, which naturally generates 2-D geometry videos and allows us to apply the well-studied video compression techniques. Due to the difference from natural videos, each dimension (i.e., X, Y and Z, respectively) of the geometry video is regarded as a video sequence and encoded separately. Meanwhile, a model-based joint bit allocation scheme is designed to allocate reasonable bitrate to each dimension by detailed analysis of rate-distortion model for geometry videos, to obtain optimal results under given target bitrate. Experimental results show that up to 25% improvement in terms of bitrate reduction can be achieved, compared to existing algorithms.

This paper considers the problem of automatically segmenting an image into a small number of regions that correspond to objects conveying semantics or high-level structure. Although such object-level segmentation usually requires additional high-level knowledge or learning process, we explore what low level cues can produce for this purpose. Our idea is to construct a feature vector for each pixel, which elaborately integrates spectral attributes, color Gaussian mixture models, and geodesic distance, such that it encodes global color and spatial cues as well as global structure information. Then, we formulate the Potts variational model in terms of the feature vectors to provide a variational image segmentation algorithm that is performed in the feature space. We also propose a heuristic approach to automatically select the number of segments. The use of feature attributes enables the Potts model to produce regions that are coherent in color and position, comply with global structures corresponding to objects or parts of objects and meanwhile maintain a smooth and accurate boundary. We demonstrate the effectiveness of our algorithm against the state-of-the-art with the data set from the famous Berkeley benchmark.

This paper presents an interactive dynamic controller used to generate locomotion patterns for humanoid robots. The purpose of this work is to provide animators and artists easy and intuitive tools to design expressive motions for humanoid robots. A review of similar work in the computer animation community has guided our choices regarding the implementation and level of interaction between the user and an inverse dynamics solver. We have used our controller on a model of the Aldebaran humanoid robot Nao, and have generated a few expressive locomotion patterns that are presented in the experimental section of this paper.

This film is a making of from a public event, the New Media Village at IMI, that took place in Singapore in presence of a large public audience in October 2011. It's a combination of live fashion models and their 3D counterparts.

Category hierarchies often evolve at a much slower pace than the documents reside in. With newly available documents kept adding into a hierarchy, new topics emerge and documents within the same category become less topically cohesive. In this paper, we propose a novel automatic approach to modifying a given category hierarchy by redistributing its documents into more topically cohesive categories. The modification is achieved with three operations (namely, sprout, merge, and assign) with reference to an auxiliary hierarchy for additional semantic information; the auxiliary hierarchy covers a similar set of topics as the hierarchy to be modified. Our user study shows that the modified category hierarchy is semantically meaningful. As an extrinsic evaluation, we conduct experiments on document classification using real data from Yahoo! Answers and AnswerBag hierarchies, and compare the classification accuracies obtained on the original and the modified hierarchies. Our experiments show that the proposed method achieves much larger classification accuracy improvement compared with several baseline methods for hierarchy modification.

Stereotypical trust modeling can be adopted by a buyer to effectively evaluate trustworthiness of a seller who has little or no past experience in e-marketplaces. The buyer forms trust stereotypes based on her past experience with other sellers. However, when the buyer has limited past experience with sellers, the formed stereotypes cannot accurately reflect her trust evaluation towards sellers. To address this issue, we propose a novel generalized stereotypical trust model. Specifically, we first build a semantic ontology to represent hierarchical relationships among seller attribute values. We then propose a fuzzy semantic decision tree (FSDT) learning method to construct trust stereotypes that generalizes over seller non-nominal attributes by splitting their values in a fuzzy manner, and generalizes over nominal attributes by replacing their specific values with more general terms according to the ontology. Experimental results confirm that our proposed model can more accurately measure the trustworthiness of sellers in simulated e-marketplaces where buyers have limited experience with sellers.

ABSTRACT Traversing dense crowds can be challenging, but it is especially difficult in a virtual environment where the user is limited to simple input devices. Predicting near‐future user movements with a hidden Markov model allows nearby autonomous agents to react politely, that is, to specifically avoid impeding the movement of the user. This predictive model and simple avoidance scheme are tested by six subjects in 360 short interactive games and by a 10‐participant two‐alternative forced choice questionnaire. Polite agents are found to significantly improve the player's ability to navigate quickly and without collision, and the resulting character movements showed increased naturalness for two of the three game scenarios. Copyright © 2012 John Wiley & Sons, Ltd.

In this paper, we propose a reputation mechanism for virtual marketplaces. The proposed approach is based on five-sense oriented feedback provision with the support of existing virtual reality technologies. We have conducted user studies to analyse users' attitude towards this new approach. These studies reveal that users prefer virtual marketplaces with our proposed reputation mechanism over that with traditional reputation mechanisms, and that our mechanism can effectively ensure user's trust in the virtual marketplaces and simultaneously promote user's trust in other users. Our approach is based on feedback from other users. Feedback from users could be very subjective and misleading for other users. Hence, we propose a novel mechanism to align subjective user feedback before reputation computations in virtual marketplaces. Results of our experiment demonstrate that with our feedback alignment approach, buyers can more accurately model sellers' reputation.

This interactive application will allow visitors to play with garments in three dimensions, transforming them into creative, customizable and experimental objects. Based on touch screen technology and through a simple and attractive interface, visitors will be able to dress and customize a 3 dimensional virtual fashion model. The model will pose for you to show of the physically simulated garments in real time.

In the first paper, proposed by Cihan Halit and Tolga Capin from Bilkent University in Turkey, an automatic approach to extract key frames from a motion capture sequence is proposed.The input sequence is treated as motion curves to obtain a "motion saliency."With this approach, around 8% of the frames are selected to be key frames for motion capture sequences.The next paper on feature sensitive deformation for triangular mesh models comes from Shandong University in China.The four co-authors propose a novel surface deformation method based on a feature sensitive (FS) metric.Examples are included to show that the feature sensitive deformation method better preserves mesh details, especially features, than in existing Laplacian methods.The third paper co-authored by Andreas Alexandros Vasilakis and Ioannis Fudos from the University of Ioannina in Greece proposes a GPU skinning based method based on a refined skeletonization.To avoid the artifacts generated in previous skinning approaches and the associated high training costs, the authors have developed an efficient and robust rigid skinning technique that applies blending patches around joints.To achieve real time animation, they have adapted all steps of their rigid skinning algorithm so that they are performed efficiently on the GPU.

2011 in Chengdu, China.The last two papers are regular papers.In the first paper, Min Meng, Lubin Fan and Ligang Liu from Zhejiang University, in Hangzhou, China, present a novel sketch-based tool, called iCutter (short for Intelligent Cutter), for cutting out semantic parts of 3D shapes.When a user performs a cutting task, he or she only needs to draw a freehand stroke to roughly specify where cuts should be made without much attention.Then iCutter intelligently returns the best cut that meets the user's intention and expectation.The authors demonstrate various examples to illustrate the flexibility and applicability of their iCutter tool.Inmaculada Rodriguez and Anna Puig from the University of Barcelona and Marc Esteva from the Autonomous University of Barcelona, in Spain, propose, in the third paper, a generic interaction framework, which controls intelligent objects' actions in different virtual world platforms.These actions are based on the state of external platform-independent AI-based systems such as multiagent and rule-based systems.The authors have evaluated the proposed framework by means of two intelligent objects, a door and a notice board, incorporating them in Second Life and OpenWonderland virtual worlds.These objects allow to work along three advanced aspects of a serious virtual environment.

This issue contains six papers. In the first paper, Changbo Wang, Chenhui Li, Jinqiu Dai, and Yang Li, from East China Normal University in Shanghai, China, present a new rendering method based on adaptive lattice for lighting participating media of translucent materials such as marble, wax, and shaft light. Firstly, multi-scale hierarchical lattice is constructed. Then, an adaptive method to trace diffuse photons and marked specular photons with different phase functions is suggested. Multiple lights and heterogeneous materials are also considered here. Further, the mixed rendering method and GPU accelerate technology are introduced to render different light effects under different participating media. Yijiang Zhang, Julien Pettré, Jan Ondrej, Xueying Qin, Qunsheng Peng, and Stéphane Donikian from Zhejiang University in Hangzhou, China and IRISA in Rennes, France, propose, in the second paper, a video-based mixed reality system enabling interaction between real and virtual humans in real scenes. The real scene is captured by a camera, and the evolution of the environment is obtained by video analysis, which provides the constraint for the motion of virtual characters. Next, the virtual characters are modeled to move around the scene, keeping away from real obstacles and other virtual characters. To seamlessly integrate the virtual characters, the alpha map of each real pedestrian is carefully extracted. In the third paper, Jong-Hyuk Kim and Jung-Ju Choi from Ajou University in Suwon, Korea, and Christoph M. Hoffmann from Purdue University in USA propose a method to parameterize the skin deformation by skeletal motion. They present skin deformation by multiplication of two matrices, one of which encodes the basis matrix that deformation takes place and the other does the time-varying signals that capture the virtual muscle action. The time-varying signals are then parameterized by joint angles using RBF, which provides a fast and intuitive way to control the skin deformation. Furthermore, given a set of two skin deformations, they capture the style between the deformations and transfer the style into another deformation data by using only matrix computations. The fourth paper by Jianping Hu and Qi Xie from Northeast Dianli University, in Jilin, China, and Xiuping Liu from Dalian University of Technology in China present a subdivision connectivity remeshing approach. The main contribution lies in adopting a low distortion spherical parameterization approach to generate high-quality subdivision connectivity meshes. Besides, a simple and efficient point location method on the sphere based on the uniform partition of the rectangle is presented, which is used to find the containing triangle in the spherical mesh for each point on the sphere rapidly. The method can generate high-quality subdivision connectivity meshes fast, which can be applied to level of detail and progressive transmission. In the fifth paper, Kyusung Cho, Jinki Jung, Sang-Wook Lee, Sang Ok Lim, and Hyun Seung Yang from KAIST in Daejeon, Korea, present a markerless tracking method for augmented reality books, which recognizes and tracks a large number of pages in real-time, even on PCs with low computation power. For fast recognition with respect to a large number of pages, they propose a generic randomized forest. They also define the spatial locality of the subregions in an image to resolve the problem of a dropping recognition rate under a complex background. For tracking with minimal jittering, they propose the adaptive keyframe-based tracking method, which automatically updates the current frame as a keyframe when it describes the page better than the existing one. The last paper by Erdem Akagunduz, Ilkay Ulusoy, and Murat Yirci from Middle East Technical University in Turkey presents a new dynamic volume spline model for human face skin. The performance of the model is discussed on the basis of the estimation of the results of facial surgery of three different patients. Face models of the patients are obtained from 3D CT/MR scans by segmenting the skull, muscle, and skin layers. The control points of the dynamic volume spline are localized masses with viscoelastic material properties. These parameters are computed from the skin material properties that were published in the literature. Once the face models are generated, facial surgery plannings are simulated.

This special issue contains 28 papers selected from the Computer Animation and Social Agents 2011 (CASA'2011) Conference. This conference was founded by the Computer Graphics Society in 1988 in Geneva and has, since then, been held in various countries. Last year in France, this year in China and it will be held next year in Singapore. This year we received 163 papers and selected only 28 of those for this special issue; meaning an acceptance rate of only 17.2%. It is needless to say that these are of high quality; all having been reviewed by at least three reviewers. Animation techniques: Motion Control, Motion Capture and Retargeting, Path Planning, Physics-Based Animation, Image-Based Animation, Behavioral Animation, Artificial Life, Deformation, Facial Animation, Multi-Resolution and Multi-Scale Models, Knowledge-Based Animation and Motion Synthesis. Social agents: Social Agents and Avatars, Emotion and Personality, Virtual Humans, Autonomous Actors, AI-Based Animation, Social and Conversational Agents, Inter-Agent Communication, Social Behavior, Gesture Generation and Crowd Simulation. Other related: Animation Compression and Transmission, Semantics and Ontologies for Virtual Humans and Virtual Environments, Animation Analysis and Structuring, Anthropometric Virtual Human Models, Acquisition and Reconstruction of Animation Data, Level of Details, Semantic Representation of Motion and Animation, Medical Simulation, Cultural Heritage, Interaction for Virtual Humans, Augmented Reality and Virtual Reality, Computer Games and Online Virtual Worlds. We would like to thank Prof. Yueting Zhuang from Zhejiang University in China, Prof. Daniel Thalmann from EPFL in Switzerland and Prof. Enhua Wu from University of Macao in China, the co-conference chairs of CASA'2011, for their strong support in the conference. We also like to thank the organizing co-chairs, Prof. Jieqing Feng from Zhejiang University in China and Prof. Leiting Chen, from University of Electronic Science and Technology of China, for their intense collaboration as well as the international program committee for their strong commitment and the external referees. We would like also to thank the authors for having submitted a paper to CASA'2011. Conference Co-Chairs Yueting Zhuang Daniel Thalmann Enhua Wu Program Co-Chairs Zhigeng Pan Nadia Magnenat-Thalmann Jinxiang Chai International Program Committee Abdennour El-Rhalibi Ahmad Nasri Ana Paiva Anton Nijholt Arie Kaufman Arjan Egges Bing-Yu Chen Carlos Martinho Catherine Pelachaud Chris Joslin Courty Nicolas Daniel Thalmann Dinesh Manocha Dinesh Pai Donald House Dumont Georges Elisabeth Andre Enhua Wu Fabian Di Fiore Feng Dong Florence Bertails Franck Multon Grisoni Laurent Hans-Peter Seidel Herwin Welbergen Hujun Bao Hwan-Gue Cho Hyewon Seo Igor Pandzic J.P. Lewis James Hahn Jean-Paul Laumond Jian Zhang Jieqing Feng Jinhui Yu Jinxiang Chai John Patterson Jos Stam Julien Pettre Kangkang Yin Kuffner James Kulpa Richard Lau Rynson Lee Tong-Yee Louis-Philippe Morency Luiz Velho Magnenat-Thalmann Marc Cavazza Marcelo Kallmann Marie-Paule Cani Mark Overmars Martin Jean-Claude Massimo Bergamasco Matthias Teschner Michael Gleicher Min-Hyung Choi Nancy Amato Neeharika Adabala Ning Wang Norman Badler Paolo Petta Petros Faloutsos Philip Willis Porcher-Nedel Luciana Prem Kalra Qunsheng Peng Raupp-Musse Soraia Rick Parent Stephane Donikian Stephane Redon Sung-Yong Shin Taku Komura Tolga Capin Van-Reeth Frank Weidong Geng Weiwei Xu William Baxter Wonsook Lee Xiaogang Jin Yangsheng Wang Ying-Qing Xu Yiying Tong Yizhou Yu Yu Qizhi Yueting Zhuang Zhigang Deng Zsofi Ruttkay

Animated human body models are widely used in computer graphics applications. Creating such models requires extensive design efforts, specialized hardware such as body scanners, model databases and authoring tools. Customizing these characters or creating a virtual population with them requires multiple design efforts or powerful computers. In this paper we present a simple and robust method to create variously sized bodies based on a single template animated human body model. To achieve this, our method uses the skinning information attached to the model. By taking the positions of skeletal joints as a reference, our method automatically segments the body into its anatomical regions. Then, by using the anthropometric landmarks as parameters, the regions are deformed and the underlying skeletal structure is adapted to fit the new morphology. Our method preserves the skeleton-mesh integrity for variously sized bodies. We demonstrate our method by using a database of anthropometry measurements to generate a population of virtual humans generated from our single template model.

Musculoskeletal disorders (MSDs) account for the largest fraction of temporary and permanent disabilities. Osteoarthritis (OA) is one of the most common MSDs which is characterized by a degeneration of articular cartilages. Understanding and preventing OA are of paramount importance in our aging yet very active society, and in this context, computer-assisted models of articulations are highly demanded by biomechanical and medical communities. To investigate the causes of some idiopathic OA, we have devised a unique comprehensive methodology to simulate musculoskeletal models of human articulations. Built from a rich variety of acquisition modalities (MRI, Motion capture, Body scanning, etc.) and innovative research, these models are fully subject-specific and account for anatomy, motion and biomechanical behavior of the articulations. This paper presents a complete overview of the methodology with medical validation and clinical case studies.

The MTDL 2010 workshop in its second edition aims to continue in the contribution and evaluation of the impact of multimedia technologies to e-Learning. This workshop is held in conjunction with the ACM Multimedia 2010 Conference in Firenze (Italy). As a cover paper of this workshop, we briefly summarize important issues to be addressed in e-learning in the first section, followed by a discussion of important issues proposed in the 6 papers accepted to the workshop (among the 14 submissions).

In this paper, we address the question of how to create episodic memory based long-term affective interactions with a human-like robot. A key challange for long-term interaction is the recall of past important events during conversation. We suggest that episodic memory is a core concept for realizing this intelligence. In this paper, we propose a model of episodic memory and integrate it with a decision making module based on a Hierarchical Task Network (HTN) planner. Plans generated by the HTN planner are executed by a Finite-state-machine (FSM) based dialogue system in order to produce appropriate responses. Finally, we present a prototype implementation demonstrating the preliminary results we obtained.

This issue contains four regular papers. The first paper, by Qingqiong Deng and Xiaopeng Zhang from the CAS Institute of Automation in China, Gang Yang from the Beijing Forestry University, and Marc Jaeger from the INRIA Saclay in Montpellier, France present a new foliage simplification framework for level of detail (LOD) of plant geometric models and forest rendering. Leaf density is introduced to adapt compression to the local distribution of leaves, so that more visually relevant details are kept. With a specific design of LOD storage structure, the costly hierarchical traversal of a binary tree is replaced by simple linear lookup array retrieval. This structure is GPU-oriented and decreases the communication between the CPU and the GPU for LOD level model loading in rendering. Sun-Uk Hwang, Beom-Chan Lee, Jeha Ryu, Kwan H. Lee, and Yong-Gu Lee from GIST-Mechatronics in Korea propose a temporal smoothing technique for haptic interaction using a sensing glove in multi-modal applications. The technique employs two processes: (1) a noise reduction method to reduce jitter noise at the sensors in the sensing glove, and (2) an adaptive force extrapolation for time-varying haptic and video frame rates. The authors have developed a test platform to assess a simple box model and relatively complex models such as gamephone, portable media player. The third paper, by Nicolas Stoiber from Orange Laboratory in France, Renaud Seguier from Supelec, and Gaspard Breton from Orange Laboratory present an animation system that gathers the advantages of two approaches in animation: parameter-based animation and performance-based animation. By analyzing a database of facial motion, the authors create the human appearance space, which provides a coherent and continuous parameterization of human facial movements, while encapsulating the coherence of real facial deformations. The method optimally constructs an analogous appearance face for a synthetic character. In the fourth paper, I-Chen Lin, Wen-Hsing Chang, Yung-Sheng Lo, Jen-Yu Peng, and Chan-Yu Lin also from the National Chiao Tung University in Taiwan introduce, in the last paper, a novel optimization framework for estimating the static or dynamic surfaces with details. The proposed method uses dense depths from a structured-light system or sparse ones from motion capture as the initial positions, and exploits non-Lambertian reflectance models to approximate surface reflectance. Multi-stage shape-from-shading is then applied to optimize both shape geometry and reflectance properties.

This contains five regular papers. The first paper, by Hui Chen and Hanqiu Sun, from The Chinese University of Hong Kong and Xiaogang Jin, from State Key Lab of CAD&CG, Zhejiang University in China propose a novel haptic-constraint modeling method based on interactive metaballs, during which the haptic-constraint tools are attracted to the target location and then control the touch-enabled deformation within the constrained areas. The interactive force feedbacks facilitate designers to accurately deform the target regions and fine carve the details as their intention on the objects. Experimental results show that the dynamic sense of touch during the haptic manipulation is intuitively simulated to users, via the interacting interface the authors have developed. Chunyong Ma, Ge Chen, Yong Han, Yongyang Qi, and Yong Chen from Ocean University of China introduce a virtual city oriented VR-GIS platform which synthesizes several latest information technologies including virtual reality, 3D geographical information system, remote sensing, and multi-dimensional visualization. The platform is a seamless integration of VR functions and GIS analysis methods, which can be used to organize and present massive spatial data. It also supplies 3D spatial analysis functions, 3D visualization for spatial process and natural simulation, and serves as an engine platform for digital city The third paper, by Jieje Zhu, from University of Central Florida and Zhigeng Pan, Sun Chao, and Wenzhi Chen from Zhejiang University in China present a method to separate occluded objects in multiple layers by utilizing depth, color, and neighborhood information. Scene depth is obtained by stereo cameras and two Gaussian local kernels are used to represent color, spatial smoothness. These three cues are intelligently fused in a probability framework, where the occlusion information can be safely estimated. They apply their method to handle occlusions in Video-based Augmented Reality where virtual objects are simply overlapped on real scenes. In the fourth paper, Hyewan Seo, Joon-Won Bang, Soon-Hyun Jeon from Chungnam National University, and Ji-Man Park, from Samsung Data Systems in Korea present their work on a 3D billiards game featuring a haptic interface. Most of the existing games available today concentrate on the development of sophisticated physics engines to obtain realistic motion of billiards balls or of sound engines for realistic sound effect. Their focus in this work is on the improved realism on the hitting action of the game with the aid of haptic device. They compute accurate feedback force on the haptic handle, considering the exact point of contact and physically faithful collision response. They present the physically based modeling of the game environment, including the collision interaction among billiards balls, and between the table and billiards balls. Soo Jeong Kim, Jyung Jun Gil, HyungSeok Kim, Sang Beom Lim, and Jee-in Kim from Konkuk University in Korea propose a method to adapt the interfaces and interaction processes of heterogeneous devices. The interaction capability of the devices is modeled by categorizing elementary actions and measuring its effect in semantic behaviors. With this model, an interaction process can be modified to a given device by changing sequence of elementary actions for each behavior. In the pilot test, the authors showed the possibility of interaction adaptation for different situations. With the proposed adaptation mechanism, interface, and interaction can be modeled for the device independently and can be transferred over different interaction environments.

The 11 papers in this special issue can be categorized into five groups: Emotional speech synthesis and recognition; affective video content analysis; facial expressions and head movements; affect analysis in small groups; and audio-visual affective corpus.

Despite decades of research, digital hairstyling is still a difficult, long and tedious task. One reason for the absence of efficient and simple, yet accurate hair modeling interfaces is the lack of appropriate interaction metaphors enabling efficient styling functionalities. This paper discusses tool-based interaction metaphors for digital hairstyling applications with a specific focus on haptic modeling methods. We define users tasks, ergonomic aspects, as well as haptics-based styling and fine-tuning tools, concluding with a validation of selected interactions on an experimental prototype.

As computers become computationally more powerful, the gap between virtual and physical reality is continuously diminishing. Striving for increased accuracy, computer simulations are becoming more and more specialized by the day - but are less capable of generalization. There is currently no comprehensive physically-based simulation approach for real-time animation of deformable objects within one unique framework. The first step required to tackle this challenge is to investigate invariant aspects in the correlation between dynamic behavior and physical properties of objects with different shape and consistency. In this paper we discuss the mechanical behavior of deformable one-dimensional rods and two-dimensional surfaces: hair and clothes.

We present a dynamic hair animation method designed for use in real-time virtual environments. Basing on recent research results in elastic rod simulation, we introduce an enhanced physically plausible technique which utilizes specific properties of hair strands to achieve better performance and more stable simulation. Stability is also enhanced by decoupling major sources of dynamic equation stiffness into a separate post-integration step. We also use the same decoupling principle to treat hair-head collisions in a very efficient and robust way. Thanks to this, we can afford to use a fast explicit integration scheme, which, combined with a short time step, allows our method to provide smooth results even in frequency-sensitive areas such as haptics-based hair modelling.

This work describes a methodology for the simulation of the mechanical behavior of the dancers hip. Patient-specific anatomical and kinematical models are used as input to simulate the hip cartilages deformations during motion. Based on computer graphic techniques, this methodology allows a 3D visualization and assessment of pressure distribution, offering orthopedists a complementary tool for diagnosis.

One-dimensional structures are very important for simulating a variety of slender objects such as ropes, hair, wires, cables or tubes. Because of their practical relevance, an increasing number of dynamic simulation methods have been proposed during the last years. However, interaction techniques have not followed this evolution. Hence, we propose to combine the recent advances in the computation of physically plausible rod dynamics with dedicated force rendering methods. We present a novel approach for computing the haptic interaction with slender objects in a virtual environment. Our interaction framework allows for an enhanced control over the rod by taking into account user-induced torques in the dynamics equations. Interaction forces are computed according to the rod's bending stiffness and frictional properties. Our force rendering method can thus be applied to a variety of simulation models based on the Cosserat Theory of Elastic Rods. The results of this paper are relevant for including haptic feedback within applications involving 1D-rods, such as virtual hair modeling systems for the digital effects industry, or assembly simulation environments in the automotive industry using flexible parts such as wire harnesses and hoses.

In current computer games and simulation environments, individuality of virtual character bodies are mainly constructed using different textures and accessories. However, this type of modeling generates anthropometrically similar shapes due to the reliance on a single or few body models. Alternatively, using large variety of body size models require larger storage resources and design efforts. We present an efficient method for generating and storing variety of body size models derived from a skinned template. Our method doesn't require additional design efforts and uses the existing skinning data that are already attached to the template model. Algorithm used for sizing the model is based on anthropometric body measurement standards that are used in ergonomic design application. Resulting new body size models use the same skinning information for animation by adapting the underlying skeleton according to the anthropometric parameters. Our developed system is useful in CAD applications from ergonomic design of cloths to parametrically resizing avatars.

Abstract This paper describes a methodology for the simulation of musculoskeletal disorders. Our clinical study is related to osteoarthritis (OA) of the hip, a pathogenesis possibly due to impingements. These bone collisions lead to abnormal joint mechanics which is characterized by contact pressure and stress distribution upon the joint cartilages. The proposed methodology combines different approaches from modeling to simulation. The simulation is based on patient‐specific anatomical models, where acquisition modalities are noninvasive and flexible. Based on static magnetic resonance imaging (MRI) data, a discrete deformable models method is used for modeling the organs of the musculoskeletal system. Femoroacetabular movements are estimated using an optical motion capture system and are validated by a dynamic MRI analysis. To achieve accurate deformations, techniques to generate volumetric meshes are developed based on the medial axis (MA) information. Finally, a computationally efficient fast functional joint model is used to simulate the mechanical behavior of the soft tissues. The goal of such a simulation is to allow the investigation of the relevant contact and cartilages deformation under movement, which can be useful for diagnosis, pre‐ or post‐operative planning and training. This will benefit further developments in surgical techniques and minimally invasive procedures. Copyright © 2008 John Wiley & Sons, Ltd.

The field of computer graphics has impacted a number of application domains in medicine. Medical volume visualization, in particular, has a long and active history in the computer graphics community. However, there are many other techniques in computer graphics that have been applied successfully in medicine. Virtual Reality Simulators have been used to train physicians on difficult procedures. Image-guidance has been used to assist surgeons using different imaging modalities. Different types of visualization techniques have been applied to analyze medical images. This special issue is devoted to novel applications of computer graphics in medical domains. The emphasis has been on technical novelty as opposed to novel medical applications of well-known techniques. The hope is that these papers will be a catalyst for computer graphics community to become more aware of the enormous potential that they hold in making significant contributions to medical domain. The first two papers deals with visualization of medical datasets. Virtual colonoscopy has been used successfully as a faster and painless alternative to real colonoscopy. The paper “Coherence aware GPU based ray casting for virtual colonoscopy” proposes using Graphic Processing Units (GPUs) to ray-trace volumes for use in real-time virtual colonoscopy. An emerging imaging modality is the 3D ultrasound. In “GPU-based Interactive Visualization Framework for Ultrasound Datasets,” the volumetric data generated from a 3D ultrasound is rendered in real-time using the GPU. Simulation can be used in two different ways. They can be use used to train surgeons to perform difficult procedures as described in the next two papers. They can also be used to analyze the physiology as a part of the diagnostic process. In “Haptic Ventriculostomy Simulation in a Grid Environment,” a full-featured surgery simulator that encompass haptic interaction with deformable models in a shared virtual environment is described. Currently, there is little work on evaluating surgical simulators for their effectiveness in training. In “A New Assessment Methodology for Virtual Reality Surgical Simulators,” an assessment methodology for evaluating virtual reality surgical simulators is presented. Simulation offer physicians the ability to analyze biomechanical properties. In “From MRI to Anatomical Simulation of The Hip Joint,” a system for simulating a patient-specific hip joint is presented. The anatomy is generated from Magnetic Resonance Imaging and motion capture is used to drive the simulation. “Image Guided Medialization Laryngoplasty,” describes a system to guide surgeons intra-operatively using image-based modeling and registration.

Abstract Virtual characters are being modeled and animated with increasing accuracy and photorealism in current games and virtual reality simulations. However, the detailed modeling of dynamic skin deformations due to movement of the muscles during articulation is still prohibitively expensive for real‐time simulations. We present in this paper a fast approach for modeling such deformations driven by Electromyography (EMG) data. We demonstrate the method on the muscles of the upper leg during a gait cycle. A muscle map, created from a 3D muscle model, is applied as a deformation map to deform the skin surface as per the underlying muscle geometry. EMG signals recorded from a gait cycle are then used to dynamically vary the weights in the deformation map during animation thus producing the dynamic skin deformations. The whole operation is done on the GPU to make it very fast and suitable forreal‐time simulations. Copyright © 2009 John Wiley & Sons, Ltd.

This issue contains three papers. The first one presents a combined compression and simplification of dynamic 3D meshes and is proposed by Libor Vasa and Vaklav Scala from the University of West Bohemia in Pilsen, Czech Republic. The paper deals with a new approach to dynamic mesh compression. The algorithm allows fast progressive transmission of dynamic 3D content. The second paper is written by R. Farouki from the University of California, USA and Carlotta Giannelli from the University of Florence, Italy. It presents a spatial camera orientation control by rotation-minimizing directed frames research. The third paper is authored by several researchers from the University of Barcelona, Spain. A novel approach to generate augmented video sequences in real-time, involving interactions between virtual and real agents in real scenario is explained and discussed.

Recent mechanical models for cloth simulation have evolved toward accurate representation of elastic stiffness based on continuum mechanics, converging to formulations that are largely analogous to fast finite element methods. In the context of tensile deformations, these formulations usually involve the linearization of tensors, so as to express linear elasticity in a simple way. However, this approach needs significant adaptations and approximations for dealing with the nonlinearities resulting from large cloth deformations. Toward our objective of accurately simulating the nonlinear properties of cloth, we show that this linearization can indeed be avoided and replaced by adapted strain-stress laws that precisely describe the nonlinear behavior of the material. This leads to highly streamlined computations that are particularly efficient for simulating the nonlinear anisotropic tensile elasticity of highly deformable surfaces. We demonstrate the efficiency of this method with examples related to accurate garment simulation from experimental tensile curves measured on actual materials.

The aim of this paper is to propose a communication middleware which makes it possible to easily and efficiently develop the networking support for multi-user applications. Even though existing middleware and development tools provides lots of functionalities to realize distributed applications, they are purely low-level services passing the most development to developers or too specialized for a specific application. It brings a challenging issue of how to support sufficient and general high-level mechanisms by the middleware. To meet different networking and interaction requirements of multiple users, our approach is to address various possibilities of the communication architecture, the user membership management, the content transmission mechanism and the event management. In each requiring feature, the proposed middleware provides several options with application-level APIs and configuration so that the different interaction needs of a multi-user application can be easily handled in the developerspsila point of view.

Multimodal dialogue management capabilities involve and combine input and output from various interaction modalities and technologies. In this paper, we present our research done in the framework of the European Project Indigo. In this project, we try to communicate in a natural way with a virtual human and possibly with a very realistic robot skin-based face head. We have defined memory models, recognition of emotions, and dialogue interaction based on recognition of emotions of the user. We can have a dialogue either with a virtual human or with our robot head that can recognize us, ask some specific questions concerning our habits and is able to understand our answers and behave accordingly.

Multimedia has been widely applied in life science research and clinical applications, including research on musculoskeletal system. Musculoskeletal disorders are the most notorious and common causes of severe long-term pain and physical disability, affecting hundreds of millions of people across the world. In the European Research Project 3D Anatomical Human (http:// 3dah.miralab.unige.ch/), we are building an accurate generic lower limb model that can be simulated in motion, using individual multimodal medical data. For clinical every-day use, medically relevant validation and an efficient visualization and interaction framework are required. We are defining a generic functional model of the lower limb (consisting of bones and soft-tissues) that can be simulated in motion. Relevant patientpsilas anatomical, kinematical and mechanical data extracted from images (MRI, CT), motion capture (dynamic MRI, optical motion capture) and other modalities (EMG, mechanical properties measuring device), as well as statistical data, are being adjust the generic model to the patient. A fully functional model will be presented with many individual cases study and medical validation.

Musculoskeletal disorders are the most notorious and common causes of severe long-term pain and physical disability, affecting hundreds of millions of people across the world. To prevent and treat these disabling conditions, we are building an accurate generic lower limb model (consisting of bones and soft tissues) that can be simulated in motion, using individual multimodal data. For clinical every-day use, medically relevant validation and an efficient interactive visualization framework are required. Relevant patientpsilas anatomical, kinematical and mechanical data extracted from images (MRI), motion capture (dynamic MRI, optical motion capture) and other modalities (body scanning, EMG, mechanical properties measuring device), as well as statistical data, are adjusting the generic model to the patient. A fully functional model will be presented with some individual case studies and medical validation.

This tutorial will present the latest techniques to model fast individualized animatable virtual humans for Real-Time applications. As a human is composed of a head and a body, we will analyze how these two parts can be modeled and globally animated as in real-life. More precisely, we will show how we can model and deform human bodies and heads. Facial animation will be also addressed from motion facial capture and voice to the simulation of interactive realistic talking virtual humans, including personality models and complete body gestures. We will describe how we can model crowds in realtime using dynamic meshes, static meshes and impostors. Techniques to introduce variety in crowds including individual animation with accessories will be explained.

The International Conference on Computer Animation and Social Agents (CASA) organized by the Computer Graphics Society (CGS) is one of the premier academic conferences in the field of computer animation, specializing in character/object modeling, animation, and behavior simulation of social agents. This year, the conference was held in Seoul, Korea marking its 21st occurrence. The CASA international program committee has selected 32 full papers from 100 submissions around the world based on comments and scores from at least four reviewers assigned for each submitted paper. It is our great pleasure to have those distinguished work to be presented in this special issue of the Computer Animation and Virtual Worlds Journal (CAVW). The 32 papers presented in this journal are nicely categorized into: (1) character animation, (2) facial animation, (3) crowd modeling and simulation, (4) synthesis and 2D methods, (5) general modeling and deformation techniques, (6) physical simulation, and (7) applications. The Program Co-Chairs would like to thank the international program committee for spending their valuable time and effort in the reviewing process and the selected authors for their contribution.

Abstract With the increasing performances of the graphics hardware, virtual reality applications are reaching a new level of realism. It is now possible to animate entire crowds of virtual avatars, or to display in real‐time highly realistic deformable characters. This new type of applications requires that characters are deformed, either to generate a large number of individuals for creating a crowd, or to match a particular shape. To make the animation fit to the animated character and to prevent self‐collisions, adapting the motion is mandatory. Unlike previous approaches which relied on skeletal models, this paper presents a method considering the actual shape of the character's body to perform the adaptation. Our approach uses spacetime optimization to remove the self‐penetration, and finally re‐establishes the balance of the motion. We also introduce an interpolation scheme based on radial basis functions that can blend pre‐calculated adaptations, and thus achieve real‐time performances. Copyright © 2008 John Wiley & Sons, Ltd.

This issue contains five regular papers. The first paper, by Marcelo Kallmann from University of California in Merced, USA, presents a novel whole-body analytical Inverse Kinematics (IK) method integrating collision avoidance and customizable body control for animating reaching tasks in real-time. Body control is achieved with the interpolation of pre-designed key body postures, organized as a function of the direction to the goal to be reached. Arm postures are computed by the analytical IK solution for human-like arms and legs, extended with a new simple search method for achieving postures avoiding joint limits and collisions. In addition, a new IK resolution is presented that directly solves for joints parameterized in the swing-and-twist decomposition. The second paper, by Man Hee Lee and In Kyu Park from Inha University in Korea, explains an image-based method for modeling 3D objects with curved surfaces based on the NURBS representation. The user fits the feature curves on a few calibrated images with 2D NURBS curves using the interactive user interface. Then, 3D NURBS curves are constructed by stereo reconstruction of the corresponding feature curves. Using these as building blocks, NURBS surfaces are reconstructed by the known surface building methods including bilinear surfaces, ruled surfaces, generalized cylinders, and surfaces of revolution. In the third paper, Frank Patrick Vidal and Nigel W. John from University of Wales and Derek A. Gould and Andrew E. Healey from the Royal Liverpool University in UK present an integrated system for training ultrasound guided needle puncture. Their aim is to provide a validated training tool for Interventional Radiology (IR) that uses actual patient data. A hybrid surface/volume haptic rendering of an ultrasound transducer is proposed to constrain the device to remain outside the bony structures when scanning the patient's skin. The authors also propose a volume haptic model that implements an effective model of needle puncture. Force measurements have been made on real tissue and the resulting data are incorporated into the model. Seunghyun Han, Mingyu Lim, Dongman Lee, and Soon J. Hyun, from the Information and Communications University in Korea, introduce, in the next paper, a scalable interest management scheme for Distributed Virtual Environments. Their new scalable filtering scheme reduces not only the number of messages during interaction in a region and among neighboring regions, but also the number of multicast addresses without significant computational overhead. For interaction among neighboring regions, the authors propose a sub-region concept to select only a subset of users from the neighboring regions based on proximity, the distribution of the users, locations, and the viewing direction of a user. The last paper, by Anderson Maciel and Suvranu De from the Rensselaer Polytechnic Institute in USA, presents an efficient dynamic point algorithm for line-based collision detection in real-time Virtual Environments. The algorithm for computing the interaction with a line-shaped haptic cursor has a near constant complexity. It relies on space-time coherence and topological information to maintain proximity information between a line segment and triangle meshes. The line is represented by a dynamic point, which is chosen to be the closest point on the line to any potentially colliding triangle. The algorithm may be used for collision detection and response with rigid and deformable objects with no performance penalty.

This issue contains four regular papers and the two best papers of ICEC 2007. The first paper, by Silvia Rueda, Pedro Morillo, and Juan Orduña from the University of Valencia, Spain, proposes a comparative study of the performance provided by awareness methods that are supposed to fully solve the awareness problem in peer-to-peer distributed virtual environments. The paper evaluates the performance of the considered methods on actually distributed systems. The evaluation results show that only a single method actually provides full awareness to avatars. The second paper, by Fan Zhang, Hanqiu Sun, and Chong Zhao, from The Chinese University of Hong Kong, and Lifeng Wang from Autodesk Software, presents an enhanced variant of light space perspective shadow maps to optimize perspective aliasing distribution in possible general cases where the light and view directions are not orthogonal. Authors' experiments have shown the enhanced shadow quality using their algorithm in dynamic scenes. This paper is one of the best papers of ICEC 2007. In the third paper, Liying Wang, Wei Hua, and Hujun Bao, from Zhejiang University, China, formulate the urban planning as a constrained layout optimization problem; they propose an algorithm to solve the problem and procedurally generate models of urban zone. It produces extensive virtual urban environment for computer games and simulations at a low cost. Experimental results prove that their method can efficiently produce the virtual urban scene similar to that designed by urban planners. This paper is one of the best papers of ICEC 2007. Marco Gillies, Xueni Pan, Mel Slater, and John Shawe-Taylor from University College London, UK, introduce, in the next paper, an important aspect of non-verbal expression: people respond to each other's behavior and are highly attuned to picking up this type of response. The paper takes a data driven approach to generating interactive social behavior. Listening behavior is motion captured, together with the audio being listened to. These data are used to learn an animation model of the responses of one person to the other. This allows us to create characters that respond in real-time during a conversation with a real human. Umberto Castellani, from University of Verona, Italy, and Vincent Gay-Bellile and Adrien Bartoli from Universitè Blaise Pascal, The Puy-de-Dôme, France, present, in the next paper, a deformation capture system for surfaces from temporal range data obtained by 3D sensors. The proposed method is based on planar mesh that is deformed so as to fit each range image. The authors show how this is achieved by minimizing a compound cost function combining several data and regularization terms, needed to make the overall system robust so that it can deal with low quality datasets. In the last paper, by James Anderson, Nancie Davidson, Hazel Morton, and Mervyn Jack, from University of Edinburgh, UK, the authors introduce The Spoken Electronic Language Learning (SPELL) system. It is a self-access computer-assisted language learning package that integrates speaker-independent continuous speech recognition technology with virtual worlds and embodied virtual agents. The goal is to create an environment in which learners can converse in the target language within meaningful contextualized scenario. The paper focuses on the technological aspects of the system and its evaluation for usability and robustness, rather than its pedagogical methodology.

Abstract In this issue, you will find a special section on Wireless and Mobile Technologies in Mixed Realities, with four papers introduced by the two guest editors Gurminder Singh and Georges Papagiannakis. The last paper of the issue is a regular paper by I. Mansa et al. entitled “Analysis of coherence strategies for stereo occlusion culling.” This paper explores the benefits that can be achieved for stereoscopic visualization when using occlusion‐culling strategies. It shows the limitations placed by compact and not densely occluded scenarios and the gains that can be achieved when rendering stereo images using occlusion culling. The experiments carried out test combinations of occlusion stereo coherence and occlusion frame coherence. The algorithm presented can be easily implemented and provides a significant boost in performances.

Abstract Recent advances in hardware and software for mobile computing have enabled a new breed of mobile augmented reality (AR) systems and applications. A new breed of computing called ‘augmented ubiquitous computing’ has resulted from the convergence of wearable computing, wireless networking, and mobile AR interfaces. In this paper, we provide a survey of different mobile and wireless technologies and how they have impact AR. Our goal is to place them into different categories so that it becomes easier to understand the state of art and to help identify new directions of research. Copyright © 2008 John Wiley & Sons, Ltd.

Cloth simulation and fabric measurement are tightly linked areas of research. In order to obtain high quality animations of dressed models, the properties of the simulated garment must first be evaluated in an accurate and adapted way. As cloth is a very complex, isotropic material, the evaluation of its properties is difficult to achieve, and various approaches exist. Depending on the design of the simulation engine, the measurements will be done differently, so that the outputted parameters match the inputs required by the simulator. Various issues must be considered, and depending on the complexity of the simulated garments, tradeoffs must be made in order to reconcile the real features of the cloth (stitches, layers...) and the computational capacities of the simulator (numerical integration, collision detection...)

We present an efficient technique for real-time human body model generation from sizing parameters. Based on a template model, various body sizes are generated according to the anthropometric measurement standards. These standards are used during the design stage of the template body model to segment it into regions where each region is deformed by corresponding measurement parameter. Depending on the anatomical shape of the regions, deformation functions with specific control parameters are blended to change the body sizes. Implementation of the deformation functions are based on splines and radial distance of the surface vertices. Before applying any deformation, displacement of the vertices are filtered to preserve the continuity between neighbouring and overlapping regions. With a set of high level parameters, this technique could be used to generate different size models in virtual environments.

The number of various existing fabric materials for different usages is unlimited. Therefore it is important to judge each textile material regarding quality and suitability before any manufacturing process. Related fabric characteristics can be subjectively assessed or objectively measured. A new field of research tries to imitate the subjective fabric evaluation method by virtually simulating the touch of fabrics with new haptic and tactile technologies. However, the today existing technology does not allow the rendering of complex interactions between hand and fabric, as it occurs during the real assessment method. Thus, the subjective fabric evaluation needs to be simplified to allow a direct comparison of the real and the virtual process. The main difference of the traditional subjective assessment, where the fabric is touched with both hands, to the simplified one lays basically in the fixation of the fabrics, so that the specimen can be judged with two fingers. The main mechanical properties such tensile, shear, bending, compression, friction, surface and weight have been assessed with the new test arrangement. The results of the assessment have been reported for all tested properties.

In this paper, we focus on adaptive visuo-haptic simulation of hair using force feedback haptic devices, and propose an easy-to-use interactive hair modelling interface. The underlying idea is to explore ways of integrating visual hair simulation and haptic into one multirate-multilayer-multithread application allowing for intuitive interactive hair modeling. The user is allowed to interact with the simulated hair on a virtual human's head through a haptic interface. By adding the sense of touch in the proposed system, we enter the domain of multimodal perception and stimulate both vision and touch of the user. This will allow the user to see a realistic hair simulation performing at interactive rates and easily use virtual tools to model the hair style. The proposed research tackles many significant challenges in the domains of multimodal simulation, collision detection, hair simulation and haptic rendering.

Combining the accuracy of continuum surface mechanics with the versatility of particle systems, we propose a very simple and fast mechanical model for representing bending elasticity in surfaces. This model, formulated as a grid particle system, simulates accurately bending elasticity, and offers very good computational efficiency through a very simple and completely linear formulation. We demonstrate its accuracy and efficiency through quantitative comparative tests.

In this paper we propose a method for the automatic extraction and annotation of the animation control skeleton of virtual humans, which relies on an a-priori knowledge of the human anatomy. The method is based on a segmentation of the virtual human shape into semantically meaningful features, like arms or legs, and on an automatic location and labeling of joints of the control skeleton. The method is particularly relevant for computer animation where the process still largely relies on manual tasks, and especially for virtual characters built on real scanned data. Several examples will show the results obtained with our approach.

Visually, wet hair is easily distinguishable from dry hair because of the increased highlights and intense darkening displayed by them. It is therefore essential for realism to capture these characteristics under certain real world conditions. In this regard we propose a model for rendering wet hair at interactive rates. We start by analyzing the physical aspect behind this special effect in hair and then present a model for incorporating the variations in visual appearance of the hair due to presence of water. For simulating the increased specularity because of the water layer on hair, we present a parameter controlled Gaussian-based model. To simulate darkening in hair, for outer hair we consider total internal reflection at water-hair interface as dominant and propose a probabilistic approach to determine the amount of light absorbed. For inner hair, we consider that increase in opacity due to water results in stronger self-shadow and propose a model that updates the opacities based on water content and accumulates them to calculate the self-shadow term. By preprocessing and optimising our algorithm both for the self-shadow in dry hair and the special effects due to water presence, we can get visually pleasing results at interactive rates. Furthermore, the model is highly versatile and can easily be adaptable to other liquids and hair styling products.

With the advent of ubiquitous and pervasive computing environments, one of promising applications is a guidance system. In this paper, we propose a mobile mixed reality guide system for indoor environments, [email protected] A mobile computing device (Sony's Ultra Mobile PC) is hidden inside a jacket and a user selects a destination inside a building through voice commands. A 3D virtual assistant then appears in the see-through HMD and guides him/her to destination. Thus, the user simply follows the virtual guide. [email protected] also suggests the most suitable virtual character (e.g. human guide, dog, cat, etc.) based on user preferences and profiles. Depending on user profiles, different security levels and authorizations for content are previewed. Concerning indoor location tracking, WiFi, RFID, and sensor-based methods are integrated in this system to have maximum flexibility. Moreover smart and transparent wireless connectivity provides the user terminal with fast and seamless transition among Access Points (APs). Different AR navigation approaches have been studied: [Olwal 2006], [Elmqvist et al.] and [Newman et al.] work indoors while [Bell et al. 2002] and [Reitmayr and Drummond 2006] are employed outdoors. Accurate tracking and registration is still an open issue and recently it has mostly been tackled by no single method, but mostly through aggregation of tracking and localization methods, mostly based on handheld AR. A truly wearable, HMD based mobile AR navigation aid for both indoors and outdoors with rich 3D content remains an open issue and a very active field of multi-discipline research.

In this paper, we present a simple and robust mixed reality (MR) framework that allows for real-time interaction with virtual humans in mixed reality environments under consistent illumination. We will look at three crucial parts of this system: interaction, animation and global illumination of virtual humans for an integrated and enhanced presence. The interaction system comprises of a dialogue module, which is interfaced with a speech recognition and synthesis system. Next to speech output, the dialogue system generates face and body motions, which are in turn managed by the virtual human animation layer. Our fast animation engine can handle various types of motions, such as normal key-frame animations, or motions that are generated on-the-fly by adapting previously recorded clips. Real-time idle motions are an example of the latter category. All these different motions are generated and blended on-line, resulting in a flexible and realistic animation. Our robust rendering method operates in accordance with the previous animation layer, based on an extended for virtual humans precomputed radiance transfer (PRT) illumination model, resulting in a realistic rendition of such interactive virtual characters in mixed reality environments. Finally, we present a scenario that illustrates the interplay and application of our methods, glued under a unique framework for presence and interaction in MR.

In this paper, we present a simple and robust mixed reality (MR) framework that allows for real-time interaction with virtual humans in real and virtual environments under consistent illumination. We look at three crucial parts of this system: interaction, animation and global illumination of virtual humans for an integrated and enhanced presence. The interaction system comprises of a dialogue module, which is interfaced with a speech recognition and synthesis system. Next to speech output, the dialogue system generates face and body motions, which are in turn managed by the virtual human animation layer. Our fast animation engine can handle various types of motions, such as normal key-frame animations, or motions that are generated on-the-fly by adapting previously recorded clips. All these different motions are generated and blended on-line, resulting in a flexible and realistic animation. Our robust rendering method operates in accordance with the previous animation layer, based on an extended for virtual humans precomputed radiance transfer (PRT) illumination model, resulting in a realistic display of such interactive virtual characters in mixed reality environments. Finally, we present a scenario that illustrates the interplay and application of our methods, glued under a unique framework for presence and interaction in MR

Abstract Expressive facial animation synthesis of human like characters has had many approaches with good results. MPEG‐4 standard has functioned as the basis of many of those approaches. In this paper we would like to lay out the knowledge of some of those approaches inside an ontology in order to support the modeling of emotional facial animation in virtual humans (VH). Inside this ontology we will present MPEG‐4 facial animation concepts and its relationship with emotion through expression profiles that utilize psychological models of emotions. The ontology allows storing, indexing and retrieving prerecorded synthetic facial animations that can express a given emotion. Also this ontology can be used a refined knowledge base in regards to the emotional facial animation creation. This ontology is made using Web Ontology Language and the results are presented as answered queries. Copyright © 2006 John Wiley & Sons, Ltd.

Robust handling of collisions on non-oriented deformable surfaces requires advanced methods for recovering intersecting surfaces. We present a novel method that resolves intersections between two intersecting surface regions by inducing relative displacements which minimize the length of the intersection contour between them. This method, which does not rely on intersection regions, has a broader application field than existing methods, and its implementation is also much simpler, allowing integration into most existing collision response schemes. We demonstrate the efficiency of this method through examples in the context of cloth simulation.

True real-time animation of complex hairstyles on animated characters is the goal of this work, and the challenge is to build a mechanical model of the hairstyle which is sufficiently fast for real-time performance while preserving the particular behavior of the hair medium and maintaining sufficient versatility for simulating any kind of complex hairstyles. Rather than building a complex mechanical model directly related to the structure of the hair strands, we take advantage of a volume free-form deformation scheme. We detail the construction of an efficient lattice mechanical deformation model which represents the volume behavior of the hair strands. The lattice is deformed as a particle system using state-of-the-art numerical methods, and animates the hairs using quadratic B-Spline interpolation. The hairstyle reacts to the body skin through collisions with a metaball-based approximation. The model is highly scalable and allows hairstyles of any complexity to be simulated in any rendering context with the appropriate trade off between accuracy and computation speed, fitting the need of Level-of-Detail optimization schemes.

Simulating fast and realistic complex light effect within hair is one of the most challenging problems in the field of virtual humans, both in terms of research and of development. In this paper we propose a scattering-based fast and efficient algorithm that handles both the local specular highlights and global self-shadow in animated hair at interactive rates. We introduce a fast refinement technique for incorporating illumination variations in animated hair from the encoded hair density changes considering the spatial coherency in hair data. Our method does pre-computations for any static hairstyle, optimizes it and then efficiently utilizes it along with the refined data for the animated hairstyle. The various optimizations implemented result in interactive simulation while maintaining the aesthetic visual appearance of the hair.

In this paper we present a discussion about believability for Virtual Environments, emotional simulation and also Embodied Conversational Agents (ECAs). We will discuss about the definition of believability and the three elements of believability environments (immersion, presentation and interaction). We also present a discussion about believability and interfaces. Finally, ECA, emotional and personnality simulation are explained and presented.

Abstract A data‐driven approach for the real‐time processing of clothes, particularly suitable for simulating dresses worn by virtual characters, is proposed. It starts, prior to real‐time simulation, by analyzing cloth behavior in relation to the underlying skeleton movement from a presimulated sequence of the cloth obtained using any high‐quality off‐line simulators. The idea is to use this analysis to find an optimal combination of physics‐based simulation and geometric approximation of the simulator; potentially colliding regions are defined on the cloth such that they will hold true for the skeleton movement that closely matches that of presimulated sequence. At runtime, using these analyses, our simulation process provides both visually pleasing results and performance, as long as the motion of the character remains sufficiently close to the original sequence used for the precomputation. The key contributions of this paper are (1) efficient collision handling that prunes out potentially colliding objects by using the off‐line simulation sequence as examples; (2) data‐driven fix‐up process for the coarse mesh simulation that deduces the gross behavior of the cloth; and (3) geometric approximation of the fine mesh deformation, responsible for details in the shape of the cloth such as wrinkles.

Abstract Interactive environments for dynamically deforming objects play an important role in surgery simulation and entertainment technology. These environments require fast deformable models and very efficient collision handling techniques. While collision detection for rigid bodies is well investigated, collision detection for deformable objects introduces additional challenging problems. This paper focuses on these aspects and summarizes recent research in the area of deformable collision detection. Various approaches based on bounding volume hierarchies, distance fields and spatial partitioning are discussed. In addition, image‐space techniques and stochastic methods are considered. Applications in cloth modeling and surgical simulation are presented.

This paper presents an innovative 3D reconstruction of ancient fresco paintings through the real-time revival of their fauna and flora, featuring groups of virtual animated characters with artificial-life dramaturgical behaviours in an immersive, fully mobile augmented reality (AR) environment. The main goal is to push the limits of current AR and virtual storytelling technologies and to explore the processes of mixed narrative design of fictional spaces (e.g. fresco paintings) where visitors can experience a high degree of realistic immersion. Based on a captured/real-time video sequence of the real scene in a video-see-through HMD set-up, these scenes are enhanced by the seamless accurate real-time registration and 3D rendering of realistic complete simulations of virtual flora and fauna (virtual humans and plants) in a real-time storytelling scenario-based environment. Thus the visitor of the ancient site is presented with an immersive and innovative multi-sensory interactive trip to the past. Copyright © 2005 John Wiley & Sons, Ltd.

Abstract Implicit integration methods have contributed to large performance enhancements in the field of simulation of particle‐system mechanical models. While Backward Euler and BDF‐2 methods are now widely used for cloth simulation applications, the Implicit Midpoint method is often overlooked, because of its poor stability properties. It is however as simple to implement as Backward Euler, and offers significantly better accuracy and even shorter computation times. Starting with a study of the stability properties of these integration methods, we will find out how moderate viscous damping may efficiently restore stability of Implicit Midpoint integration method. An extension is also proposed for BDF‐2 methods. Comparative examples demonstrate the benefits of this approach. Copyright © 2005 John Wiley & Sons, Ltd.

The goal of the research is creation of an adaptation mechanism for the delivery of three-dimensional content. The adaptation of content, for various network and terminal capabilities - as well as for different user preferences, is a key feature that needs to be investigated. Current state-of-the art research of the adaptation shows promising results for specific purpose and limited types of content but still it is not well adaptable for the massive heterogeneous environments. We present a method for transmitting adapted thee-dimensional contents to multiple target devices. We present some theoretical and practical methods for adapting three-dimensional contents, which includes shapes and animation. We also discuss practical details to the integration of our methods into MPEG-21 and MPEG-4 architectures.

Although graphical human modeling has been a long sought subject in computer graphics, when it comes to dealing with real-time applications, it raises a number of unique requirements for both CG artists and researchers. For example, for a real-time performance it is highly desirable to lighten the geometry and optimize the runtime animation of characters wherever possible. An important need in such systems is to assist the CG artists in seamlessly transforming their modeling work from CG packages to the real-time application, without limiting their expressivity. In this paper we describe techniques by which dressed human characters are modeled, loaded into VR scenes, and simulated in real-time. In doing so, we point out challenging issues and solutions that such tasks imply at each phase of modeling and simulation. In particular, we turn our attention to what we consider as key technical components: body and clothes. Case studies show that our modelers successfully meet the need of real-time requirements over a wide variety of scenarios.

We present two complementary methodologies to automatically generate 3D surfaces of human bodies including internal bones. The first methodology is a data-driven approach and uses a database of human bodies in conjunction with a learning model synthesizer to create 3D surfaces subject to desired shape and size. In the second methodology, we use medical scanning devices to capture the internal anatomy of the individual and reconstruct 3D surfaces of bones. The final result is an animatable human body including skin external surface as well as the internal skeleton.

To present three-dimensional data both in heavy and light-weight clients, an adaptation scheme is required. Current state-of-the art research shows promising results for specific purposes but it is still not well adoptable for light-weight clients such as mobile devices. In this research, we present a method for transmitting adapted 3D content to multiple target devices. To accomplish this goal, we devised a clustered representation of a multi-resolution model that is flexible, simple, efficient, and works with the MPEG-21 adaptation mechanism.

A data-driven approach for real-time processing of clothes, particularly suitable for simulating dresses worn by virtual characters, is proposed. It starts, prior to realtime simulation, by analyzing cloth behavior in relation to the underlying skeleton movement from a pre-simulated sequence of the cloth obtained using any high quality offline simulators. The idea is to use this analysis to find an optimal combination of physics-based simulation and geometric approximation of the simulator; potentially colliding regions are defined on the cloth such that they will hold true for the skeleton movement that closely matches that of pre-simulated sequence. At runtime, using these analyses, our simulation process provides both visually pleasing results and performance, as long as the motion of the character remains sufficiently close to the original sequence used for the pre-computation. The key contributions of this paper are (1) efficient collision handling that prunes out potentially colliding objects by using the off-line simulation sequence as examples; (2) data-driven fix-up process for the coarse mesh simulation that deduces the gross behavior of the cloth; and (3) geometric approximation of the fine mesh deformation, responsible for details in the shape of the cloth such as wrinkles.

In this paper, we propose an animation approach based on principal component analysis that allows generating two layers of subtle motions: small posture variations and personalised change of balance. Such a motion generator is needed in many cases when one attempts to create an animation sequence out of a set of existing clips. In nature there exists no motionless character, while in computer animation we often encounter cases where no planned actions, such as waiting for another actor finishing his/her part, is implemented as a stop/frozen animation. We identify many situations where a flexible idle motion generator can help: from synchronisation of speech/body animation duration, to dynamic creation of stand still variations in between two active plays. Our approach overcomes the limitations of using a small set of existing clips as a basis for synthesizing idle motions, such as unnatural repetition of movements and difficulties to insert idle motions into an animation without breaking its continuity. A realistic animation is obtained by blending small posture variations with personalised balance shifting animations.

Summary form only given. Appropriate combinations of statistical and geometric methods permit modeling and estimation of complex objects. This paper investigates such combinations in the modeling of human body models in computer graphics. By concentrating on such a particular class of objects, we show how example based techniques together with interpolation ones can exploit existing data and knowledge about body shape and size modelling. Graphical modeling of human bodies obviously has applications in the entertainment industry for character animation and simulations involving people. Other applications range from virtual reality to ergonomic and manufacturing studies.

True real-time animation of complex hairstyles on animated characters is the goal of this work, and the challenge is to build a mechanical model of the hairstyle which is sufficiently fast for real-time performance while preserving the particular behavior of the hair medium and maintaining sufficient versatility for simulating any kind of complex hairstyles.Rather than building a complex mechanical model directly related to the structure of the hair strands, we take advantage of a volume free-form deformation scheme. We detail the construction of an efficient lattice mechanical deformation model which represents the volume behavior of the hair strands. The lattice is deformed as a particle system using state-of-the-art numerical methods, and animates the hairs using quadratic B-Spline interpolation. The hairstyle reacts to the body skin through collisions with a metaball-based approximation. The model is highly scalable and allows hairstyles of any complexity to be simulated in any rendering context with the appropriate tradeoff between accuracy and computation speed, fitting the need of Level-of-Detail optimization schemes.

Abstract This paper describes a generic model for personality, mood and emotion simulation for conversational virtual humans. We present a generic model for updating the parameters related to emotional behaviour, as well as a linear implementation of the generic update mechanisms. We explore how existing theories for appraisal can be integrated into the framework. Then we describe a prototype system that uses the described models in combination with a dialogue system and a talking head with synchronized speech and facial expressions. Copyright © 2004 John Wiley & Sons, Ltd.

This last issue of 2004 contains three papers.The first deals with the rendering of natural waters taking fluorescence into account.This is a step forward to more realism.

Abstract We present a methodology for motion tracking and visualization of the hip joint by combining MR images and optical motion capture systems. MRI is typically used to capture the subject's anatomy while optical systems are used to capture and analyse the relative movement between adjacent bones of the joint. Reflective markers are attached to the subject's skin and their trajectories are tracked and processed. However, the skin surface deforms while in motion due to muscle contraction leading to significant errors in the estimation of trajectories. In order to reduce these errors, we use MR images to capture both the anatomy and the trajectories of the bone. Prior to the scanning, we attach skin markers to the subject in order to analyse the markers displacements relative to the bone. We reconstruct the anatomical models of the subject and we compute the markers trajectories from the images. Using these calculated trajectories, we select the best markers configuration based on the criteria of markers displacements. The optimized configuration is used for recording external movements with the optical motion capture system. The resulting animation is mapped onto the virtual body of the subject including internal bones and the joint motion is visualized. Copyright © 2004 John Wiley & Sons, Ltd.

We present a new method for the modeling and animation of deformable and flexible thin or thick linear bodies. Our method highly reduces the computation required to animate deformations and motions of objects with a cylindrical topology, while preserving their visual quality and their control. To increase the speed we introduce a bi-layered mass-spring system with two main properties: it is not constructed as the whole geometric mesh thus limiting the number of dynamically computed points, and it is partly constrained to reduce physical computations. The visual quality is improved with an appropriate integration scheme to preserve volume as much as possible and to ensure high stability at low cost. The two mass-spring layers and their linkage provide all the required controls on the objects: the first layer, corresponding to an axial flexible backbone structure, is used as a controller for global motions. The second layer, geometrically and dynamically constrained, controls envelope elastic deformations.

In most vehicle simulators, complete physical mockups equipped with steering wheel, gearshift and pedals are required. These devices are essential in trying to simulate real conditions, but as a drawback the system becomes expensive, huge (non mobile), and limited to reflect changes on the vehicle type, dimensions, or interior design. We have implemented different configurations for an immersive vehicle simulator, ranging from the use of a real mockup equipped with force-feedback steering wheel, gearshift and pedals, to fully virtual control metaphors, which are based only on trackers and datagloves. We propose such fully virtual control metaphors as an alternative to minimize the use of physical devices in simulators.

We present the architecture of the VHD++ real-time development framework that after several years of intensive research, design, and development effort has been released and enters its validation phase. We discuss the key aspects involved in architectural structure, design and practical implementation of an efficient, flexible and extendible real-time software framework based on the modern 3D game-engine design principles. This framework supports researchers and application developers with rapid, component based development of VR/AR systems featuring advanced virtual character simulation technologies. The discussion covers motivation, main concepts, survey of related work, the main functional and design requirements, design principles and key architectural elements. It concludes with the initial validation results including overview of existing VHD++ based VR/AR virtual character simulation applications.

While level of detail (LoD) methods for the representation of 3D models are efficient and established tools to manage the trade-off between speed and quality of the rendering, LoD for animation has not yet been intensively studied by the community, and especially virtual humans animation has not been focused in the past. Animation, a major step for immersive and credible virtual environments, involves heavy computations and as such, it needs a control on its complexity to be embedded into real-time systems. Today, it becomes even more critical and necessary to provide such a control with the emergence of powerful new mobile devices and their increasing use for cyberworlds. With the help of suitable middleware solutions, executables are becoming more and more multi-platform. However, the adaptation of content, for various network and terminal capabilities - as well as for different user preferences, is still a key feature that needs to be investigated. It would ensure the adoption of the "Multiple Target Devices Single Content" concept for virtual environments, and would in theory provide the possibility of such virtual worlds in any possible condition without the need for multiple content. It is on this issue that we focus, with a particular emphasis on 3D objects and animation. This paper presents some methods for adapting a virtual human's representation and animation stream, both for their skeleton-based body animation and their deformation based facial animation, we also discuss practical details for the integration of our methods into MPEG-21 and MPEG-4 architectures.

In this paper, we present an automatic, runtime modeler for modeling realistic, animatable human bodies. A user can generate a new model or modify an existing one simply by inputting a number of sizing parameters. We approach the problem by forming deformation functions that are devoted to the generation of appropriate shape and proportion of the body geometry by taking the parameters as input. Starting from a number of 3D scanned data of human body models as examples, we derive these functions by using radial basis interpolation. A prerequisite of such formulation is to have correspondence among example models in the database. We obtain the correspondence by fitting a template onto each scanned data. Throughout the paper, body geometry is considered to have two distinct entities, namely rigid and elastic component of the deformation. The rigid deformation is represented by the corresponding joint parameters, which will determine the linear approximation of the physique. The elastic deformation is essentially vertex displacements, which, when added to the rigid deformation, depicts the detail shape of the body. Having these interpolators formulated, the runtime modeling can be reduced to the function evaluation and application of the evaluated results to the template model. We demonstrate our method by applying different parameters to generate a wide range of different body models.

The tutorial will explain the techniques for integrating virtual humans into virtual and real environments. It will first present the concept of avatars and autonomous virtual actors and the main techniques to create and animate them (body and face). It will present the concepts of behavioral animation, crowd simulation, intercommunication between virtual humans, and interaction between real humans and autonomous virtual humans. The tutorial will also address the advanced topics of real-time cloth animation with application in fashion design. Finally, the tutorial will present the use of the virtual human technology in virtual heritage, social phobia, training, and health emergency.

Virtual environments containing humans moving and performing actions in real time are ubiquitous in computer graphics applications. Such characters are often clothed in woven fabrics. This paper presents a technique for visualizing woven clothes in real time, while optimizing the realistic appearance. The proposed approach supports rendering of complex weave patterns by adopting Weaving Information File (WIF), a standard from textile Computer Aided Design (CAD) for representing the grammar of weaving. We develop a realistic rendering scheme by combining the grammar representation obtained from the WIF with a procedural thread texture, a suitable cloth Bi-directional Reflectance Distribution Function (BRDF) and horizon maps. We employ the multi-texturing approach to meet the real time constraint. Thus our approach to visualizing woven clothes begins from weaving grammar specifications and converts them into textures that can be applied on clothes. We demonstrate the versatility of the proposed approach with examples.

Sound rendering requires that many different aspects are considered simultaneously, especially when rendering a real-time virtual environment. In 3D sound rendering, much the same as for graphics, one of the major influencing factors is the number of reflective polygons in a scene and due to the increase in the ability of most common graphics cards this number can now be very high, especially when scene designers produce an optimum scene using other optimizing tools such as Polygon Cruncher or Rational Reducer. In addition, the use of programs such as Lightscape™ [20], which is used to produce realistic lighting, by using per vertex shading, increases the number of polygons in a scene by several factors. Therefore a strong, pre-processing method is proposed that dramatically reduces the number of polygons in the scene to a suitable level for real-time sound rendering. The method can also be combined with other methods (e.g. scene partitioning) for even lower CPU usage.

Abstract Visemes are visual counterpart of phonemes. Traditionally, the speech animation of 3D synthetic faces involvesextraction of visemes from input speech followed by the application of co‐articulation rules to generate realisticanimation. In this paper, we take a novel approach for speech animation — using visyllables, the visual counterpartof syllables. The approach results into a concatenative visyllable based speech animation system. The key contributionof this paper lies in two main areas. Firstly, we define a set of visyllable units for spoken English along withthe associated phonological rules for valid syllables. Based on these rules, we have implemented a syllabificationalgorithm that allows segmentation of a given phoneme stream into syllables and subsequently visyllables. Secondly,we have recorded the database of visyllables using a facial motion capture system. The recorded visyllableunits are post‐processed semi‐automatically to ensure continuity at the vowel boundaries of the visyllables. We defineeach visyllable in terms of the Facial Movement Parameters (FMP). The FMPs are obtained as a result of thestatistical analysis of the facial motion capture data. The FMPs allow a compact representation of the visyllables.Further, the FMPs also facilitate the formulation of rules for boundary matching and smoothing after concatenatingthe visyllables units. Ours is the first visyllable based speech animation system. The proposed technique iseasy to implement, effective for real‐time as well as non real‐time applications and results into realistic speechanimation. Categories and Subject Descriptors (according to ACM CCS): 1.3.7 [Computer Graphics]: Three‐Dimensional Graphics and Realism

This paper presents a hip joint motion simulation method using accurate hip joint center (HJC) and hip range of motion. We calculate the extreme hip joint range of motion centered on the HJC that is located by an automatic calculation algorithm on 3D reconstructed surface models. We make 3D bone surface models from computer tomography or magnetic resonance imaging. The medical objective is to quantify hip kinematics in the function of hip morphology. Since only extreme ranges of motions are evaluated from bones, we can estimate the motions that are restricted by bones impingements by comparing the computed values with real extreme range of motion.

While mechanical cloth simulation systems are widely used for creating draped garments on virtual characters, the animation of virtual garments raise challenges on its own. The perception of garment beauty is first related to the dynamic accuracy of the cloth simulation model, which has to reproduce precisely the dissipative behaviors of the material, such as viscosity and plasticity, rendering the visual features of realistic cloth motion. The other important aspect is the need of adequate techniques for the real-time visualization of animated virtual garments, required for many real-time or interactive applications. This work present techniques yielding visible advances for addressing these two aspects.

Simulating virtual environments populated with virtual but realistic crowds requires dozens of different face and body geometries. We present GENESIS (GENeration of E-population based on Statistical Information), an application that allows automatic generation of desired population models. The aim of this application is to generate any population group that is statistically calculated to satisfy given properties. We divide the population generation module into the face part and the body part. Each of them is based on a database, which is an organization of information collected and processed from the real population dataset. Upon the specification of the population parameters by the user, GENESIS transforms them onto a number Of queries to the underlying databases, and process the results from the database in a statistical manner, allowing dynamic generation of desired number of individuals while satisfying the given properties of on the overall population.

In this paper we describe a filter that can be used to dramatically reduce the amount of data flowing between clients connected in a networked virtual environment, when using the MPEG4 animation parameters. The approach consists of three parts: real life experimentation and parameter determination, filter construction and filter testing. The real life experiments are used to determine base figures for the filter construction, the filter construction details aspects of design and visual compensation and the final evaluations are used to establish if the filter works correctly, both for data reduction and aesthetically and how well.

In this paper, we describe a method for cloth animation in real-time. The algorithm works in a hybrid manner exploiting the merits of both the physical-based and geometric deformations. It makes use of predetermined conditions between the cloth and the body model, avoiding complex collision detection and physical deformations wherever possible. Garments are segmented into pieces that are simulated by various algorithms, depending on how they are laid on the body surface and whether they stick or flow on it. Tests show that the method is well suited to fully dressed virtual human models, achieving real-time performance compared to ordinary cloth-simulations.

Their contribution is in the field of image segmentation.They observe the possibility of using the overall and class-conditional error rates as criteria for selecting the optimal sampling of the linear and morphological scale spaces.

This paper presents a computational model for studying the mechanical properties of skin with aging. In particular, attention is given to the folding capacity of skin, which may be manifested as wrinkles. The simulation provides visual results demonstrating the form and density of folds under the various conditions. This can help in the consideration of proper measures for a cosmetic product for the skin.

We discuss how to morph between several different human body-shapes. The main idea is to create one human body structure from given human body shape data (either from photographs, designer's tool, laser scanner or infrared scanner), then use a rapid 3D-morphing system to create different versions of the body shape. This is done using interpolation and 2D-image metamorphosis based on triangulation where the shared structure is inherited from the generic human. It is an integration of 3D shape and 2D skin information. This system enables a user to intuitively alter a virtual human's shape and skin color. Automatic texture clothing methods are also introduced with several examples. We also describe the H-Anim body data structure as our generic body structure of choice.

We describe a new technique for expressive and realistic speech animation. We use an optical tracking system that extracts the 3D positions of markers attached at the feature point locations to capture the movements of the face of a talking person. We use the feature points as defined by the MPEG-4 standard. We then form a vector space representation by using the principal component analysis of this data. We call this space "expression and viseme space". Such a representation not only offers insight into improving realism of animated faces, but also gives a new way of generating convincing speech animation and blending between several expressions. As the rigid body movements and deformation constraints on the facial movements have been considered through this analysis, the resulting facial animation is very realistic.

Any cloth simulation system needs efficient numerical methods for integrating the equations that describe the mechanical behavior of the discrete representation of the cloth. Choosing the adequate method should be done with full knowledge of the advantages and weaknesses of the main techniques. This paper presents a quantitative comparison of the efficiency of the most common integration techniques used for cloth simulation, and raises the key considerations for optimal implementations depending on the practical kind of simulation problematic.

We present our system for personalized face and speech communication over the Internet. The overall system consists of three parts: the cloning of real human faces to use as the representative avatars; the Networked Virtual Environment System performing the basic task of network and device management; and the speech system which includes a text-to-speech engine and a real time phoneme extraction engine from natural speech. The combination of these three elements provides a system to allow real humans, represented by their virtual counterparts, to communicate with each other even when they are geographically remote. In addition to this, all elements present use MPEG-4 as a common communication and animation standard and were designed and tested on the Windows operating system (OS). The paper presents the main aim of the work, the methodology and the resulting communication system.

In this paper we address the difficult problem of hair dynamics, particularly hair-hair and hair-air interactions. To model these interactions, we propose to consider hair volume as a continuum. Subsequently, we treat the interaction dynamics to be fluid dynamics. This proves to be a strong as well as viable approach for an otherwise very complex phenomenon. However, we retain the individual character of hair, which is vital to visually realistic rendering of hair animation. For that, we develop an elaborate model for stiffness and inertial dynamics of individual hair strand. Being a reduced coordinate formulation, the stiffness dynamics is numerically stable and fast. We then unify the continuum interaction dynamics and the individual hair's stiffness dynamics.

Abstract In this paper, we present a framework for both skin and cloth deformation. Traditionally, skin and cloth deformations are managed by a non‐unified strategy although the skin interacts on clothes and vice versa. In general, the skin is considered as a rigid surface on which the clothes have no effect. We propose in this paper a unified approach for cloth and body deformation. Our system is able to handle both skin and cloth deformation with collision response. These deformations are managed with a particle system. The skin deformation model is a hybrid model, where deformation due to the skeleton motion is controlled geometrically, and where the collision response is handled by a physical‐based model. In case of collision among skin and clothes, a reaction and friction forces are applied on both skin and cloth to avoid penetration. Copyright © 2001 John Wiley & Sons, Ltd.

present a virtual mobile or moving sculpture system in the third paper.In order to display the virtual mobile interactively, the authors describe the use of the virtual wind model, the constraint dynamics solver and the impulse dynamics solver incorporated in the developed system.The fourth and last paper by G. Besuievsky and X. Pueyo from the University de Girona presents an efficient algorithm to compute animations in radiosity environments.The authors claim that the method is an alternative for computing non-interactive radiosity animations for moderately complex scenarios.

We present an integrated system for skin deformation that is able to handle deformations due to both the skeleton animation and collisions. This method is based on a physically-based deformation of a skin surface linked to a reference shape; the reference shape is animated with a geometric deformation. Collision detection has been implemented to avoid penetration. Our method for collision response consists of a kinematical correction of positions and forces during the simulation. The resulting model of the skin deformation enables an efficient simulation and interaction of the skin with deformable objects. Results are presented as examples.

We present a very general geometrical correction method for enforcing collisions and other geometrical constraints between polygonal mesh surfaces. It is based on a global resolution scheme that takes advantage of an efficient use of the conjugate gradient algorithm to find the appropriate displacement of the mesh vertices that would satisfy all the constraints simultaneously, and according to momentum conservation laws. This method has been implemented in a cloth simulation system along with a collision response model that enforces a minimum "thickness" distance between cloth surfaces, which can be efficiently integrated in an simulation scheme based on implicit integration. Some provided examples illustrate the efficiency of the method.

We present a system for the animation of a human hand that plays violin. A neural network controls the hand movement. We make use of an optimization method to generate examples for the neural network training. The musical decision of which finger to use is automatically made by best first search. We show that the movements of the violinist's hands are physically and musically feasible, and that the musical decisions are consistent with those recommended in the violin pedagogy. A description of the system, the results of the decisions, and the animations are presented.

Presents the VPARK (Virtual Park) system, which includes a networked virtual environment (NVE) system called W-VLNET and an "attraction building system" that is able to create and modify the attractions used in the NVE. Both systems have been developed in the Windows NT environment. The paper outlines the techniques for communication, scene management, facial and body animation, and general user interaction modules. The use of VRML97 and MPEG-4 SHNC is overviewed for the purpose of outlining the compatability of the system with other similar virtual reality systems. The software provides realistic virtual actors as well as sets of high-level actions that are applicable to them in real-time. Related issues on obtaining actor models and animating them in real time are presented. The creation process of an attraction incorporates assembling animation units through a timeline. Using this software, the users are able to bring their own scenario-based applications into a shared virtual environment.

The article details and implements efficient techniques for cloth simulation, both in the area of numerical simulation and the area of collision detection and response. Emphasis is put on the efficient implementation of implicit numerical methods with many improvements toward better realism, as well as computation simplicity. A constraint based collision response scheme is adapted to this scheme in order to provide an accurate and stable collision response.

Presents our work on a level-of-detail (LoD) technique for human-like face models in virtual environments. Conventional LoD techniques have been adapted to allow facial animation on simplified geometric models. This includes the optimization of both geometric and animation parameters. Simplified models are generated in a region-based manner, considering the mobility of each region. The animation process is decomposed into two sub-processes, and each step is optimized. In the MPA (minimum perceptible action) level optimization, a hierarchical structure is devised for the multi-level animation model. The deformation level is simplified by reducing the number of control points. At run-time, the animation level is selected in combination with viewpoint information at the geometric level.

In this paper we present our Networked Virtual Environment (NVE) System, called W-VLNET (Windows Virtual Life Network), which has been developed on the Windows NT Operating System (OS). This paper emphasizes the Real-Time aspect of this NVE system, the advanced interactivity that the system provides and its ability to transfer data across the Internet so that geographically distant users can collaborate with each other. Techniques for communication, scene management, facial and body animation, and general user interaction modules are detailed in this paper. The use of VRML97 and MPEG4 SHNC is overviewed to stress the compatibility of the system with other similar Virtual Reality systems. The software provides realistic virtual actors as well as sets of applicable high-level actions in real-time. Related issues on obtaining actor models and animating them in real-time are presented. We also introduce a case study to show an example of how the system can be used.

We present an easy, practical and efficient full body cloning methodology. This system utilizes photos taken from the front, side and back of a person in any given imaging environment without requiring a special background or a controlled illuminating condition. A seamless generic body specified in the VRML H‐Anim 1.1 format is used to generate an individualized virtual human. The system is composed of two major components: face‐cloning and body‐cloning. The face‐cloning component uses feature points on front and side images and then applies DFFD for shape modification. Next a fully automatic seamless texture mapping is generated for 360° coloring on a 3D polygonal model. The body‐cloning component has two steps: (i feature points specification, which enables automatic silhouette detection in an arbitrary background (ii two‐stage body modification by using feature points and body silhouette respectively. The final integrated human model has photo‐realistic animatable face, hands, feet and body. The result can be visualized in any VRML compliant browser.

Ohtsuka from the Nippon Telegraph and

Columbia University present a method of computation of swept volumes that permits an arbitrary polyhedral object to be swept through an arbitrary trajectory.The method has an advantage above other methods in terms of ¯exibility and robustness.The authors of the second paper, H. Sun and M. Green from the University of Alberta, present a framework for interactive animation that allows an animator to specify the reactions of the objects to events generated by other objects in the 3D scene and also by the user interacting with it.This approach enables the animators to compose interactively complex scene behaviors.The third paper, by

Medical imaging can provide data for useful views of the interior details of human anatomy. In addition to visualization, which in general has been the primary reason for obtaining these data, many other uses are possible. These include modeling of different elements and their inter-relationships-topological modeling, simulation of physical processes, analysis of movements, and validation of models. Here, we describe some of the modeling issues from medical imaging. The issues are particularly related to topological modeling of different anatomical elements: bones, muscles, articulations, etc. A three-dimensional topological modeler is presented with which anatomists and other users can build a topological database containing structural, topological, and mechanical information of anatomical elements.

MPEG-4 is scheduled to become an international standard in March 1999. The paper demonstrates an experiment for a virtual cloning method and animation system, which is compatible with the MPEG-4 standard facial object specification. Our method uses orthogonal photos (front and side view) as input and reconstructs the 3D facial model. The method is based on extracting MPEG-4 face definition parameters (FDP) from photos, which initializes a custom face in a more capable interface, and deforming a generic model. Texture mapping is employed using an image composed of the two orthogonal images, which is done completely automatically. A reconstructed head can be animated immediately inside our animation system, which is adapted to the MPEG-4 standard specification of face animation parameters (FAP). The result is integrated into our virtual human director (VHD) system.

We propose a layered facial simulation model for skin aging with wrinkles, which includes muscle, connective tissue and skin layer. Our aim is to simulate relevant facial animation and aging with the guidance of general facial tissue anatomy, so that the model can be extended to medical and cosmetic applications. B-spline muscle patches are automatically adapted to each individual face by mapping the anatomical facial muscle image. Connective tissues are simulated as simple springs with the length of hypodermis thickness that constrain skin movement. Facial skin deformation and aging are estimated based on an elaborated biomechanical model considering large strain deformation and wrinkle formation. Finally, multi-layered color and bump texture mapping are used to represent wrinkle forms and to render an aged face.

Human interfaces for computer graphics systems are now evolving towards a total multi-modal approach. Information gathered using visual, audio and motion capture systems are now becoming increasingly important within user-controlled virtual environments. This paper discusses real-time interaction through the visual analysis of human face feature. The underlying approach to recognize and analyze the facial movements of a real performance is described in detail. The output of the program is directly compatible with MPEG-4 standard parameters and therefore enhances the ability to use the available data in any other MPEG-4 compatible application. The real-time facial analysis system gives the user the ability to control the graphics system by means of facial expressions. This is used primarily with real-time facial animation systems, where the synthetic actor reproduces the animator's expression. The MPEG-4 standard mainly focuses on networking capabilities and it therefore offers interesting possibilities for teleconferencing, as the requirements for the network bandwidth are quite low.