Spatio-temporal information is key to resolve occlusion and depth ambiguity in 3D human pose estimation. Previous methods have focused on either temporal contexts or local-to-global architectures that embed fixed-length spatiotemporal information. To date, there have not been effective proposals to simultaneously and flexibly capture varying spatiotemporal sequences and effectively achieves real-time 3D human pose estimation. In this work, we improve the learning of kinematic constraints in the human skeleton: posture, local kinematic connections, and symmetry by modeling local and global spatial information via attention mechanisms. To adapt to single- and multi-frame estimation, the dilated temporal model is employed to process varying skeleton sequences. Also, importantly, we carefully design the interleaving of spatial semantics with temporal dependencies to achieve a synergistic effect. To this end, we propose a simple yet effective graph attention spatio-temporal convolutional network (GAST-Net) that comprises of interleaved temporal convolutional and graph attention blocks. Experiments on two challenging benchmark datasets (Human3.6M and HumanEva-I) and YouTube videos demonstrate that our approach effectively mitigates depth ambiguity and self-occlusion, generalizes to half upper body estimation, and achieves competitive performance on 2D-to-3D video pose estimation. Code, video, and supplementary information is available at: http://www.juanrojas.net/gast/

As an important technology in LBS (Location Based Services) field, Wi-Fi based indoor localization suffers signal fluctuation problem which prevents lifelong and high performance running. With the fluctuation of wireless signal over time, fingerprints collected at the same location become different; therefore existing model cannot fit the new collected data well, which decreases the localization accuracy. In this paper, a novel indoor localization method COSELM (Constraint Online Sequential Extreme Learning Machine) is proposed, utilizing incremental data to update the old model and overcome the fluctuation problem. The performance of COSELM is validated in real Wi-Fi indoor environment. Compared with OSELM, it can improve more than 5% localization accuracy on average; and in contrast to batch learning, COSELM can save more than 50% time consumption.

Bluetooth information can efficiently capture characteristics of user-centric surrounding contexts, such as formal meeting or chatting with friends, shopping with friends or alone, etc. In this paper, we extract novel features from Bluetooth traces and use these features for recognizing contextual behavior as well as inferring continuous episode transition. Evaluation results show that extracted novel features are very effective, which enable the model to achieve an average of 87% accuracy for specific context classification and the ability of episode inference from real-life Bluetooth traces.

Traditional keyboard and mouse based presentation prevents lecturers from interacting with the audiences freely and closely. In this paper, we propose a gesture-aware presentation tool named SlideShow to liberate lecturers from physical space constraints and make human-computer interaction more natural and convenient. In our system, gesture data is obtained by a handle controller with 3-axis accelerometer and gyro and transmitted to host-side through bluetooth, then we use Bayesian change point detection to segment continuous gesture series and HMM to recognize the gesture. In consequence Slideshow could carry out the corresponding operations on PowerPoint(PPT) to make a presentation, and operation states can be switched automatically and intelligently during the presentation. Both the experimental and testing results show our approach is practical, useful and convenient.

Activity recognition system is the key part in E-Health field. Traditional system needs more labeled training data to meet higher recognition accuracy. This means more calibration effort and time consumption. In this paper, with collaboration of heterogeneous multimodal sensors like a microphone, a camera and an accelerometer etc, we propose to design and implement a system to reduce the required amount of labeled data as well as achieve even better performance than traditional systems. The system consists of three phases: collaborative data collection, collaborative classifier training and collaborative classifier combination. The experimental results validate that with only 9% labeled data, our system can obtain as high accuracy as other systems which use 100% unimodal labeled data.

Activity recognition using mobile phones has great potential in many applications including mobile healthcare. In order to let a person easily know whether he is in strict compliance with the doctor’s exercise prescription and adjust his exercise amount accordingly, we can use a smart-phone based activity reporting system to accurately recognize a range of daily activities and report the duration of each activity. A triaxial accelerometer embedded in the smart phone is used for the classification of several activities, such as staying still, walking, running, and going upstairs and downstairs. The model learnt from a specific person often cannot yield accurate results when used on a different person. To solve the cross-people activity recognition problem, we propose an algorithm known as TransEMDT (Transfer learning EMbedded Decision Tree) that integrates a decision tree and the k-means clustering algorithm for personalized activity-recognition model adaptation. Tested on a real-world data set, the results show that our algorithm outperforms several traditional baseline algorithms.

Recently, 3D caricature generation and applications have attracted wide attention from both the research community and the entertainment industry. This paper proposes a novel interactive approach for various and interesting 3D caricature generation based on double sampling. Firstly, according to user's operation, we obtain a coarse 3D caricature with local features transformation by sampling in well-built principle component analysis (PCA) subspace. Secondly, to utilize information of the 2D caricature dataset, we sample in the local linear embedding (LLE) manifold subspace. Finally, we use the learned 2D caricature information to further refine the coarse caricature by applying Kriging interpolation. The experiments show that the 3D caricature generated by our method can preserve highly artistic styles and also reflect the user's intention.

Abstract Recently, automatic 3D caricature generation has attracted much attention from both the research community and the game industry. Machine learning has been proven effective in the automatic generation of caricatures. However, the lack of 3D caricature samples makes it challenging to train a good model. This paper addresses this problem by two steps. First, the training set is enlarged by reconstructing 3D caricatures. We reconstruct 3D caricatures based on some 2D caricature samples with a Principal Component Analysis (PCA)‐based method. Secondly, between the 2D real faces and the enlarged 3D caricatures, a regressive model is learnt by the semi‐supervised manifold regularization (MR) method. We then predict 3D caricatures for 2D real faces with the learnt model. The experiments show that our novel approach synthesizes the 3D caricature more effectively than traditional methods. Moreover, our system has been applied successfully in a massive multi‐user educational game to provide human‐like avatars.

This paper proposes a system for 3D facial caricature synthesis in the interactive mode. In our system, three steps are involved: The first step is to construct a 3D facial caricature database involving in head and facial components. The second step is to define the diverse styles for each facial component using clustering method. The last step is to personal caricature designing by replacing various facial components. The main feature of our system is that it can produce easily expressive 3D caricature just needing some simple operations, such as clicking the mouse button. And the result of experiments show it is convenient and efficient to create vivid caricature by personalized designing way. We also build it successfully into a Maya plug-in, which may be applied widely because there are large numbers of users of Maya.

3D caricature generation is becoming an interesting and important research program, but it is impossible for the computer to draw an elegant caricature without any background. In this paper, we build a training set which contains 110 true face photographs and corresponsive 3D caricature models. Based on that training set, the LLE manifold learning is performed to discover the low dimensional embeddings. Between the 2D true face embeddings and 3D caricature embeddings, a regressive model is learnt by the Extreme Learning Machine. Experiments show that, the regressive manifold model is effective to generate the final 3D caricature for a 2D facial photograph.

In a Wi-Fi based indoor localization system (WILS), mobile clients use received Wi-Fi signal strength to determine their locations. A major problem is the variation of signal distributions caused by multiple factors, which makes the old localization model inaccurate. Therefore, the transfer learning problem in a WILS aims to transfer the knowledge from an old model to a new one. In this paper, we study the characteristics of signal variation and conclude the chief factors as time and devices. An algorithm LuMA is proposed to handle the transfer learning problem caused by these two factors. LuMA is a dimensionality reduction method, which learns a mapping between a source data set and a target data set in a low-dimensional space. Then the knowledge can be transferred from source data to target data using the mapping relationship. We implement a WILS in our wireless environment and apply LuMA on it. The on-line performanceevaluation shows that our algorithm not only achieves better accuracy than the baselines, but also has ability for adaptive localization, regardless of time or device factors. As a result, the calibration efforts on new training data can be greatly reduced.

In this paper, we propose a new kind of WiFi-based embedded device and two power efficient killer applications on it. First, the definition of our pervasive device and the key challenge research issues on it is introduced. Then we specially present our work on it, including hardware and software part. For hardware part, we will show our new Loongson SOC (system on a chip) chip based hardware architecture, which is power efficient and flexible connective interface one. For software part, we focus on two killer applications for this kind of pervasive device: location estimation and video codec. The power aware method we used in above two applications will be introduced in detail.

Geometry-controlled image warping performs well in exhibiting shape variations but bad in exhibiting wrinkle such as fossette. Successfully applied in expression cloning, expression ratio image (ERI) provides a way to quantize transferable common wrinkle. However, the warping used in ERI makes it difficult to restrain alignment error, which however prevents further analysis such as PCA. In this paper, flexible expression ratio image (FERI) is defined and further compressed as eigen FERI with PCA to quantize compactly common wrinkle. Based on FERI, a geometry-mapped mechanism consisting of shape-varying mechanism and wrinkling mechanism is constructed to synthesize and transfer realistic facial animation. Given the geometric parameter facial animation parameter (FAP), the shape variation deriving from shape-varying mechanism and the wrinkle deriving from wrinkling mechanism combine to generate realistic facial animation. Having no need for any example photo gallery but only two frontal neutral photos respective captured from source face and target face, as demonstrated in the experiment, the geometry-mapped mechanism based on FERI can map FAP series to the synchronous realistic source animation and target animation.

This paper proposes a mapping learning approach for caricature auto-generation. Simulating the artist’s creativity based on the object’s facial feature, our approach targets discovering what are the principal components of the facial features, and what’s the difference between facial photograph and caricature measured by those components. In training phase, PCA approach is adopted to obtain the principal components. Then, machine learning of SVR (Support Vector Regression) is carried out to learn the mapping model in principal component space. With the mapping model, in application phase, users just need to input a frontal facial photograph for the caricature generation. The caricature is exaggerated based on the original face while reserving essential similar features. Experiments proved comparatively that our approach could generate more harmonious caricatures.