This paper considers the challenging problem of 3D Human Pose Estimation (HPE) from a sparse set of Inertial Measurement Units (IMUs). Existing efforts typically reconstruct a pose sequence by either directly tackling whole-body motions or focusing on distinctive spatio-temporal features of local body parts. Unfortunately, these methods ignore existing interdependent motor synergies amongst body parts, which may lead to pose estimation with ambiguous local parts. This observation motivates us to propose a hierarchical learning-based approach, HiPoser, which utilizes a hierarchical shared structure using Mamba blocks as the backbone to focus on the following estimation tasks, involving: 1) torso pose, 2) lower limbs pose, 3) upper limbs pose, and finally 4) global translation. These tasks selectively incorporate body motion states and are to be carried out sequentially in reconstructing part-based poses, which are amalgamated to estimate the final full-body pose with the global translation that satisfies inter-part consistencies. Our hierarchical structure allows HiPoser the flexibility in prioritizing different aspects of pose estimation, to emphasize more on detail or stability. Empirical evaluations over three benchmark datasets demonstrate the superiority of HiPoser over existing state-of-the-art models, suggesting that analyzing the synergistic movement of body parts is indeed important for advancing IMU-based 3D HPE.

Wearable Human Activity Recognition (WHAR) is a prominent research area within ubiquitous computing. Multi-sensor synchronous measurement has proven to be more effective for WHAR than using a single sensor. However, existing WHAR methods use shared convolutional kernels for indiscriminate temporal feature extraction across each sensor variable, which fails to effectively capture spatio-temporal relationships of intra-sensor and inter-sensor variables. We propose the DecomposeWHAR model consisting of a decomposition phase and a fusion phase to better model the relationships between modality variables. The decomposition creates high-dimensional representations of each intra-sensor variable through the improved Depth Separable Convolution to capture local temporal features while preserving their unique characteristics. The fusion phase begins by capturing relationships between intra-sensor variables and fusing their features at both the channel and variable levels. Long-range temporal dependencies are modeled using the State Space Model (SSM), and later cross-sensor interactions are dynamically captured through a self-attention mechanism, highlighting inter-sensor spatial correlations. Our model demonstrates superior performance on three widely used WHAR datasets, significantly outperforming state-of-the-art models while maintaining acceptable computational efficiency.

Fashion image editing is a valuable tool for designers to convey their creative ideas by visualizing design concepts. With the recent advances in text editing methods, significant progress has been made in fashion image editing. However, they face two key challenges: spurious correlations in training data often induce changes in other areas when editing an area representing the intended editing concept, and these models typically lack the ability to edit multiple concepts simultaneously. To address the above challenges, we propose a novel Text-driven Fashion Image ediTing framework called T-FIT to mitigate the impact of spurious correlation by integrating counterfactual reasoning with compositional concept learning to precisely ensure compositional multi-concept fashion image editing relying solely on text descriptions. Specifically, T-FIT includes three key components. (i) Counterfactual abduction module, which learns an exogenous variable of the source image by a denoising U-Net model. (ii) Concept learning module, which identifies concepts in fashion image editing—such as clothing types and colors and projects a target concept into the space spanned from a series of textual prompts. (iii) Concept composition module, which enables simultaneous adjustments of multiple concepts by aggregating each concept’s direction vector obtained from the concept learning module. Extensive experiments show that our method can achieve state-of-the-art performance on various fashion image editing tasks, including single-concept editing (e.g., sleeve length, clothing type) and multi-concept editing (e.g., color & sleeve length).

A key challenge for controllable image editing is that visual attributes with semantic meanings are not always independent, resulting in spurious correlations in model training. However, most existing methods ignore such issues, leading to biased causal visual representation learning and unintended changes to unrelated regions or attributes in the edited images. To bridge this gap, we propose a diffusion-based causal visual representation learning framework called CIDiffuser to capture causal representations of visual attributes based on structural causal models to address the spurious correlation. Specifically, we first decompose the image representation into a high-level semantic representation for core attributes of the image and a low-level stochastic representation for other random or less structured aspects, with the former extracted by a semantic encoder and the latter derived via a stochastic encoder. We then introduce a causal effect learning module to capture the direct causal effect, that is, the difference of potential outcomes before and after intervening on the visual attributes. In addition, a diffusion-based learning strategy is designed to optimize the representation learning process. Empirical evaluations on two benchmark datasets demonstrate that our approach significantly outperforms state-of-the-art methods, enabling highly controllable image editing by modifying learned visual representations.

Imbalanced Time-Series Classification is a critical, yet challenging task across a spectrum of real-world applications. Previous oversampling and generative approaches primarily target the minority class and often rely on static decision boundaries or similarity-based heuristics. However, these methods overlook the underlying causal factors that govern the distinction between majority and minority classes, particularly in scenarios with ambiguous class boundaries. As a result, the generated samples may fail to enhance class separability, thereby limiting improvements in classification performance. To this end, we propose a CounterFactual Augmentation Minority Generation (CFAMG) method based on generative models that aims to discover the causal factors that determine different classes from a causality perspective. Specifically, our method first utilizes a disentangled classifier to distinguish between causal and non-causal factors. Next, we perform counterfactual intervention by replacing the causal factors of majority class samples with those from minority class samples, creating an intervened latent representation that reflects minority characteristics while preserving essential structures. Finally, the trained minority class decoder generates counterfactual minority samples that resemble real minority instances yet remain distinguishable from the original majority class. Extensive experiments demonstrate that our method outperforms state-of-the-art methods in both univariate and multivariate imbalanced time-series classification tasks. The code is published at https://github.com/WangLei-CQU/CFAMG.

A significant challenge in air quality index (AQI) prediction is to accurately evaluate the potential outcomes after conducting interventions in pollutant factors such as industrial emissions for each enterprise. Existed methods often suffer from spurious correlations caused by unmeasured confounders and are lack of interpretability of the model, leading to sub-optimal prediction performance. This motivates us to propose a causal AQI prediction framework (CAP) that employs a structural causal model (SCM) to characterize the causal structural variability of various AQI factors for robust AQI prediction. Specifically, we employ the front-door adjustment to explicitly eliminate unmeasured confounders by intervening in industrial emissions from the target enterprise. Meanwhile, we take industrial emissions of neighboring enterprises into account when intervening in the target enterprise and simulate the dispersion of industrial emissions through a Gaussian plume model based on meteorological factors. Experiments on two real-world datasets validate the superior performance of our model on AQI prediction compared to the state-of-the-art baselines.

Predicting spatio-temporal traffic flow presents significant challenges due to complex interactions between spatial and temporal factors. Existing approaches often address these dimensions in isolation, neglecting their critical interdependencies. In this paper, we introduce theSpatio-TemporalUnitizedModel (STUM), a unified framework designed to capture both spatial and temporal dependencies while addressing spatio-temporal heterogeneity through techniques such as distribution alignment and feature fusion. It also ensures both predictive accuracy and computational efficiency. Central to STUM is the Adaptive Spatio-temporal Unitized Cell (ASTUC), which utilizes low-rank matrices to seamlessly store, update, and interact with space, time, as well as their correlations. Our framework is also modular, allowing it to integrate with various spatio-temporal graph neural networks through components such as backbone models, feature extractors, residual fusion blocks, and the predictor to collectively enhance forecasting outcomes. Experimental results across multiple real-world datasets demonstrate that STUM consistently improves prediction performance with minimal computational cost. These findings are further supported by hyperparameter optimization, ablation studies, and result visualization. We provide our source code for reproducibility athttps://github.com/RWLinno/STUM

The ever-increasing demands for intuitive interactions in virtual reality have led to surging interests in facial expression recognition (FER). There are however several issues commonly seen in existing methods, including narrow receptive fields and homogenous supervisory signals. To address these issues, we propose in this paper a novel multimodal supervision-steering transformer for facial expression recognition in the wild, referred to as FER-former. Specifically, to address the limitation of narrow receptive fields, a hybrid feature extraction pipeline is designed by cascading both prevailing CNNs and transformers. To deal with the issue of homogenous supervisory signals, a heterogeneous domain-steering supervision module is proposed to incorporate text-space semantic correlations to enhance image features, based on the similarity between image and text features. Additionally, a FER-specific transformer encoder is introduced to characterize conventional one-hot label-focusing and CLIP-based text-oriented tokens in parallel for final classification. Based on the collaboration of multifarious token heads, global receptive fields with multimodal semantic cues are captured, delivering superb learning capability. Extensive experiments on popular benchmarks demonstrate the superiority of the proposed FER-former over the existing state-of-the-art methods.

Recently, lots of works that incorporate external lexicon information into character-level Chinese named entity recognition(NER) to overcome the lackness of natural delimiters of words, have achieved many advanced performance. However, obtaining and maintaining high-quality lexicons is costly, especially in special domains. In addition, the entity boundary bias caused by high mention coverage in some boundary characters poses a significant challenge to the generalization of NER models but receives little attention in the existing literature. To address these issues, we propose SENCR, a Span Enhanced Two-stage Network with Counterfactual Rethinking for Chinese NER, that contains a boundary detector for boundary supervision, a convolution-based type classifier for better span representation and a counterfactual rethinking(CR) strategy for debiased boundary detection in inference. The proposed boundary detector and type classifier are jointly trained with the same contextual encoder and then the trained boundary detector is debiased by our proposed CR strategy without modifying any model parameters in the inference stage. Extensive experiments on four Chinese NER datasets show the effectiveness of our proposed approach.

A key challenge in sensor-based fall prediction is the fact that a fall event can often occur in various configurations of fall poses together with their own spatio-temporal dependencies. This leads us to define a spatio-temporal model to explicitly characterize these internal configurations of poses. In particular, we introduce a graph neural network with spatio-temporal topological structure to encode such latent relations among poses by capturing representative patterns in fall events. Moreover, a human body orientation estimator is devised to capture human low limbs information, and as a result, separate pose dependencies are globally consistent. Empirical evaluations on two benchmark datasets and one in-house dataset suggest our approach significantly outperforms the state-of-the-art methods.

Predicting human falls from wearable devices is a complex task due to the inherent diversity and causality of multivariate physical changes, where each instance exhibits a unique style of motion events and their spatio-temporal causal dependencies. Consequently, we propose a multichannel causal model that utilizes the Granger causality test to explicitly delineate these internal configurations of motion events and their causal relationships from a spatio-temporal perspective. Particularly, our model incorporates a multi-head attention mechanism with a distillation component to capture the spatio-temporal dependencies among multiple channels of motion sensors in an end-to-end fashion. Empirical evaluations conducted on two benchmark datasets, as well as one in-house dataset collected by ourselves, indicate that our model significantly surpasses state-of-the-art approaches.

In database of recommender systems, users' ratings for most items are usually missing, resulting in selection bias when users selectively choose items to rate. To address this problem, propensity-based methods, e.g., inverse propensity scoring and doubly robust, have been widely studied and applied to missing rating prediction and post-click conversion rate prediction tasks. However, have we completely eliminated the selection bias? Under what missing data mechanism can previous studies completely eliminate the selection bias and lead to unbiased learning? In this paper, following the previous literature on statistics, we first formally define three missing data mechanisms, i.e., missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR), and discuss the widespread prevalence of MNAR in recommender systems. Next, we theoretically reveal that the unbiasedness of previous propensity-based debiasing methods is valid only when data are MCAR or MAR, while it leads to biased predictions when data are MNAR. To tackle this research gap, we propose to disentangle user and item embeddings into the primary latent vector for rating prediction and the auxiliary latent vector for missing mechanism modeling. We prove the identifiablility results, and show that the proposed method can achieve unbiased learning under MNAR with imposed constraints. Extensive experiments are conducted on a semi-synthetic dataset and three real-world datasets, validating the effectiveness of our proposed method.

Cognitive load assessment task faces a significant challenge regarding the neglect of rich spatio-temporal dependencies and causal dependencies in multi-channel physiological signals. To this end, we present a multi-channel spatio-temporal causal representations model that explicitly characterize the inherent causal structural variability and spatio-temporal dependencies within a single channel and interrelationships among multiple channels. Particularly, a causal structure is constructed by optimizing a score-based causal function under the constraint of causal Markov property. It can effectively disentangle the latent spatio-temporal feature variables into two groups: causal representation and task-irrelevant representation. Empirical evaluations on two public datasets and one in-house dataset suggest our model significantly outperforms the state-of-the-art methods.

The primary challenge in cognitive load recognition is the inherent diversity and causality of multivariate physiological changes, as each instance exhibits a distinctive configuration of physiological events and their spatio-temporal causal dependencies. This leads us to define a causal graph designed by prior knowledge about cognitive load to identify the latent factors hidden in the multi-instance bags constructed by the observed instances of multiple physiological channels. In particular, our model introduces the multi-instance causal representation to explicitly disentangle the unique causal configurations of a particular cognitive load state as a variable number of temporal causal variables and spurious causal variables. In addition, GADF maps are constructed to capture the inherent spatio-temporal dependency among multivariate signals in a 2D structural space. A domain adapter is employed to reduce domain bias by effectively transferring the train domain to the test domain in such continuous latent space. Empirical evaluations on two benchmark datasets and two in-house datasets collected by ourselves suggest our model significantly outperforms the state- of-the-art approaches.

Artificial intelligence generated content (AIGC) has emerged as an indispensable tool for producing large-scale content in various forms, such as images, thanks to the significant role that AI plays in imitation and production. However, interpretability and controllability remain challenges. Existing AI methods often face challenges in producing images that are both flexible and controllable while considering causal relationships within the images. To address this issue, we have developed a novel method for causal controllable image generation (CCIG) that combines causal representation learning with bi-directional generative adversarial networks (GANs). This approach enables humans to control image attributes while considering the rationality and interpretability of the generated images and also allows for the generation of counterfactual images. The key of our approach, CCIG, lies in the use of a causal structure learning module to learn the causal relationships between image attributes and joint optimization with the encoder, generator, and joint discriminator in the image generation module. By doing so, we can learn causal representations in image’s latent space and use causal intervention operations to control image generation. We conduct extensive experiments on a real-world dataset, CelebA. The experimental results illustrate the effectiveness of CCIG.

Falls are the leading cause of unintentional human injury , having become a public health event of strong social concern. The fall prediction technology based on wearable inertial sensors is a relatively reliable solution in human activity monitoring, a user scenario with mobility and high information privacy sensitivity, and has the advantages of low cost, small size, and high precision. However, a key challenge in sensor-based fall prediction is the fact that a fall event can often occur in various configurations of fall poses together with their own spatio-temporal dependencies. This leads us to define a spatio-temporal model to explicitly characterize these internal configurations of poses. In particular, we introduce a graph neural network with spatio-temporal topological structure to encode such latent relations among poses by capturing representative patterns in fall events. Moreover, a human body orientation estimator is devised to represent human low limbs information and as a result, separate pose dependencies are globally consistent. Empirical evaluations on two benchmark datasets and one in-house dataset suggest our approach significantly outperforms the state-of-the-art methods.

Sensors are the key to environmental monitoring, which impart benefits to smart cities in many aspects, such as providing real-time air quality information to assist human decision-making. However, it is impractical to deploy massive sensors due to the expensive costs, resulting in sparse data collection. Therefore, how to get fine-grained data measurement has long been a pressing issue. In this article, we aim to infer values at nonsensor locations based on observations from available sensors (termed spatiotemporal inference), where capturing spatiotemporal relationships among the data plays a critical role. Our investigations reveal two significant insights that have not been explored by previous works. First, data exhibit distinct patterns at both long- and short-term temporal scales, which should be analyzed separately. Second, short-term patterns contain more delicate relations, including those across spatial and temporal dimensions simultaneously, while long-term patterns involve high-level temporal trends. Based on these observations, we propose to decouple the modeling of short- and long-term patterns. Specifically, we introduce a joint spatiotemporal graph attention network to learn the relations across space and time for short-term patterns. Furthermore, we propose a graph recurrent network with a time skip strategy to alleviate the gradient vanishing problem and model the long-term dependencies. Experimental results on four public real-world datasets demonstrate that our method effectively captures both long- and short-term relations, achieving state-of-the-art performance against existing methods.

Deep generative models are effective in style transfer. Previous methods learn one or several specific artist-style from a collection of artworks. These methods not only homogenize the artist-style of different artworks of the same artist but also lack generalization for the unseen artists. To solve these challenges, we propose a double-style transferring module (DSTM). It extracts different artist-style and artwork-style from different artworks (even untrained) and preserves the intrinsic diversity between different artworks of the same artist. DSTM swaps the two styles in the adversarial training and encourages realistic image generation given arbitrary style combinations. However, learning style from single artwork can often cause over-adaption to it, resulting in the introduction of structural features of style image. We further propose an edge enhancing module (EEM) which derives edge information from multi-scale and multi-level features to enhance structural consistency. We broadly evaluate our method across six large-scale benchmark datasets. Empirical results show that our method achieves arbitrary artist-style and artwork-style extraction from a single artwork, and effectively avoids introducing the style image’s structural features. Our method improves the state-of-the-art deception rate from 58.9% to 67.2% and the average FID from 48.74 to 42.83.

Multimedia recommendation involves personalized ranking tasks, where multimedia content is usually represented using a generic encoder. However, these generic representations introduce spurious correlations that fail to reveal users' true preferences. Existing works attempt to alleviate this problem by learning invariant representations, but overlook the balance between independent and identically distributed (IID) and out-of-distribution (OOD) generalization. In this paper, we propose a framework called Pareto Invariant Representation Learning (PaInvRL) to mitigate the impact of spurious correlations from an IID-OOD multi-objective optimization perspective, by learning invariant representations (intrinsic factors that attract user attention) and variant representations (other factors) simultaneously. Specifically, PaInvRL includes three iteratively executed modules: (i) heterogeneous identification module, which identifies the heterogeneous environments to reflect distributional shifts for user-item interactions; (ii) invariant mask generation module, which learns invariant masks based on the Pareto-optimal solutions that minimize the adaptive weighted Invariant Risk Minimization (IRM) and Empirical Risk (ERM) losses; (iii) convert module, which generates both variant representations and item-invariant representations for training a multi-modal recommendation model that mitigates spurious correlations and balances the generalization performance within and cross the environmental distributions. We compare the proposed PaInvRL with state-of-the-art recommendation models on three public multimedia recommendation datasets (Movielens, Tiktok, and Kwai), and the experimental results validate the effectiveness of PaInvRL for both within-and cross-environmental learning.

The primary goal of image inpainting is to fix holes in a damaged image with natural contents. A key challenge is that a damaged image contains complex structures in differ-ent ways, with each consisting of its configuration of edges and spatial dependencies. As a result, filled images often converge to unnatural and implausible results. Currently, the edge-image inpainting methods adopt two stages to recover edges and images successively, which suffer from feature in-consistency and error accumulation. This leads us to present a parallel edge-image learning framework that explicitly char-acterizes these internal configurations in a single stage. The framework introduces a dual parallel network-based decoder to generate the image and the edges concurrently, leading to feature consistency at the semantic level. Also, a new cross-fire mechanism aims to exchange edge-image information in the decoder, avoiding error accumulation. Empirical evaluations on benchmark datasets suggest that our approach out-performs the state-of-the-art methods on image inpainting.

与线性因果相比, 非线性因果具有更复杂的特点和内涵. 本文主要讨论非线性因果中的若干个问题, 并着重强调因果域的概念. 本文基于广泛应用的计算模型和方法, 围绕非线性因果分析与计算以及因果域的识别问题提出相应观点和建议, 并通过几个具体案例揭示非线性因果在处理复杂因果推断问题中的重要性和现实意义.

High-performance data streaming technologies are increasingly adopted in IT companies to support the integration of heterogeneous and possibly distributed applications. Compared with the traditional message queuing middleware, a streaming platform enables the implementation of event-streaming systems (ESS) which include not only complex queues but also pipelines that transform and react to the streams of data. By analysing the centralised data streams, one can evaluate the Quality-of-Service for other systems and components that produce or consume those streams. We consider the exploitation ofprobabilistic model checkingas a performance monitoring technique for ESS systems. Probabilistic model checking is a mature, powerful verification technique with successful application in performance analysis. However, an ESS system may contain quantitative parameters that are determined by event streams observed in a certain period of time. In this paper, we present a novel theoretical framework called QV4M (meaning “quantitative verification for monitoring”) for monitoring ESS systems, which is based on two recent methods of probabilistic model checking. QV4M assumes the parameters in a probabilistic system model as random variables and infers the statistical significance for the probabilistic model checking output. We also present an empirical evaluation of computational time and data cost for QV4M.

A key challenge of oversampling in medical imbalanced data classification is that the generation of new minority samples often neglects rich causal dependencies among features, with each being responsible for disease diagnosis. This leads us to define a constraint-based approach that generates new samples by explicitly discovering and leveraging the inherent local causal variability of features under a global view. Our approach employs causal Markov property to construct a causal network that explicitly characterizes these unique causal configurations of a particular disease as a variable number of nodes and links. By perturbing those learned causal features from majority class, we synthesize new samples in the territory of minority space. An additional sample selection estimator is introduced to choose the most representative samples. Empirical evaluations on four medical datasets suggest our approach significantly outperforms the state-of-the-art methods.

Hand gesture recognition can be exploited to benefit ubiquitous applications using sensors. Currently, the inherent complexity of human physical activities makes it difficult to accurately recognize gestures with wearable sensors, especially in real time. To this end, a real-time hand gesture recognition system is presented in this paper. In particular, sliding window technology and y-axis threshold are used to detect intended gestures from a continuous data stream and then the segmented data are classified by applying a stacked Long Short-Term Memory (LSTM) model. After noise is removed, six-axis sensor data from wrist-worn devices are fed into the model without requiring feature engineering. We use twelve common hand gestures to evaluate the performance of our model. The experimental results demonstrate the feasibility of our proposed system with an accuracy of 99.8% on average. Our approach allows for an accurate and nonindividual hand gesture recognition. It holds potential to be integrated into a smart watch or other wearable devices for intuitive human computer interaction.

The primary goal of skeletal motion prediction is to generate future motion by observing a sequence of 3D skeletons. A key challenge in motion prediction is the fact that a motion can often be performed in several different ways, with each consisting of its own configuration of poses and their spatio-temporal dependencies, and as a result, the predicted poses often converge to the motionless poses or non-human like motions in long-term prediction. This leads us to define a hierarchical recurrent network model that explicitly characterizes these internal configurations of poses and their local and global spatio-temporal dependencies. The model introduces a latent vector variable from the Lie algebra to represent spatial and temporal relations simultaneously. Furthermore, a structured stack LSTM-based decoder is devised to decode the predicted poses with a new loss function defined to estimate the quantized weight of each body part in a pose. Empirical evaluations on benchmark datasets suggest our approach significantly outperforms the state-of-the-art methods on both short-term and long-term motion prediction.

A key challenge in cancer risk prediction is selecting representative features, with each being responsible for cancer diagnosis. This leads us to define a constraint-based approach that employs causal Markov property to discover local causal dependencies between features and cancer risk types. Our approach introduces a causal network generated from an identified network skeleton to explicitly characterize these unique causal configurations of a particular cancer risk as a variable number of nodes and links. It can be analytically shown that the resulting causal network satisfies the causal Markov property, and as a result, all local cause-effect dependencies can be retained and are globally consistent. An additional node selection estimator is introduced to choose the most representative features. Empirical evaluations on four cancer risk datasets suggest our approach significantly outperforms the state-of-the-art methods.

In this paper, we propose RCapsNet, a recurrent capsule network for text classification. Although a variety of neural networks have been proposed recently, existing models are mainly based either on RNN or on CNN, which are rather limited in encoding temporal features in these network structures. In addition, most of these models require to integrate prior linguistic knowledge into them, which is not practical for a non-linguistician to handcraft such knowledge. To address these issues on temporal relational variabilities in text classification, the RCapsNet is presented by employing a hierarchy of recurrent structure-based capsules. It consists of two components: the recurrent module considered as the backbone of the RCapsNet and the reconstruction module designed to enhance the generalization capability of the model. Empirical evaluations on four benchmark datasets demonstrate the competitiveness of the RCapsNet. In particular, it is shown that prior linguistic knowledge is dispensable for the training of our model.

With the rapid development of biomedical technology, discovering causality from genes and human physiological and pathological characteristics has become a hot but challenge spot over the past decades. Due to the increment of the amount of biomedical data, discovering causality from observed data becomes more and more difficult to search this large body of knowledge in a meaningful manner. To address the issues in existing causality discovering models, we introduce a generative Bayesian causal network that combines neural network to explicitly characterize these unique causal-effect relationships as a variable number of nodes and links. Particularly, a basic skeleton is generated for node selection to reduce the network size by minimizing the maximum mean discrepancy among variables. In addition, a causal generative neural network model is presented to construct causal network with cause-effect scores between variables. Empirical evaluations on two publicly available biomedical datasets and four synthetic datasets suggest our approach significantly outperforms the state-of-the-art methods in discovering causal relationships among biomedical variables.

Gesture recognition is ongoing attention in the field of human computer interaction (HCI). With development of deep neural network technology in computer vision, more complex sign languages are possible to recognize but, the research on Chinese language (CSL) recognition remain in discussion. Here we have performed our collected dataset and proposes a new solution to recognize CSL, and further insight on preliminary verification on CSL recognition using 2D image.This paper attempts to reduce the adverse impact of dataset itself on the image recognition network using continuously improved technical method. Present study addresses the following:1) Due to the lack of the CSL image dataset, we made a CSL dataset and used it in the following experiments to verify the usability of the dataset. 2) Using a self-made dataset, we combined the method of hand skeletal gesture recognition to reduce the impact of the gesture overlap and improve recognition accuracy. Finally, a network model was trained and tested on self-made dataset which include some overlapping gestures that are difficult to recognize and achieved the accuracy rate of 0.9324. 3) Put forward the idea of continuing the experiment to improve dataset and using fuzzy semantic recognition for trying to solve the time-domain problem of dynamic sign language recognition which needs linguistic studies.

Hand gesture recognition with wearables typically focuses on the characteristics of a single point on hand, but ignores the diversity of motion information over hand skeleton. As a result, current methods suffer from two key challenges to manage multiple hand joints: displacement detection and motion representation. This leads us to define a spatio-temporal framework, named STGauntlet, that explicitly characterizes the hand motion context of spatio-temporal relations among multiple joints and detects hand gestures in real-time. The framework introduces the Lie algebra to capture the inherent structural varieties of hand motions with spatio-temporal dependencies among multiple joints. In addition, we developed a hand-worn prototype with multiple motion sensors respectively attached to various joints on hand and collected 7000 samples of seven gestures from nine subjects. Our in-lab study shows that STGauntlet is capable of detecting gesture types together with their 3D tracking trajectory with 97.35% and 95.17% accuracies for subject dependent and independent recognition, respectively.

Military sign language is an important form of tactical communication, especially in restrict situations where either distance or a requirement for silence precludes oral means. Unfortunately, when soldiers cannot see each other, the communication mode of tactical gestures is no longer effective, which may hinder military operations. Vision-based approaches have been at the forefront in the field of hand gesture recognition. However, there still lacks of specific datasets and models for the task of military sign language recognition. In this paper, we collected a new first-person dataset named MSL, which contains 16 classes of 3, 840 tactical gesture samples on battle scenario with more than 11, 0000 video frames performed by 10 subjects. Moreover, we present a novel deep network, called ST-Xception architecture, in light of the depthwise separable convolutions to recognize such military sign language. By expanding the convolution filters and pooling kernels into 3D, our network can characterize the inherent spatio-temporal relationship of a certain tactical hand gesture. In particular, we further reduce computational cost and relieve overfitting by replacing the fully connected layers with adaptive average pooling. Experimental results show that our model outperforms existing models both on our in-house MSL dataset and two other benchmark datasets.

Partial multi-label learning (PML) deals with the problem where each training example is assigned multiple candidate labels, only a part of which are correct. To learn from such PML examples, the straightforward model training tends to be misled by the noise candidate label set. To alleviate this problem, a coupled framework is established in this paper to learn the desired model and perform the relabeling procedure alternatively. In the relabeling procedure, instead of simply extracting relative label confidences, or deterministically eliminating low confidence labels and preserving high confidence labels as ground-truth ones, we introduce a soft sign thresholding operator to adaptively strengthen candidate labels with high confidence and weaken candidate labels with low confidence, which enlarges the difference of confidences of candidate labels within allowable range. We further show that the resulting nonconvex quadratic programming (QP) optimization problem can be relaxed into a convex QP problem with proper conditions. Extensive experiments on synthesized and real-world data sets demonstrate the effectiveness of our proposed approach.

Human-computer interaction introduces critical open door with the proceeds with improvement of wearable gadgets. Gesture recognition through smart devices is becoming a popular research direction. This paper proposes a hand gesture recognition and real-time game control system that is capable of continues human-computer interaction in view of an off-the-rack business wearable wristband. We utilize three-axis accelerator and gyroscope sensors embedded in smart band to collect hand motion information and use Kinect camera capture video information for manual segmentation during the training model phase. A continuous gesture segmentation algorithm based on sliding window and DTW algorithm is developed to detect meaningful gestures in the real-time game control stage. In addition, an android game named Fly Birds which is controlled by gesture recognition result is presented to simulate real-time human-computer interaction. Then, we classify the data in the window using common classifiers. Finally, our experimental results show that, we can accurately identify the designed gestures during the stage of static gesture recognition, and we also achieve a perfect interactive effect in the process of dynamic real-time game control. The experiment outcomes will advance the ascent of human-PC cooperation in view of hand gesture recognition and related applications will rise in vast numbers.

Wearable device is becoming more and more popular, the emergence of wearable equipment, is widely used in daily life, medical, industrial and scientific research and so on. But most of the wearable equipment still needs people’s actual operation, which will bring a lot of inconvenience. For example , when people’s hands are not available, they can’t be operated. Now, many sensors are embedded in off-the-shelf smartwatch, which make it possible to detect different finger gestures by using these sensors. Past research mostly concentrates on identifying gross hand gestures, which usually need the help of other additional hardware that is expensive and discomfort. In this work we utilize muscle activity data getting from integrated motion sensors (accelerometer, linear accelerometer and gyroscope) nested in smartwatch to recognize tiny finger gestures. We extract features within a 1 second sliding window. We test classification effect of different classifiers and different sensors. In the result, we can recognize 5 unremarkable finger gestures well and identify specific finger gestures with a high accuracy. The result can be applied on current popular smartwatches interaction. Also our result will lay the foundation for related research in finger gestures recognition field.

Automatically identifying Chinese characters that are similar in their glyph, pronunciations and meaning are important for building smart question generation tools in a computer-assisted language-learning environment. Previous research on the Chinese character similarity measurement focused on char acter glyph (e.g. structures, strokes and radicals) with heuristic algorithms whose parameter have preset values. This article presents a machine learning (regression) approach to measure the similarity between two Chinese characters, based on the information which not only includes the glyph, but also pronunciation (pinyin) and semantic meaning derived from HowNet. We evaluated various regression models using a testing set consisting of 2586 pairs of characters selected from elementary Chinese textbooks used. The study results showed that four regression models (M5, Support Vector Machine, Gaussian Process and Linear Regression) have similar results (0.617⩽Mean Absolute Error⩽0.641, 0.772⩽Root Mean Square Error⩽0.790). In addition, the study implied that the performance of the regression model could be influenced by the character frequency. Moreover, we evaluated the regression model in a well-known Chinese language learning resource, called 100 pairs of the most confusing Chinese characters. The experiment results indicated that this approach has potential in the recognition and generation of confusing Chinese character pairs.

As the level of hospital informatization raises, it is possible to obtain huge amount of physiological data from bedside monitor and other medical instruments. The goal for this work is to recognize diseases from physiological data by unique combinations of representative patterns for different dis eases. The representative patterns are clustered from the original physiological time series data, e.g. pulse, respiration rate, blood pressure, heart rate and oxygen saturation rate. Within a disease, to compose the set of representative patterns into a interrelated structure, we bring in Allen’s interval relations to describe the temporal relations between each of two neighboring patterns. We use Chinese Restaurant Process (CRP) to draw the uncertainty of every temporal relations that links two representative patterns. The two algorithms are combined into the model we use in this work, called probabilistic model. The experimental results suggests our model has potential in recognizing diseases.

Because of the increased lifespan, there is an immense increase in the demand of healthcare services for senior wellness. In this study, we proposed a system based on biological data, such as body temperature, heart rate and blood pressure, and activity data of the elderly living in a stable enviro nment, such as nursing home, to determine their wellness conditions. The Radio Frequency Identification (RFID) is used to monitor and record real-time location information of the elderly. A novel framework integrating the daily activity data and the biological data for determining the wellness status of an elderly has been modeled by using support vector machine (SVM). In this study, the established model was evaluated on 5 elderly people living in the geriatrics department at the Third People’s Hospital of Lanzhou. The experimental results showed that with effective monitoring and alarm systems, the adverse effects on wellness conditions of elderly people living in a nursing home could be ameliorated to some extent, and the healthcare services for the elderly could be improved.

It is essential to constantly collect data with various mobile applications from diverse sources, such as smartphones and ubiquitous sensors.However, how do you conduct the analysis on such a mass of mobile data or mobile web data aiming to solve issues in different areas of applications, including human behavior recognition, medication, recommendation and transportation?Nowadays, research in mobile and social computing environments is now turning to novel concepts to address the challenge of data processing and analyzing.The special issue Mobile web data analytics addresses issues of data management in mobile and social computing environments with a special focus on data processing and applications.The goal of the special issue is to build a forum for researchers from academy and industry to investigate challenging and innovative research issues on the subject, which combines data analytics within mobile and social environment and to explore creative concepts, theories, innovative technologies and intelligent solutions.We intend this special issue to act as an initial place where people from different areas can find a forum to discuss issues of data management and processing in new and emerging mobile computing environments.We accepted 11 papers that provide deep research results to report the advance of mobile web data analytics and applications.These papers are grouped into

It is essential to constantly collect data with various mobile applications from diverse sources, such as smartphones and ubiquitous sensors.However, how do you conduct the analysis on such a mass of mobile data or mobile web data aiming to solve issues in different areas of applications, including human behavior recognition, medication, recommendation and transportation?Nowadays, research in mobile and social computing environments is turning to novel concepts to address the challenge of data processing and analyzing.This special issue Mobile web data analytics addresses issues of data management in mobile and social computing environments with a special focus on data processing and applications.The goal of this special issue is to build a forum for researchers from academia and industry to investigate challenging and innovative research issues on the subject, which combines data analytics within mobile and social environment and to explore creative concepts, theories, innovative technologies and intelligent solutions.We intend this special issue to act as an initial place where people from different areas can find a forum to discuss issues of data management and processing in new and emerging mobile computing environments.We accepted 11 papers that provide deep research results to report the advance of mobile web data analytics and applications.These papers are grouped into

Social networks contain a wealth of useful information. In this paper, we study a challenging task for integrating users' information from multiple heterogeneous social networks to gain a comprehensive understanding of users' interests and behaviors. Although much effort has been dedicated to study this problem, most existing approaches adopt linear or shallow models to fuse information from multiple sources. Such approaches cannot properly capture the complex nature of and relationships among different social networks. Adopting deep learning approaches to learning a joint representation can better capture the complexity, but this neglects measuring the level of confidence in each source and the consistency among different sources. In this paper, we present a framework for multiple social network learning, whose core is a novel model that fuses social networks using deep learning with source confidence and consistency regularization. To evaluate the model, we apply it to predict individuals' tendency to volunteerism. With extensive experimental evaluations, we demonstrate the effectiveness of our model, which outperforms several state-of-the-art approaches in terms of precision, recall and F1-score.

A key challenge in complex activity recognition is the fact that a complex activity can often be performed in several different ways, with each consisting of its own configuration of atomic actions and their temporal dependencies. This leads us to define an atomic activity-based probabilistic framework that employs Allen's interval relations to represent local temporal dependencies. The framework introduces a latent variable from the Chinese Restaurant Process to explicitly characterize these unique internal configurations of a particular complex activity as a variable number of tables.It can be analytically shown that the resulting interval network satisfies the transitivity property, and as a result, all local temporal dependencies can be retained and are globally consistent.Empirical evaluations on benchmark datasets suggest our approach significantly outperforms the state-of-the-art methods.

Grey system theory has been widely used to forecast the economic data that are often nonlinear, irregular, and nonstationary. Current forecasting models based on grey system theory could adapt to various economic time series data. However, these models ignored the importance of the model parameter optimization and the use of recent data, which lead to poor forecasting accuracy. In this article, we propose a novel forecasting model, called particle swarm optimization rolling grey model (PSO‐RGM(1,1)), based on a rolling mechanism GM with optimized parameters by using the particle swarm optimization algorithm. The simple model is shown to be very effective in forecasting the tertiary industry data sequences, which are short and noisy but regular in secular trend. The experimental results show that PSO‐RGM(1,1) outperforms other commonly used forecasting models on three real economic data sets. Our empirical study shows that PSO is found to be the best overall algorithm to optimize the parameter of RGM compared with other well‐known metaheuristics. Furthermore, we evaluated other variant PSOs and found that single particle PSO outperforms others overall in terms of prediction accuracy, convergence speed, and degree of certainty.

Existing topic models for mining sentiments from articles often ignores the fact that intra-topic correlations are common and useful to uncover a large number of fine-grained and tightly-coherent topics. This paper is concerned with the problem of social sentiment mining by modeling topic correlations. We aim to not only discover the connections between sentiments and topics, but also reveal the deeper relationship among topics where some topics may co-occur more frequently than others in articles. More specifically, we join sentiment mining with hierarchical pachinko allocation model to represent topic correlations by a hierarchy. In our model, the hierarchical pachinko allocation is employed to generate the latent hierarchical topic variables and sentiment variables. Experimental results on a collected news corpus show that our model can effectively identify latent topics in a hierarchical structure, and outperforms competing sentiment-topic models such as Latent Dirichlet Allocation based model in sentiment prediction.

In today’s race walking competition, the determination of whether an athlete fouls is mainly affected by a referee’s subjective judgment, leading to a high possibility of misjudgment. The purpose of this work is to determine whether race walking can be automatically recognized by accelerometers embedded in smartphones. In this work, acceleration data are collected by a smartphone app developed by ourselves. Nineteen features are extracted from the raw sensor data, and are used by an unsupervised classification method for activity recognition, named MCODE. We evaluate various data sampling rates and window lengths during feature extraction in the experiments. We also compare our method with other well-known methods on the metrics such as sensitivity, specificity and adjusted rank index. The results show that our method is viable to recognize race walking using smartphone accelerometers.

The Ubiquitous Awareness and Intelligent Solutions Lab (UAIS) was established in January 2009, at the School of Information Science and Engineering, Lanzhou University, China. It mainly focuses on research in pervasive computing, affective learning, the semantic web and CSCW. The lab encourages multidisciplinary cooperation among its researchers who have backgrounds in computer science, medicine, mathematics and signal processing. UAIS has undertaken research projects sponsored by the National Science Foundation China, European Framework Programme (FP7) and "985" and "211" projects from Lanzhou University. UAIS has also established strong connections with research institutes at home and abroad as well as with companies to facilitate its research and training of students. Copyright 2011 ACM.

The goal of the workshop is to build a forum for researchers from academy and industry to investigate challenging and innovative research issues in the subject, which combines Affective Interaction within ubiquitous environment and to explore creative concepts, theories, innovative technologies and intelligent solutions. Potential participants may come from communities of ubiquitous computing, intelligent computing, brain computer interaction, affective computing, neuroergonomics, cognitive neuroscience etc. in order to present their state-of-the-art progress and visions on the various overlaps across those disciplines. In this proposal, we describe the detailed purpose, topics, and format of this workshop on "Ubiquitous Affective Awareness and Intelligent Interaction".

Classical approaches to implement Grid resource information service (GRIS) are either centralized or hierarchical. It will result in problems of scalability and single point of failure as the scale of Grid systems rapidly increases. To address scalability and reliability of Grid Resource Information Service, we proposed a distributed resource information service model that extends the information service architecture of GT4 based on superpeer model which is one of the P2P techniques. In the results of our simulation experiments, it observed that this model has high scalability and robustness. Future we will implement it in our MICE-G Grid system.

The problem of how to allocate the resources optimally and adaptively in the dynamic, scalable and distribute-controlled grid environment is introduced and discussed numerously. But recent researches can't satisfy with the needs of resource allocation in grid system thoroughly. In this paper, we use the SuperScheduling concept in grid system, which is with no explicit model. And we introduce Ant Colony Optimization applied for those models to implement resource allocation. We test the validity of Ant Colony Optimization which gets good results to satisfy with the needs of resource allocation in the dynamic, scalable grid environment with no global control.

We designed a grid computing model in math based on current network computing technologies. MICE is an emerging technology to provide uniform programming, task submission, and management specifications across the large scale distributed computing nodes which deployed some famous mathematical software. MICE utilizes a three-level architecture that shields users from low-level computing resource discovery and provides globe uniform view for users. We extended MathML to solve the mathematical semantic objects' expression. CSP (computing service platform) servers are adopted in MICE to provide uniform task access, transfer and management of heterogeneous distributed resources across multiple administrative domains. This architecture enables the mathematical software resources to be deployed as services on the Internet. MICE can achieve good scalability, reliability and can be flexibly deployed and configured