Training large language models (LLMs) is fundamentally constrained by limited device memory and costly inter-device communication. Although pipeline parallelism alleviates memory pressure by partitioning models across devices, it incurs activation communication overhead that scales linearly with sequence length, limiting efficiency in long-context training. Recent weight-passing approaches (e.g., WeiPipe) mitigate this by transmitting model weights instead of activations, but suffer from redundant peer-to-peer (P2P) transfers and underutilized intra-node bandwidth. We propose TawPipe—topology-aware weight pipeline parallelism, which exploits hierarchical bandwidth in distributed clusters for improved communication efficiency. TawPipe: (i) groups devices based on topology to optimize intra-node collective and inter-node P2P communication; (ii) assigns each device a fixed shard of model weights and gradients, avoiding redundant transfers; and (iii) overlaps communication with computation to hide latency. Unlike global collective operations used in fully sharded data parallelism (FSDP), TawPipe confines most communication within node boundaries, significantly reducing cross-node traffic. Extensive experiments on up to 24 GPUs with LLaMA‑style models show that TawPipe achieves superior throughput and scalability compared to state-of-the-art baselines.

Precipitation nowcasting, a critical task for weather-sensitive applications, is highly challenging owing to the chaotic nature of atmospheric dynamics. Despite recent progress in deep learning, existing methods are limited in their capacity to model turbulent motions, one of the key drivers of precipitation evolution. Thus, we propose MoCast, the first work that incorporates turbulence knowledge to decompose turbulent motions into solvable components for precipitation nowcasting. Specifically, inspired by the continuity equation, MoCast introduces two core innovations: (1) a physics-guided motion module that learns turbulent motions from physically interpretable mean and fluctuating components based on Reynolds, Helmholtz, and Wavelet decomposition techniques, and (2) a motion-guided source-sink module that learns source-sink features considering the multi-scale impact from motions based on a mixture-of-experts architecture. Extensive experiments on three real-world datasets demonstrate that MoCast achieves the state-of-the-art performance. MoCast and its diffusion-based variant MoCast+ reduce CSI error by an average of 4.9% and 4.5% compared to the best deterministic and probabilistic baselines, respectively.

Accurate air quality forecasting is essential for public health and environmental sustainability, but remains challenging due to the complex pollutant dynamics. Existing deep learning methods often model pollutant dynamics as an instantaneous process, overlooking the intrinsic delays in pollutant propagation. Thus, we propose AirDDE, the first neural delay differential equation framework in this task that integrates delay modeling into a continuous-time pollutant evolution under physical guidance. Specifically, two novel components are introduced: (1) a memory-augmented attention module that retrieves globally and locally historical features, which can adaptively capture delay effects modulated by multifactor data; and (2) a physics-guided delay evolving function, grounded in the diffusion-advection equation, that models diffusion, delayed advection, and source/sink terms, which can capture delay-aware pollutant accumulation patterns with physical plausibility. Extensive experiments on three real-world datasets demonstrate that AirDDE achieves the state-of-the-art forecasting performance with an average MAE reduction of 8.79% over the best baselines.

Existing research on Temporal Knowledge Graph (TKG) representation learning focuses on decomposing events into entities and relations, then employing various approaches to learn entity and relation representations in a low-dimensional vector space. However, all existing research overlooks the correlations among events, even though events are the core constituent elements of TKGs and involve heterogeneous correlations. To this end, we propose a Heterogeneous evolutionary Event cluster Aware Representation learning approach in Temporal Knowledge Graphs (HEART), which is the first event-centric approach in TKGs. Specifically, a heterogeneous event graph construction module is proposed to capture the diverse pairwise correlations between events by building co-entity and proximity heterogeneous edges. In addition, an event-aware multi-step evolutionary clustering module is proposed to capture continuous high-order correlations among events at different timestamps. Furthermore, an event cluster-aware unsupervised alignment mechanism is proposed to preserve the temporal smoothness of event clusters through cross-temporal alignment. Moreover, an event cluster-based self-supervised optimization mechanism is proposed to optimize the representations of event clusters. Experimental results on seven real-world datasets demonstrate that HEART achieves the state-of-the-art performance, outperforming the runner-up by an average of 3.89%, 8.14%, 5.97%, and 6.92% in MRR, Hits@1, Hits@3, and Hits@10, respectively.

Feature interaction modeling, which exploits interactive information between features, has been widely explored in various applications. Recently, many graph or hypergraph structures-based models have been proposed to model feature interactions by predicting the existence of edges or hyperedges among nodes. However, these models lack the capability to capture the inherent comparability among samples, where multiple samples exhibit both shared and distinct characteristics, and such comparable relationships are often beneficial for prediction. To this end, we propose SR-HyperFM, Sample Relationship aware Hypergraph Factorization Machines, which incorporate sample comparable relationships into feature interaction modeling, leveraging both shared features and critical differences among samples. Specifically, the sample relationship aware hypergraph construction module is introduced to fully capture the comparable relationships among samples and discover beneficial high-order feature interactions. In addition, the dual hypergraph message passing module explicitly models feature interactions by exploiting these inherent relationships. Extensive experiments on four real-world datasets demonstrate the superiority of SR-HyperFM. In addition, case studies are conducted to further justify the effectiveness of SR-HyperFM.

In multivariate time series (MTS) forecasting, many deep learning based methods have been proposed for modeling dependencies at multiple spatial (inter-variate) or temporal (intra-variate) scales. However, existing methods may fail to model dependencies across multiple spatial-temporal scales (ST-scales, i.e., scales that jointly consider spatial and temporal scopes). In this work, we propose ST-Hyper to model the high-order dependencies across multiple ST-scales through adaptive hypergraph modeling. Specifically, we introduce a Spatial-Temporal Pyramid Modeling (STPM) module to extract features at multiple ST-scales. Furthermore, we introduce an Adaptive Hypergraph Modeling (AHM) module that learns a sparse hypergraph to capture robust high-order dependencies among features. In addition, we interact with these features through tri-phase hypergraph propagation, which can comprehensively capture multi-scale spatial-temporal dynamics. Experimental results on six real-world MTS datasets demonstrate that ST-Hyper achieves the state-of-the-art performance, outperforming the best baselines with an average MAE reduction of 3.8% and 6.8% for long-term and short-term forecasting, respectively. Code is available at https://anonymous.4open.science/ST-Hyper-83E7.

Multi-variate time series (MTS) forecasting is crucial for various applications. Existing methods have shown promising results owing to their strong ability to capture intra- and inter-variate dependencies. However, these methods often overlook lead-lag dependencies at multiple grouping scales, failing to capture hierarchical lead-lag effects in complex systems. To this end, we propose MillGNN, a novel graph neural network-based method that learns multiple grouping scale lead-lag dependencies for MTS forecasting, which can comprehensively capture lead-lag effects considering variate-wise and group-wise dynamics and decays. Specifically, MillGNN introduces two key innovations: (1) a scale-specific lead-lag graph learning module that integrates cross-correlation coefficients and dynamic decaying features derived from real-time inputs and time lags to learn lead-lag dependencies for each scale, which can model evolving lead-lag dependencies with statistical interpretability and data-driven flexibility; (2) a hierarchical lead-lag message passing module that passes lead-lag messages at multiple grouping scales in a structured way to simultaneously propagate intra- and inter-scale lead-lag effects, which can capture multi-scale lead-lag effects with a balance of comprehensiveness and efficiency. Experimental results on 11 datasets demonstrate the superiority of MillGNN for long-term and short-term MTS forecasting, compared with 16 state-of-the-art methods.

Anomaly detection based on system logs plays an important role in intelligent operations, which is a challenging task due to the extremely complex log patterns. Existing methods detect anomalies by capturing the sequential dependencies in log sequences, which ignore the interactions of subsequences. To this end, we propose CSCLog, a Component Subsequence Correlation-Aware Log anomaly detection method, which not only captures the sequential dependencies in subsequences but also models the implicit correlations of subsequences. Specifically, subsequences are extracted from log sequences based on components, and the sequential dependencies in subsequences are captured by Long Short-Term Memory Networks (LSTMs). An implicit correlation encoder is introduced to model the implicit correlations of subsequences adaptively. In addition, Graph Convolution Networks (GCNs) are employed to accomplish the information interactions of subsequences. Finally, attention mechanisms are exploited to fuse the embeddings of all subsequences. Extensive experiments on four publicly available log anomaly detection datasets demonstrate the effectiveness of CSCLog, outperforming the best baseline by an average of 3.97% in Macro F1-Measure

Predicting user click behavior and making relevant recommendations based on the user’s historical click behavior are critical to simplifying operations and improving user experience. Modeling User Interface (UI) elements is essential to user click behavior prediction, while the complexity and variety of the UI make it difficult to adequately capture the information of different scales. In addition, the lack of relevant datasets also presents difficulties for such studies. In response to these challenges, we construct a fine-grained smartphone usage behavior dataset containing 3 664 325 clicks of 100 users and propose a UI elementSpatialHierarchyAwareSmartphone userClick behaviorPrediction method (SHA-SCP). SHA-SCP builds element groups by clustering the elements according to their spatial positions and uses attention mechanisms to perceive the UI at the element level and the element group level to fully capture the information of different scales. Experiments are conducted on the fine-grained smartphone usage behavior dataset, and the results show that our method outperforms the best baseline by an average of 18.35$\%$, 13.86$\%$, and 11.97$\%$in Top-1 Accuracy, Top-3 Accuracy, and Top-5 Accuracy, respectively.

Multivariate time series (MTS) forecasting has attracted much attention in many intelligent applications. It is not a trivial task, as we need to consider both intra-variable dependencies and inter-variable dependencies. However, existing works are designed for specific scenarios and require much domain knowledge and expert efforts, which is difficult to transfer between different scenarios. In this article, we propose a scale-aware neural architecture search framework for MTS forecasting (SNAS4MTF). A multi-scale decomposition module transforms raw time series into multi-scale sub-series, which can preserve multi-scale temporal patterns. An adaptive graph learning module infers the different inter-variable dependencies under different time scales without any prior knowledge. For MTS forecasting, a search space is designed to capture both intra-variable dependencies and inter-variable dependencies at each time scale. The multi-scale decomposition, adaptive graph learning, and neural architecture search modules are jointly learned in an end-to-end framework. Extensive experiments on two real-world datasets demonstrate that SNAS4MTF achieves a promising performance compared with the state-of-the-art methods.

Temporal Knowledge Graph (TKG) representation learning aims to map temporal evolving entities and relations to embedded representations in a continuous low-dimensional vector space. However, existing approaches cannot capture the temporal evolution of high-order correlations in TKGs. To this end, we propose a **D**eep **E**volutionary **C**lustering jointed temporal knowledge graph **R**epresentation **L**earning approach (**DECRL**). Specifically, a deep evolutionary clustering module is proposed to capture the temporal evolution of high-order correlations among entities. Furthermore, a cluster-aware unsupervised alignment mechanism is introduced to ensure the precise one-to-one alignment of soft overlapping clusters across timestamps, thereby maintaining the temporal smoothness of clusters. In addition, an implicit correlation encoder is introduced to capture latent correlations between any pair of clusters under the guidance of a global graph. Extensive experiments on seven real-world datasets demonstrate that DECRL achieves the state-of-the-art performances, outperforming the best baseline by an average of 9.53\%, 12.98\%, 10.42\%, and 14.68\% in MRR, Hits@1, Hits@3, and Hits@10, respectively.

Although transformer-based methods have achieved great success in multi-scale temporal pattern interaction modeling, two key challenges limit their further development: (1) Individual time points contain less semantic information, and leveraging attention to model pair-wise interactions may cause the information utilization bottleneck. (2) Multiple inherent temporal variations (e.g., rising, falling, and fluctuating) entangled in temporal patterns. To this end, we propose Adaptive Multi-Scale Hypergraph Transformer (Ada-MSHyper) for time series forecasting. Specifically, an adaptive hypergraph learning module is designed to provide foundations for modeling group-wise interactions, then a multi-scale interaction module is introduced to promote more comprehensive pattern interactions at different scales. In addition, a node and hyperedge constraint mechanism is introduced to cluster nodes with similar semantic information and differentiate the temporal variations within each scales. Extensive experiments on 11 real-world datasets demonstrate that Ada-MSHyper achieves state-of-the-art performance, reducing prediction errors by an average of 4.56%, 10.38%, and 4.97% in MSE for long-range, short-range, and ultra-long-range time series forecasting, respectively. Code is available at https://github.com/shangzongjiang/Ada-MSHyper.

Nowadays, different types of context information are integrated into mobile messaging to increase expressiveness and awareness, including mobile device setting, location, activity, and heart rate. Due to low recognition accuracy, sometimes users cannot accurately infer others’ status through activity awareness. Recently, activity recognition technology has advanced. However, the user behaviors of activity awareness with improved technology have not been studied. In this study, we design ActAware, a mobile instant messaging application that integrates activity awareness based on improved activity recognition technology, i.e., improved recognition accuracy and the addition of activity transition notification. We conduct a field study to explore user behaviors and found that activity awareness allows users to speculate on the reasons for chat interruption, plan communication, speculate on whether the chat partner is departing/arriving, and deepen the understanding of living patterns. Compared with disclosing other types of context information, users have fewer privacy concerns about disclosing activity information in ActAware. Based on these findings, we provide design recommendations for mobile messaging to better support activity awareness.

With the increasingly serious air pollution, people are paying more and more attention to air quality. However, air quality information is not available for all regions, as the number of air quality monitoring stations in a city is limited. Existing air quality estimation methods only consider the multisource data of partial regions and separately estimate the air qualities of all regions. In this article, we propose a deep citywide multisource data fusion-based air quality estimation (FAIRY) method. FAIRY considers the citywide multisource data and estimates the air qualities of all regions at a time. Specifically, FAIRY constructs images from the citywide multisource data (i.e., meteorology, traffic, factory air pollutant emission, point of interest, and air quality) and uses SegNet to learn the multiresolution features from these images. The features with the same resolution are fused by the self-attention mechanism to provide multisource feature interactions. To get a complete air quality image with high resolution, FAIRY refines low-resolution fused features by employing high-resolution fused features through residual connections. In addition, the Tobler's first law of geography is used to constrain the air qualities of adjacent regions, which can fully use the air quality relevance of nearby regions. Extensive experimental results demonstrate that FAIRY achieves the state-of-the-art performance on the Hangzhou city dataset, outperforming the best baseline by 15.7% on MAE.

The problem of air pollution threatens public health. Air quality forecasting can provide the air quality index hours or even days later, which can help the public to prevent air pollution in advance. Previous works focus on citywide air quality forecasting and cannot solve nationwide city forecasting problems, whose difficulties lie in capturing the latent dependencies between geographically distant but highly correlated cities. In this article, we propose the group-aware graph neural network (GAGNN), a hierarchical model for nationwide city air quality forecasting. The model constructs a city graph and a city group graph to model the spatial and latent dependencies between cities, respectively. GAGNN introduces a differentiable grouping network to discover the latent dependencies among cities and generate city groups. Based on the generated city groups, a group correlation encoding module is introduced to learn the correlations between them, which can effectively capture the dependencies between city groups. After the graph construction, GAGNN implements message passing mechanism to model the dependencies between cities and city groups. The evaluation experiments on two real-world nationwide city air quality datasets, including the China dataset and the US dataset, indicate that our GAGNN outperforms existing forecasting models.

Temporal Knowledge Graph (TKG) representation learning embeds entities and event types into a continuous low-dimensional vector space by integrating the temporal information, which is essential for downstream tasks, e.g., event prediction and question answering. Existing methods stack multiple graph convolution layers to model the influence of distant entities, leading to the over-smoothing problem. To alleviate the problem, recent studies infuse reinforcement learning to obtain paths that contribute to modeling the influence of distant entities. However, due to the limited number of hops, these studies fail to capture the correlation between entities that are far apart and even unreachable. To this end, we propose GTRL, an entity Group-aware Temporal knowledge graph Representation Learning method. GTRL is the first work that incorporates the entity group modeling to capture the correlation between entities by stacking only a finite number of layers. Specifically, the entity group mapper is proposed to generate entity groups from entities in a learning way. Based on entity groups, the implicit correlation encoder is introduced to capture implicit correlations between any pairwise entity groups. In addition, the hierarchical GCNs are exploited to accomplish the message aggregation and representation updating on the entity group graph and the entity graph. Finally, GRUs are employed to capture the temporal dependency in TKGs. Extensive experiments on six real-world datasets demonstrate that GTRL achieves the state-of-the-art performances on the event prediction task, outperforming the best baseline by an average of 7.35%, 6.09%, 8.31%, and 11.21% in MRR, Hits@1, Hits@3, and Hits@10, respectively.

Event prediction is a vital and challenging task in temporal knowledge graphs (TKGs), which have played crucial roles in various applications. Recently, many graph neural networks based approaches are proposed to model the graph structure information in TKGs. However, these approaches only construct graphs based on quadruplets and model the pairwise correlation between entities, which fail to capture the high-order correlations among entities. To this end, we propose DHyper, a recurrent Dual Hypergraph neural network for event prediction in TKGs, which simultaneously models the influences of the high-order correlations among both entities and relations. Specifically, a dual hypergraph learning module is proposed to discover the high-order correlations among entities and among relations in a parameterized way. A dual hypergraph message passing network is introduced to perform the information aggregation and representation fusion on the entity hypergraph and the relation hypergraph. Extensive experiments on six real-world datasets demonstrate that DHyper achieves the state-of-the-art performances, outperforming the best baseline by an average of 13.09%, 4.26%, 17.60%, and 18.03% in MRR, Hits@1, Hits@3, and Hits@10, respectively.

In practice, Wearable Human Activity Recognition (WHAR) models usually face performance degradation on the new user due to user variance. Unsupervised domain adaptation (UDA) becomes the natural solution to cross-user WHAR under annotation scarcity. Existing UDA models usually align samples across domains without differentiation, which ignores the difference among samples. In this paper, we propose an unsupervised domain adaptation model with sample weight learning (SWL-Adapt) for cross-user WHAR. SWL-Adapt calculates sample weights according to the classification loss and domain discrimination loss of each sample with a parameterized network. We introduce the meta-optimization based update rule to learn this network end-to-end, which is guided by meta-classification loss on the selected pseudo-labeled target samples. Therefore, this network can fit a weighting function according to the cross-user WHAR task at hand, which is superior to existing sample differentiation rules fixed for special scenarios. Extensive experiments on three public WHAR datasets demonstrate that SWL-Adapt achieves the state-of-the-art performance on the cross-user WHAR task, outperforming the best baseline by an average of 3.1% and 5.3% in accuracy and macro F1 score, respectively.

Traffic forecasting is an essential problem in urban planning and computing. The complex dynamic spatial-temporal dependencies among traffic objects (e.g., sensors and road segments) have been calling for highly flexible models; unfortunately, sophisticated models may suffer from poor robustness especially in capturing the trend of the time series (1st-order derivatives with time), leading to unrealistic forecasts. To address the challenge of balancing dynamics and robustness, we propose TrendGCN, a new scheme that extends the flexibility of GCNs and the distribution-preserving capacity of generative and adversarial loss for handling sequential data with inherent statistical correlations. On the one hand, our model simultaneously incorporates spatial (node-wise) embeddings and temporal (time-wise) embeddings to account for heterogeneous space-and-time convolutions; on the other hand, it uses GAN structure to systematically evaluate statistical consistencies between the real and the predicted time series in terms of both the temporal trending and the complex spatial-temporal dependencies. Compared with traditional approaches that handle step-wise predictive errors independently, our approach can produce more realistic and robust forecasts. Experiments on six benchmark traffic forecasting datasets and theoretical analysis both demonstrate the superiority and the state-of-the-art performance of TrendGCN. Source code is available at https://github.com/juyongjiang/TrendGCN.

Reconstructing 3D clothed human avatars from single images is a challenging task, especially when encountering complex poses and loose clothing. Current methods exhibit limitations in performance, largely attributable to their dependence on insufficient 2D image features and inconsistent query methods. Owing to this, we present the Global-correlated 3D-decoupling Transformer for clothed Avatar reconstruction (GTA), a novel transformer-based architecture that reconstructs clothed human avatars from monocular images. Our approach leverages transformer architectures by utilizing a Vision Transformer model as an encoder for capturing global-correlated image features. Subsequently, our innovative 3D-decoupling decoder employs cross-attention to decouple tri-plane features, using learnable embeddings as queries for cross-plane generation. To effectively enhance feature fusion with the tri-plane 3D feature and human body prior, we propose a hybrid prior fusion strategy combining spatial and prior-enhanced queries, leveraging the benefits of spatial localization and human body prior knowledge. Comprehensive experiments on CAPE and THuman2.0 datasets illustrate that our method outperforms state-of-the-art approaches in both geometry and texture reconstruction, exhibiting high robustness to challenging poses and loose clothing, and producing higher-resolution textures. Codes are available at https://github.com/River-Zhang/GTA.

Geo-tagged photo-based tourist attraction recommendation can discover users’ travel preferences from their taken photos, so as to recommend suitable tourist attractions to them. However, existing visual content-based methods cannot fully exploit the user and tourist attraction information of photos to extract visual features, and do not differentiate the significance of different photos. In this article, we propose multi-level visual similarity-based personalized tourist attraction recommendation using geo-tagged photos (MEAL). MEAL utilizes the visual contents of photos and interaction behavior data to obtain the final embeddings of users and tourist attractions, which are then used to predict the visit probabilities. Specifically, by crossing the user and tourist attraction information of photos, we define four visual similarity levels and introduce a corresponding quintuplet loss to embed the visual contents of photos. In addition, to capture the significance of different photos, we exploit the self-attention mechanism to obtain the visual representations of users and tourist attractions. We conducted experiments on two datasets crawled from Flickr, and the experimental results proved the advantage of this method.

Multivariate time series (MTS) forecasting plays an important role in the automation and optimization of intelligent applications. It is a challenging task, as we need to consider both complex intra-variable dependencies and inter-variable dependencies. Existing works only learn temporal patterns with the help of single inter-variable dependencies. However, there are multi-scale temporal patterns in many real-world MTS. Single inter-variable dependencies make the model prefer to learn one type of prominent and shared temporal patterns. In this article, we propose a multi-scale adaptive graph neural network (MAGNN) to address the above issue. MAGNN exploits a multi-scale pyramid network to preserve the underlying temporal dependencies at different time scales. Since the inter-variable dependencies may be different under distinct time scales, an adaptive graph learning module is designed to infer the scale-specific inter-variable dependencies without pre-defined priors. Given the multi-scale feature representations and scale-specific inter-variable dependencies, a multi-scale temporal graph neural network is introduced to jointly model intra-variable dependencies and inter-variable dependencies. After that, we develop a scale-wise fusion module to effectively promote the collaboration across different time scales, and automatically capture the importance of contributed temporal patterns. Experiments on six real-world datasets demonstrate that MAGNN outperforms the state-of-the-art methods across various settings.

Unsupervised user adaptation aligns the feature distributions of the data from training users and the new user, so a well-trained wearable human activity recognition (WHAR) model can be well adapted to the new user. With the development of wearable sensors, multiple wearable sensors based WHAR is gaining more and more attention. In order to address the challenge that the transferabilities of different sensors are different, we propose SALIENCE (unsupervised user adaptation model for multiple wearable sensors based human activity recognition) model. It aligns the data of each sensor separately to achieve local alignment, while uniformly aligning the data of all sensors to ensure global alignment. In addition, an attention mechanism is proposed to focus the activity classifier of SALIENCE on the sensors with strong feature discrimination and well distribution alignment. Experiments are conducted on two public WHAR datasets, and the experimental results show that our model can yield a competitive performance.

Outcome prediction aims to predict the future health condition of patients from Electronic Health Record (EHR) data. Because of the sequential characteristic of EHR data, recurrent neural network (RNN)-based outcome prediction methods have achieved state-of-the-art results. However, the major drawback of RNN-based outcome prediction methods is lack of interpretability, which would lead to trust issues. Aiming at this problem, this paper proposes interpretable outcome prediction model with hierarchical attention (IoHAN), an interpretable outcome prediction model by leveraging attention mechanism. The main novelty of IoHAN is that it can pinpoint the fine-grained influence on the final prediction result of each medical component by decomposing the attention weights hierarchically into hospital visits, medical variables, and interactions between medical variables. We evaluated IoHAN on MIMIC-III, a large real-world EHR data set. The experiment results demonstrate that IoHAN can achieve higher prediction accuracy than state-of-the-art outcome prediction models. In addition, the hierarchical decomposed attention weights can interpret the prediction results in a more natural and understandable way.

Time-series forecasting is a key component in the automation and optimization of intelligent applications. It is not a trivial task, as there are various short-term and/or long-term temporal dependencies. Multiscale modeling has been considered as a promising strategy to solve this problem. However, the existing multiscale models either apply an implicit way to model the temporal dependencies or ignore the interrelationships between multiscale subseries. In this article, we propose a multiscale interactive recurrent network (MiRNN) to jointly capture multiscale patterns. MiRNN employs a deep wavelet decomposition network to decompose the raw time series into multiscale subseries. MiRNN introduces three key strategies (truncation, initialization, and message passing) to model the inherent interrelationships between multiscale subseries, as well as a dual-stage attention mechanism to capture multiscale temporal dependencies. Experiments on four real-world datasets demonstrate that our model achieves promising performance compared with the state-of-the-art methods.

When a detection model that has been well-trained on a set of classes faces new classes, incremental learning is always necessary to adapt the model to detect the new classes. In most scenarios, it is required to preserve the learned knowledge of the old classes during incremental learning rather than reusing the training data from the old classes. Since the objects in remote sensing images often appear in various sizes, arbitrary directions, and dense distribution, it further makes incremental learning-based object detection more difficult. In this article, a new architecture for incremental object detection is proposed based on feature pyramid and knowledge distillation. Especially, by means of a feature pyramid network (FPN), the objects with various scales are detected in the different layers of the feature pyramid. Motivated by Learning without Forgetting (LwF), a new branch is expended in the last layer of FPN, and knowledge distillation is applied to the outputs of the old branch to maintain the old learning capability for the old classes. Multitask learning is adopted to jointly optimize the losses from two branches. Experiments on two widely used remote sensing data sets show our promising performance compared with state-of-the-art incremental object detection methods.

Hyperspectral image (HSI) denoising is a crucial preprocessing task to improve the performance of the subsequent HSI interpretation and applications. With recent progress in deep learning, HSI denoising methods based on deep neural networks have attracted increasing interest and achieved the state-of-the-art performance. Nevertheless, most of these methods are based on network structures originally developed for grayscale and color images and require a change of network structure to be applicable to HSIs. The new network architectures often lead to complicated models and limited flexibility, which, in turn, result in difficulty in learning and demand of a large number of training samples. In this article, we propose an innovative two-stage learning method including pretraining and fine-tuning procedures. In the first stage, a denoising convolutional neural network can be pretrained with pairs of corrupted and clean images. In the second stage, the pretrained network is fine-tuned via a self-supervised learning strategy to capture the spectral correlation in HSIs. The training pairs in the second stage are constructed from the neighboring band images in the target noisy HSI, leading to a novel idea of embedding spectral information into denoiser through the target image, rather than the change of the network architecture. This model has strong adaptability such that many image denoising networks can be easily adopted for HSIs, while the external hyperspectral training set is optional but not mandatory. Experimental results show that our method has competitive performance compared with the state-of-the-art approaches.

Traffic flow prediction is a fundamental part of ITS (Intelligent Transportation System). Since the correlations of traffic data are complicated and are affected by various factors, traffic flow prediction is a challenging task. Existing traffic flow prediction methods generally take limited static factors (e.g., the distance between sensors and road network topological structure) into consideration and model the correlations of the traffic data separately to predict the future traffic. In this paper, we propose AARGNN (Attentive Attributed Recurrent Graph Neural Network), a GNN (graph neural network) based method considering multiple dynamic factors to predict short-term traffic flow. With multi-source urban data (e.g., POI, road network, incident, weather, etc.), AARGNN considers both static factors and dynamic factors (e.g., spatial distance, semantic distance, road characteristic, road situation, and global context) to predict the short-term traffic flow. Specifically, AARGNN constructs an attributed graph and encodes various factors into the attributes. The correlations of the traffic data are modeled by utilizing the GNN combined with LSTM (long short-term memory). In addition, AARGNN specifies the contributions of each factor based on attention mechanism. Experiments on real-world datasets show that the proposed method outperforms all baseline methods.

Predicting user positive response (e.g., purchases and clicks) probability is a critical task in Web applications. To identify predictive features from raw data, the state-of-the-art extreme deep factorization machine model (xDeepFM) introduces a new interaction network to leverage feature interactions at the vector-wise level explicitly. However, since each hidden layer in the interaction network is a collection of feature maps, it can be viewed essentially as an ensemble of different feature maps. In this case, only using a single objective to minimize the prediction loss may lead to overfitting and generate correlated errors. In this article, an ensemble diversity enhanced extreme deep factorization machine model (DexDeepFM) is proposed, which designs the ensemble diversity measure in each hidden layer and considers both ensemble diversity and prediction accuracy in the objective function. In addition, the attention mechanism is introduced to discriminate the importance of ensemble diversity measures with different feature interaction orders. Extensive experiments on three public real-world datasets are conducted to show the effectiveness of the proposed model.

Temporal knowledge graph (TKG) representation learning embeds relations and entities into a continuous low-dimensional vector space by incorporating temporal information. Latest studies mainly aim at learning entity representations by modeling entity interactions from the neighbor structure of the graph. However, the interactions of relations from the neighbor structure of the graph are neglected, which are also of significance for learning informative representations. In addition, there still lacks an effective historical relation encoder to model the multi-range temporal dependencies. In this article, we propose a d ual gr a ph c onvolution network based TKG representation learning method using h istorical rel a tions (DACHA). Specifically, we first construct the primal graph according to historical relations, as well as the edge graph by regarding historical relations as nodes. Then, we employ the dual graph convolution network to capture the interactions of both entities and historical relations from the neighbor structure of the graph. In addition, the temporal self-attentive historical relation encoder is proposed to explicitly model both local and global temporal dependencies. Extensive experiments on two event based TKG datasets demonstrate that DACHA achieves the state-of-the-art results.

Trajectory prediction for mobile phone users is a cornerstone component to support many higher-level applications in LBSs (Location-Based Services). Most existing methods are designed based on the assumption that the explicit location information of the trajectories is available (e.g., GPS trajectories). However, collecting such kind of trajectories lays a heavy burden on the mobile phones and incurs privacy concerns. In this paper, we study the problem of trajectory prediction based on cell-id trajectories without explicit location information and propose a deep learning framework (called DeepCTP) to solve this problem. Specifically, we use a multi-graph embedding method to learn the latent spatial correlations between cell towers by exploiting handoff patterns. Then, we design a novel spatial-aware loss function for the encoder-decoder network to generate cell-id trajectory predictions. We conducted extensive experiments on real datasets. The experiment results show that DeepCTP outperforms the state-of-the-art cell-id trajectory prediction methods in terms of prediction error.

Traffic flow prediction plays an important role in ITS (Intelligent Transportation System). This task is challenging due to the complex spatial and temporal correlations (e.g., the constraints of road network and the law of dynamic change with time). Existing work tried to solve this problem by exploiting a variety of spatiotemporal models. However, we observe that more semantic pair-wise correlations among possibly distant roads are also critical for traffic flow prediction. To jointly model the spatial, temporal, semantic correlations with various global features in the road network, this paper proposes T-MGCN (Temporal Multi-Graph Convolutional Network), a deep learning framework for traffic flow prediction. First, we identify several kinds of semantic correlations, and encode the non-Euclidean spatial correlations and heterogeneous semantic correlations among roads into multiple graphs. These correlations are then modeled by a multi-graph convolutional network. Second, a recurrent neural network is utilized to learn dynamic patterns of traffic flow to capture the temporal correlations. Third, a fully connected neural network is utilized to fuse the spatiotemporal correlations with global features. We evaluate T-MGCN on two real-world traffic datasets and observe improvement by approximately 3% to 6% as compared to the state-of-the-art baseline.

Traffic forecasting is of great importance to transportation management and public safety, and very challenging due to the complicated spatial-temporal dependency and essential uncertainty brought about by the road network and traffic conditions. Latest studies mainly focus on modeling the spatial dependency by utilizing graph convolutional networks (GCNs) throughout a fixed weighted graph. However, edges, i.e., the correlations between pair-wise nodes, are much more complicated and interact with each other. In this paper, we propose the Multi-Range Attentive Bicomponent GCN (MRA-BGCN), a novel deep learning model for traffic forecasting. We first build the node-wise graph according to the road network distance and the edge-wise graph according to various edge interaction patterns. Then, we implement the interactions of both nodes and edges using bicomponent graph convolution. The multi-range attention mechanism is introduced to aggregate information in different neighborhood ranges and automatically learn the importance of different ranges. Extensive experiments on two real-world road network traffic datasets, METR-LA and PEMS-BAY, show that our MRA-BGCN achieves the state-of-the-art results.

With the development of technology, mobile and wearable devices can obtain many different kinds of real-time activities from users. In social domains, the sharing of real-time activities is proved to help users to understand the current state of others and promote social relationships. However, most studies focused on real-time simple activities sharing, and the willingness of sharing real-time daily activities has not been fully studied. In this paper, we conduct an in-situ study to explore the willingness of sharing real-time daily activities among friends and study the factors that could influence sharing willingness. We find that daily activity types, friend types, location types, time periods, and participants’ privacy attitude classifications could influence sharing willingness. Our findings provide a basis for real-time communication systems to support the sharing of different types of daily activities. Furthermore, we present some design implications to better support real-time daily activities sharing.

Nowadays, smartphones and wearable devices can obtain health- and fitness-related information for better health management. With the popularity of social network sites (SNSs), sharing such information becomes increasingly ubiquitous, and there are some researches about such sharing behaviors. However, the sharing behaviors of health- and fitness-related information generated by wearable devices on public SNSs have not been fully studied. In this study, the authors collected health- and fitness-related information generated by Mi band on Weibo and analyzed the sharing behaviors of Weibo users. They selected 126 sharers to study the motivations of sharing different types of health- and fitness-related information and explore the factors of influencing motivations. They collected 2322 pieces of such shared information and corresponding responses to study what sharers posted, what audiences responded, and the relationship between the features of sharing and the number of responses. In addition, the authors present some design recommendations for the sharing function of wearable devices applications.

Topic models have achieved big success in recent years. To detect topics in a text stream, various online topic models have been proposed in the literature. The limitations of these works include that: 1) most of them run with fixed topic numbers and 2) the overlaps between the topics may enlarge in the evolving process. Hierarchical topic model is a candidate solution to these problems since it can reveal many useful relationships between the topics. These relationships can help to find high quality topics and reduce topic overlaps. In this paper, a knowledge-based semisupervised hierarchical online topic detection framework is proposed. The proposed framework can detect topics in an online hierarchical way. In addition, it has been proven that introducing external knowledge can improve the performance of text mining. Therefore, the knowledge from external knowledge sources and human experts are also integrated in the proposed framework. Experiments are conducted to evaluate the proposed framework with different metrics. The results show that compared with the baseline methods, our framework can achieve better performance with competitive time efficiency.

Complex activity recognition is a valuable issue in mobile and wearable computing. Since complex activities are strongly relevant to users' locations, location data can be used in complex activity recognition. Current activity recognition methods utilizing location data either do not make the most of location data or have a “cold-start” problem. In this paper, we propose CAROLINA (complex activity recognition using acceleration, vital sign, and location data) to recognize complex activities from wearable sensors. CAROLINA divides a map into grid cells and builds a grid cell-activity matrix by integrating check-in and ATUS (American Time Use Survey) datasets. Since some grid cells do not have check-in records, there are some null entries in the matrix, and all the entries are reconstructed by matrix factorization. Meanwhile, a POI (point of interest) category dataset is exploited to build a grid cell-grid cell similarity matrix, which is used to reduce the loss of matrix factorization. The results of experiments show that CAROLINA can effectively utilize location data to improve complex activity recognition performance.

Summary In the past few years, executing high‐concurrency queries with interactive SQL query engines on Hadoop has become an important activity for many organizations. However, these systems do not adopt Multi‐Query Optimization (MQO) to accelerate the process. There are two major concerns. Firstly, traditional MQO researches assume that multiple queries have high similarity. However, these systems usually serve a variety of applications. Although queries from the same application have high similarity, queries from different applications may have low similarity, so using traditional MQO will be inefficient and time consuming. Secondly, integrating MQO may lead to lots of system modifications. To integrate MQO into interactive SQL query engines on Hadoop efficiently, a query grouping–based MQO framework is proposed. A lightweight mechanism is used to represent SQL queries, on which a grouping method is exploited to speed up the optimization process. A cost model is integrated to estimate the execution cost of interactive SQL query engines on Hadoop. By using the proposed framework, we modify Impala system to support MQO, and the experimental results on TPC‐DS show significant performance improvements.

Summary Impala system is an open source, analytic MPP database for Apache Hadoop. Impala system uses a query execution scheduling scheme that assigns near‐equal bytes retrieval tasks for different hosts to ensure system load balance. However, such “load balance” cannot guarantee a short response time for Impala system, when there are original loads in the system. Traditional query execution scheduling methods require either some assumptions or particular architecture, which cannot be directly used in Impala system. In this paper, we present a query execution scheduling scheme for Impala system. If the query fetches data from a single table, the scheme exploits the maximum flow algorithm. If the query fetches data from multiple tables, the scheme employs a cost‐based algorithm with heuristic pruning rules. In addition, we propose a cost model for Impala system, which considers parallel execution, communication cost, and cluster load. The performance of the proposed scheme is evaluated by the TPC‐DS benchmark, and experimental results show that the scheme can reduce the query response time by 10%‐30%.

Rare category detection aims to find interesting and statistically significant anomalies and incorporates ideas from active learning and semisupervised learning. The challenge of rare category detection is to find the rare classes of the anomalies in a data set where the data distribution is skewed. Most existing rare category detection methods suppose that the user knows the specific number of all classes in advance, which cannot be satisfied in most real scenarios. In this paper, we propose a new rare category detection framework composed of active learning and semisupervised hierarchical density-based clustering. The advantage of our method is that it is prior free and can benefit the rare category detecting process with the labeled data. In addition, the proposed framework can handle tasks with nonlinear mappings, which increases the ability to find rare classes when the class boundary is sophisticated. Compared to existing methods, better results are achieved by our method on both real and synthetic data sets in the experiment.

The sparse problem of traffic volume data is unavoidable due to budget limits and device malfunctions in traffic systems. To address this problem, we propose a license plate recognition (LPR) data and collaborative tensor decomposition (CTD)-based method to estimate the sparse traffic volume data. The method works in two phases: first, a vehicle-time matrix is created based on LPR data, and non-negative matrix factorization is employed to analyze vehicle types; second, a road traffic volume tensor and the corresponding matrix of vehicle types are created, and people’s check-in data and point of interest information are introduced to complement the sparse tensor with CTD. Experimental results show that our method outperforms traditional estimation methods, and it can estimate traffic volume data even when the missing rate is high.

Semantic human activity (SHA) refers to users' activities performed in their daily lives (e.g., having dinner, shopping, etc.). SHA recognition is a promising issue in wearable and mobile computing. Most existing methods represent a SHA based on a single view, e.g., representing a SHA as a combination of human body actions, representing a SHA as a distribution of latent semantics. Since SHAs are complicated in nature, single views lack the ability of comprehensively profiling SHAs. In this paper, we propose a bi-view semi-supervised learning based method for recognizing SHAs using accelerometers. First, we represent a SHA based on two different views. One view represents a SHA as a distribution of latent activities in an unsupervised manner, and the other view represents a SHA as a set of human crafted features extracted in a hierarchical way. Second, we use a semi-supervised learning framework, which exploits the complementary information provided by the two views, to improve the classification accuracy based on both labeled and unlabeled data. Extensive experiments show that representing SHAs based on bi-views is more effective than representing SHAs based on single views, and our method is able to yield a competitive SHA recognition performance.

With the advent of multi-touch tabletops, software developers are creating innovative interfaces for applications such as interactive map systems. In this article, we summarized three representative interfaces from prevalent electronic map applications and literatures—gesture based, widget based, and hybrid, which supported both gestures and widgets—and then compared the effectiveness and the efficiency of these three interfaces by experiments. The results show that the hybrid interface had a better performance, requiring less task-completion time and operations, and causing less arm fatigue. The widget-based interface had the least distance traversed by fingers, but it required many tapping operations. The gesture-based interface was easier to learn compared to the widget-based interface, but the distance traversed during finger movements was the longest. Based on the findings from our experiments, we discuss the implications for the design of map exploration interfaces on multi-touch tabletops.

Mobile and wearable devices based activity recognition systems utilize built-in sensors to identify the activities performed by users pervasively. However, most of these systems do not explicitly present the sensing process to users and are prone to uncertainty. The presence of uncertainty makes users feel confused about the behaviors of activity recognition systems, which may affect the confidence of users. Uncertainty visualization has become an interesting research topic purporting to help users better understand systems. In this paper, we present an uncertainty visualization to reveal the process of mobile and wearable devices based activity recognition systems. We conducted an experiment to evaluate the uncertainty visualization by using a particular simulated mobile and wearable devices based activity recognition application. The results showed that the uncertainty visualization was effective in helping users understand and trust the judgments and inferences of the activity recognition application. Based on the advice of participants, we concluded a few directions to improve the uncertainty visualization.

The vast volumes of community-contributed geotagged photos (CCGPs) available on the Web can be utilized to make travel location recommendations. The sparsity of user location interactions makes it difficult to learn travel preferences, because a user usually visits only a limited number of travel locations. Static topic models can be used to solve the sparsity problem by considering user travel topics. However, all travel histories of a user are regarded as one document drawn from a set of static topics, ignoring the evolving of topics and travel preferences. In this paper, we propose a dynamic topic model (DTM) and matrix factorization (MF)-based travel recommendation method. A DTM is used to obtain the temporally fine-grained topic distributions (i.e., implicit topic information) of users and locations. In addition, a large amount of explicit information is extracted from the metadata and visual contents of CCGPs, check-ins, and point of interest categories datasets. The information is used to obtain user-user and location-location similarity information, which is imposed as two regularization terms to constraint MF. The proposed method is evaluated on a publicly available Flickr dataset. Experimental results demonstrate that the proposed method can generate significantly superior recommendations compared to other state-of-the-art travel location recommendation studies.

Air pollution has adverse effects on humans and ecosystem, and spatially fine-grained air quality information (i.e., the air quality information of every fine-grained area) can help people to avoid unhealthy outdoor activities. However, the number of air quality monitoring stations is usually limited, and thus spatially fine-grained air quality estimation is a challenging task. This paper proposes a method for inferring spatially fine-grained air quality information throughout a city. On one hand, since air quality is affected by multiple factors (e.g., factory waste gases and automobile exhaust fumes), this method employs various data sources, including traffic, road network, point of interests (POIs), and check-ins from social network services, which are related to air quality, to conduct the estimation. On the other hand, since the labeled data are highly limited due to the sparseness of monitoring stations, this method uses an improved ensemble semi-supervised learning (Semi-EP) to establish the relationship between the various data sources and urban air quality. Semi-EP firstly generates multiple classifiers from the original labeled data set and these classifiers are retrained in the iterative co-training process. Then, ensemble pruning technique is used to select the most-diverse subset from these multiple classifiers. This method is evaluated on the real-world dataset of Hangzhou city, China, and the experimental results have demonstrated its advantages over state-of-the-art methods.

Effectively utilizing multimodal information (e.g., heart rate and acceleration) is a promising way to achieve wearable sensor based human activity recognition (HAR). In this paper, an activity recognition approach MARCEL (Multimodal Activity Recognition with Classifier Ensemble) is proposed, which exploits the diversity of base classifiers to construct a good ensemble for multimodal HAR, and the diversity measure is obtained from both labeled and unlabeled data. MARCEL uses neural network (NN) as base classifiers to construct the HAR model, and the diversity of classifier ensemble is embedded in the error function of the model. In each iteration, the error of the model is decomposed and back-propagated to base classifiers. To ensure the overall accuracy of the model, the weights of base classifiers are learnt in the classifier fusion process with sparse group lasso. Extensive experiments show that MARCEL is able to yield a competitive HAR performance, and has its superiority on exploiting multimodal signals.

This article presents nonspeech audio (i.e., English-based spearcons and Chinese-based spearcons) to represent distance, and forward-direction for pedestrian navigation in an eyes-free environment. Experiment in the field is carried out with the involvement of 10 participants (i.e., native Chinese) using within-subject design to evaluate English-based spearcons, Chinese-based spearcons, and Chinese text-to-speech (TTS). Results from the experiment suggest that Chinese-based spearcons are efficient in task completion compared to Chinese TTS. Moreover, Chinese-based spearcons are more effective in conveying distance and forward-direction compared to English-based spearcons in pedestrian navigation. Overall, participants have shown their satisfaction with Chinese-based spearcons as an auditory feedback in pedestrian navigation.

With the prevalence of mobile social network services, people can post location-based questions on their social networks to satisfy their needs anytime anywhere. In this article, the authors study location-based questions that people post on microblogs, which is a popular form of social network service. The authors collected posts with geo-tags from Sina Weibo and conducted the study based on about a thousand location-based questions. Their results reveal unique characteristics of location-based questions by analyzing what people ask, how they ask, why they ask, and the context when asking. Location-based questions are closely related to people’s offline activities. Spatial restriction, subjectivity, interactivity, and propagation are the main characteristics that people value for choosing social networks to ask location-based questions. People also apply different phrasing skills to different types of questions. The questions people ask in different contexts also have different focuses. Based on their findings, the authors discuss practical design implications for social networks, location-based Q&A systems, and other applications with location-based features.

A personally semantic place is a space that is frequently visited by an individual user and carries important semantic meanings (e.g. home, work, etc.) to the user. Many location-aware applications could be greatly enhanced by the ability of automatic discovery of personally semantic places. The discovery of a user's personally semantic places involves obtaining the physical locations and semantic meanings of these places. In this paper, we propose approaches to address both of the problems. For the physical place extraction problem, a hierarchical clustering algorithm is proposed to firstly extract visit points from the GPS trajectories, and then clusters these visit points to form physical places. For the semantic place recognition problem, the temporal, spatial and sequential features in which the places have been visited are explored to categorize them into pre-defined types. An extensive set of experiments conducted based on a dataset of real-world GPS trajectories has demonstrated the effectiveness of the proposed approaches.

Knowledge workers frequently change activities, either by choice or through interruptions. With an increasing number of activities and activity switches, it is becoming more and more difficult for knowledge workers to keep track of their desktop activities. This article presents our efforts to achieve activity awareness through automatic classification of user's everyday desktop activities. For getting a deeper understanding, we investigate performance of various classifiers with respect to discriminative power of time-, interaction-, and content-based feature sets for different work scenarios and users. Specifically, by viewing an activity as a sequence of desktop interactions we present (1) a methodology for translating a user's desktop interactions into activities, (2) evaluation of the discriminative power of different activity features and feature types, and (3) analysis of supervised classification models for classifying desktop activity under two different scenarios, i.e., an activity-centric scenario and a user-centric scenario. The experiments are carried out on a real-world dataset, and the results show satisfactory accuracy using relatively few and simple types of features.

Road congestion has been one of the major issues in most metropolises, and thus, it is crucial to detect road congestions effectively and efficiently. Traditional solutions require the deployment of dedicated sensors on the roadside or on the vehicles, which suffer from high installation and maintenance costs and limited coverage. In this paper, we propose an alternative solution by exploiting the sensing ability of mobile phones. However, it is challenging to detect road congestions in a daily-living environment using undedicated mobile phones while guaranteeing energy efficiency. The proposed system only depends on the accelerometer and cellular signal, which have been proven to be energy efficient as compared with other built-in sensors (e.g., GPS). It consists of three interactive modules: (a) an accelerometer-based vehicular movement detection module for detecting the periods when the mobile phone user is traveling by vehicle; (b) a map-matching module relying on the cellular signal for determining the traveled road segments; and (c) a road congestion estimation module for inferring the congestion degree of the traveled road segments. We evaluated the proposed system based on real-world datasets, with promising results.

This article presents a new design of using nonspeech audio (i.e., earcons, spearcons, and short pulses) to represent distance and forward-direction for pedestrian navigation in eyes-free environment. Experiment in the field is carried out with the involvement of 15 participants using within-subject design to evaluate the newly developed earcons, spearcons, and short pulses for distance and forward-direction in pedestrian navigation. Results from the experiment suggest that spearcons are efficient in tasks completion, and it conveys distance and forward-direction information to participants more accurately compared with earcons and short pulses. Overall, participants have shown their satisfaction with spearcons as an audio feedback in pedestrian navigation.

In this paper, we propose an approach to build a multi-way concept hierarchy from a text corpus, which is based on WordNet and multi-way hierarchical clustering. In addition, a new evaluation metric is presented, and our approach is compared with 4 kinds of existing methods on the Amazon Customer Review data set.

The proliferation of digital cameras and the growing practice of online photo sharing using social media sites such as Flickr have resulted in huge volumes of geotagged photos available on the Web. Based on users' traveling preferences elicited from their travel experiences exposed on social media sites by sharing geotagged photos, we propose a new method for recommending tourist locations that are relevant to users (i.e., personalization) in the given context (i.e., context awareness). We obtain user-specific travel preferences from his/her travel history in one city and use these to recommend tourist locations in another city. Our technique is illustrated on a sample of publicly available Flickr dataset containing photos taken in various cities of China. Results show that our context-aware personalized method is able to predict tourists' preferences in a new or unknown city more precisely and generate better recommendations compared to other state-of-the-art landmark recommendation methods.

With the development of pervasive computing, location-aware pervasive games have proliferated quickly. To counter the inefficiency of information adaptation based on a location and a radius, this article presents the design and implementation of Your Way Your Missions (YWYM), which is a location-aware pervasive game exploiting the routes of players. YWYM provides a Google Maps–based tool for players to predefine routes, and utilizes self-reporting method to obtain the planned routes of players. By such a design, missions are returned to players in terms of the location properties of missions and the planned routes of players. A field study was conducted to investigate the usage of YWYM, and the results indicate that route predefining and self-reporting methods are an effective approach to obtain the planned routes of players, and information adaptation based on the planned routes of players could help them find missions at any place and any time.

This article presents the design of iReminder, an intuitive location-based reminder that delivers reminding messages based on users' future routes. iReminder is implemented on mobile phones, and it can predict users' future routes by collecting their daily trajectory data. Then it delivers a reminding message via the mobile phone when it senses that the user is going to the task location. A field study was conducted on how iReminder extends its potential to help users perform everyday tasks and compared the method adopted by iReminder with the method used by traditional location-based reminders. The experimental results show that iReminder outperforms traditional location-based reminder because it delivers reminding messages more appropriately. A detailed discussion is also given to investigate the ideal message delivery point, and the discussion results show that a location-based reminding message is more useful and more likely to be accepted by the user if it is triggered by considering his or her future route.

Geotagged photos of users on social media site, i.e., Flickr provide plentiful location-based data, which has been exploited for location-based services, such as mapping geotags to places and recommendation of personalized landmarks. As users’ preferences to visit a location or multiple locations in a certain sequence could be affected by their current temporal, and weather context. This paper considers the problem of mining context-aware significant semantic travel sequences from geotagged photos.

In this paper we present a novel concept of hybrid auditory feedback in mobility assistance for people with visual disabilities in indoor environment. Hybrid auditory feedback is a gradual con-version of sound from speech-only to non-speech (i.e., spearcons) based on the sound repetitiveness and the users' frequency of the travelled route. Using a within-subject design, eight participants carried out a task using a mobility assistant application and followed a same route for few days. Preliminary results suggest that hybrid sounds in auditory feedback are more effective than non-speech and are pleasant compared to speech-only.

A place is a locale that is frequently visited by an individual user and carries important semantic meanings (e.g. home, work, etc.). Many location-aware applications will be greatly enhanced with the ability of the automatic discovery of personally semantic places. The discovery of a user's personally semantic places involves obtaining the physical locations and semantic meanings of these places. In this paper, we propose approaches to address both of the problems. For the physical place extraction problem, a hierarchical clustering algorithm is proposed to firstly extract visit points from the GPS trajectories, and then these visit points can be clustered to form physical places. For the semantic place recognition problem, Bayesian networks (encoding the temporal patterns in which the places are visited) are used in combination with a customized POI (i.e. place of interest) database (containing the spatial features of the places) to categorize the extracted physical places into pre-defined types. An extensive set of experiments have been conducted to demonstrate the effectiveness of the proposed approaches based on a dataset of real-world GPS trajectories.

An average white-collar worker deals with enormous amount of digital information on daily basis. Recently, there has been a growing interest to support their work. However, in order to be really supportive there is a need to know the current activity of the user at all times. In this paper we present a new technique that takes advantage of temporal aspects of user activity behavior to infer when it is most likely that an activity switch is occurring. We then describe "Activity Switch Detector" an interactive switch notification system embodying these ideas, and an extensive user study by ten participants to test the validity of the approach and present its results.

We study the question of hand posture recognition by developing a new class of Bag-of-Features (BoF), namely Hierarchical BoF. The Hierarchical BoF captures spatial information by dividing whole hand area into several sub-regions and projecting local features onto horizontal and vertical directions. A similarity measurement based on Histogram Intersection Kernel (HIK) is proposed to classify hand postures. According to experimental result in Section 4, Hierarchical BoF works well in hand posture recognition task owing to the following properties: 1) the model captures spatial information of each component of hand postures; 2) the model has good adaptability for various complex background conditions.

Hand posture is a natural and effective interaction between human and robot. In this paper, we use monocular camera as input device, and an improved Bag-of-Words (BoW) method is proposed to detect and recognize hand posture based on a new descriptor ARPD (Appearance and Relative Position Descriptor) and spectral embedding clustering algorithm. To track hand motion rapidly and accurately, we have designed a new framework based on improved BoW and CAMSHIFT algorithm. The thorough evaluation of our algorithm is presented to show its usefulness.

Awareness of the user's current context, that is, the task the user resides in, can allow for various ways of supporting their work. In this article we present a novel approach for mining user's regular tasks on the basis of temporal proximity of users’ desktop actions without user intervention in a completely automatic, unobtrusive, and unsupervised manner. The proposed method is based on time-based clustering of a user's desktop actions and, in contrast to previous approaches, it does not enforce fixed time constraints on the mined actions, and typical transition times, when they exist, emerge from the logged activity data for that specific user. The performance of our technique was evaluated on a large data set of 724 days of desktop work by five knowledge workers. The results showed that our approach performed very well and was able to cluster those temporally proximate user actions that commonly used fix time window techniques fail to deal with. Notably, the main improvement is in the recall of the tasks, where our approach recalled 10.18% more actions than the predefined time window technique.

This paper presents a system where the personal route of a user is predicted using a probabilistic model built from the historical trajectory data . Route patterns are extracted from personal trajectory data using a novel mining algorithm, Continuous Route Pattern Mining (CRPM), which can tolerate different kinds of disturbance in trajectory data. Furthermore, a client–server architecture is employed which has the dual purpose of guaranteeing the privacy of personal data and greatly reducing the computational load on mobile devices. An evaluation using a corpus of trajectory data from 17 people demonstrates that CRPM can extract longer route patterns than current methods. Moreover, the average correct rate of one step prediction of our system is greater than 71%, and the average Levenshtein distance of continuous route prediction of our system is about 30% shorter than that of the Markov model based method.

Invariant feature descriptors such as SIFT and GLOH have been demonstrated to be very robust for image matching and object recognition. However, such descriptors are typically of high dimensionality, e.g. 128-dimension in the case of SIFT. This limits the performance of feature matching techniques in terms of speed and scalability. A new compact feature descriptor, called Kernel Projection Based SIFT (KPB-SIFT), is presented in this paper. Like SIFT, our descriptor encodes the salient aspects of image information in the feature point's neighborhood. However, instead of using SIFT's smoothed weighted histograms, we apply kernel projection techniques to orientation gradient patches. The produced KPB-SIFT descriptor is more compact as compared to the state-of-the-art, does not require pre-training step needed by PCA based descriptors, and shows superior advantages in terms of distinctiveness, invariance to scale, and tolerance of geometric distortions. We extensively evaluated the effectiveness of KPB-SIFT with datasets acquired under varying circumstances.

This paper presents a novel local image descriptor for dense wide-baseline matching purposes, coined SULD (Speeded-Up Local Descriptor). SULD approximates or even outperforms than previously proposed schemes such as SURF and DAISY, and can be computed and compared much faster. This is achieved by summing up the Haar wavelet responses rather than the gradient, by computing convolutions recursively and by using low dimensions descriptor. The proposed approach was tested with ground truth laser scanned depth maps as well as on image pairs of different resolutions and the results show that good reconstruction is achieved even with only two small images.

The development of multimedia annotation technique provides the possibility to redesign the interfaces of widely used media players, and EmoPlayer is such a media player that can be used to play video clips with affective annotations. A user can select a character in a video clip and view the distribution of his/her emotions along the video timeline through a colour bar based interface. Two experiments were conducted to evaluate the efficiency of affective annotation. The results of these experiments indicate that affective annotation is effective in both improving the speed of locating a specific scene within a video clip and helping comprehend a video clip in a limited viewing time period. Based on the analysis of recorded operations of participants, the strategies employed by participants and the factors that might influence the utilization of affective annotation are also highlighted.

Although there exists dozens of vision based 3D head tracking methods, none of them considers the problem of large motion, especially the movement along the Z axis. In this paper we propose a novel tracking method to handle this problem by using Scale Invariant Feature Transform (SIFT) based registration algorithm. Salient SIFT features are first detected and tracked between two images, and then the 3D points corresponding to these features are obtained from a stereo camera. With these 3D points, a registration algorithm in a RANSAC framework is employed to detect the outliers and estimate the head pose. Performance evaluation shows an accurate pose recovery (3° RMS) when the head has large motion, even with movement along the Z axis was about 150 cm.

Local-lag mechanism can maintain consistency for replicated continuous applications, but with a tradeoff of adding delay to local operations. To relieve the negative effects of the delay, this paper proposes an HCI method named echo. With the help of the echo method users can immediately perceive the results of their operations and how large the lag is. In order to evaluate the proposed method, a desktop collaborative virtual environment (CVE) system and a virtual object control task were employed to study the effects of the echo method on human performance (including task completion time, error count, and interaction quality). Experimental results indicate that when the lag exceeds 100 ms the echo method can improve human performance with the effects becoming more evident when a larger lag is used.

In this paper, an adaptive-filtering policy for distributed virtual environments (DVEs) was brought forward. It took advantages of two traditional filtering policies (static-filtering policy and equidistance-transmission policy) and could adaptively select the fittest filtering policy corresponding to current motion characteristic of the virtual object in DVEs. An experiment was designed to compare this filtering policy with the two traditional filtering policies on task performance. Objective (task completion time, state updates transmitted and state updates per second) and subjective (filtering quality) evaluations were recorded in the experiment. The experiment results show that if mean state updates per second is equal or greater than 20 in the task, SUT and SUPS can be decreased by adaptive-filtering policy without task performance affected obviously. However, if mean state updates per second is around 10 in the task, accompanied with SUT and SUPS decreased by adaptive-filtering policy, task performance also deteriorated badly

Nowadays, handheld wireless devices are being widely used by people to collaborate with one another using same handheld device in field or using desktop PC in office. Therefore, to study the effects of platform heterogeneity on task performance in collaborative virtual environments (CVEs) is helpful to improve future collaboration efficiency. In this paper, an experiment with highly interactive task was designed to study these effects. The task performance was evaluated under different combinations of three platforms: PC with a sensor, PDA with a stylus and PDA with direction buttons. The experiment results indicate that for simple 3D collaboration task, if right type of visualization and input device was chosen, we would be satisfied with performance both on homogeneous platforms and heterogeneous platforms

The "priority-based directed minimum spanning tree" (PST) is designed for distributed interactive applications (DIAs) in which the data are delivered to the receiver node according to their priority; however PST cannot balance the bandwidth usage among the nodes and the system will be unstable when its scale increases. This paper proposes a novel overlay multicast protocol named priority based overlay multicast (POM), which uses the priority, delay and available bandwidth to construct the multicast tree, besides a filtering mechanism is proposed to rebuild the tree when the available bandwidth shortage appears. The effectiveness of our protocol is demonstrated in the simulation results, showing that POM uses the available bandwidth efficiently, besides it keeps the system stable when its scale increases

Distributed Interactive Applications (DIAs) is different with other network-based applications, since the data should be delivered to the receiver node according to its priority. Among the existing overlay multicast protocols the "priority-based directed minimum spanning tree" (PST) is designed for DIAs. But in PST the available bandwidth (avail-bw) is used inefficiently and the system becomes unstable when its scale increases. This paper proposed a novel overlay multicast protocol named Priority Based Overlay Multicast (POM), which uses the entities’ priority, links’ delay and node’s avail-bw to build multicast tree. By using POM the nodes with high priority receive the data in short delay, while the nodes with low priority tolerate a long delay in order to balance the available bandwidth usage among the nodes. The simulation results show that POM uses the avail-bw efficiently and keeps the system stable when its scale increases. Keywords: DIAs; Priority; Available bandwidth.

The influences of network characteristics on the task performance in a desktop collaborative virtual environment (CVE) system were studied. In the experiment, two participants worked together to accomplish a collaborative task and a network emulator was utilized to simulate real network characteristics. The task used in the experiment was designed to be highly interactive and had rigid requirements on network characteristics (e.g. latency and packet loss). The objective performance and subjective assessment were recorded and analyzed quantitatively as a function of network latency and packet loss rate. The experiment results show that network latency has no distinct effect on task performance when network latency is less than 200 ms and lightly affect task performance when network latency is between 200 and 600 ms; packet loss has no effect on task performance when there are 40 state update messages received per second and lightly affect task performance when there are between 15 and 40 state update messages received per second.

An efficient DDM (data distribution management) approach-fuzzy grouping mechanism was presented. In order to reduce the matching times and increase the data filtering rate, this new mechanism adopted two techniques based on the fuzzy correlation space approach: fuzzy consistent relation based weight distribution and grid based preprocessing. Simulation results indicate that: compared with existing approaches, fuzzy grouping mechanism not only keeps high and stable data filtering rates in different situations but also greatly reduces matching times.

Local-lag mechanism can eliminate short-term inconsistencies for collaborative virtual environments (CVEs). However, local-lag mechanism increases the response time and deteriorates the task performance of the system when lag is large. This paper first analyzes the effects of local-lag mechanism on task performance in a pilot experiment. We find that the key reason for the task performance deterioration is that subjects do not know the current position of the virtual object controlled by him/her and how long the lag is. Based on this observation, this paper proposes a method named echo, which gives some cues to subjects and makes them perceive these information. Experiment results indicate that the method can improve the task performance significantly. Especially when lag exceeds 600 ms, almost half of subjects cannot finish the task without the help of echo method, but with the help of the echo method all subjects can finish the task fluently.

The increased speeds of PCs and networks have made media communication possible on the Internet. However, nearly ten years after the first release of Microsoft NetMeeting, Internet video telephony is still limited to the point-to-point communication mode. Today, people have a need for an easy-to-use multi-party video conferencing tool that can connect families and friends around the world over the Internet. We present DigiParty, a fully distributed multi-party video conferencing system. DigiParty employs a full mesh conferencing architecture and adopts a loosely coupled conferencing mode. A novel conference control protocol is designed with the system. DigiParty can be integrated with any existing instant messaging services and is applicable to all types of Internet connections.

In a collaborative virtual environment (CVE), users need to do some collaborative work, which is highly synchronous. To achieve effective collaboration, the level of simulation accuracy (LOA) of a user's behavior in CVE must be high. However, if we adopt a static high LOA, a system's scalability is greatly decreased. Thus a scalability request must compromise with a collaborative request. The paper analyses a CVE system's scalability request and collaborative request, and presents a classification of collaborative work and a definition of collaborative knowledge base (CKB). Based on CKB, a knowledge-based adaptive message filtering technique, which improves the CVE system's scalability, is put forward.