Recent advances in explainable recommendation have explored the integration of language models to analyze natural language rationales for user–item interactions. Despite their potential, existing methods often rely on ID-based representations that obscure semantic meaning and impose structural constraints on language models, thereby limiting their applicability in open-ended scenarios. These challenges are intensified by the complex nature of real-world interactions, where diverse user intents are entangled and collaborative signals rarely align with linguistic semantics. To overcome these limitations, we propose BEAT, a unified and transferable framework that tokenizes user and item behaviors into discrete, interpretable sequences. We construct a behavior vocabulary via a vector-quantized autoencoding process that disentangles macro-level interests and micro-level intentions from graph-based representations. We then introduce multi-level semantic supervision to bridge the gap between behavioral signals and language space. A semantic alignment regularization mechanism is designed to embed behavior tokens directly into the input space of frozen language models. Experiments on three public datasets show that BEAT improves zero-shot recommendation performance while generating coherent and informative explanations. Further analysis demonstrates that our behavior tokens capture fine-grained semantics and offer a plug-and-play interface for integrating complex behavior patterns into large language models.

Path representations are widely applied in smart-city tasks, such as travel time estimation, road classification, and trajectory search. Existing approaches effectively learn these representations by jointly utilizing multi-mode data, including GPS trajectories, road paths, and road networks. Although these methods achieve state-of-the-art performance, the theoretical understanding of the importance and effectiveness of key designs in path representation pre-training remains unclear. Moreover, these methods often rely on hard-constrained labeled data and manual design, resulting in limited performance gains and high annotation costs. In this paper, we proposeSimPRL, a simple yet powerful contrastive learning framework that capitalizes on unannotated data. Building on the exploration of key factors contributing to successful path representation learning, SimPRL integrates unannotated GPS trajectories and road paths directly through contrastive self-supervised learning. Specifically, we introduce a pre-training task that predicts correspondences between trajectories from different modes using a shared encoder, without requiring labeled data. We also design an auxiliary segment imputation task and a minimal trajectory sampling technique with random offsets for data augmentation. Additionally, we employ any trajectory within the mini-batch, except for the anchor itself, as the negative sample. Extensive experiments on two large-scale, real-world datasets from Chengdu and Porto demonstrate the effectiveness of SimPRL in two downstream tasks: travel time estimation and road classification. The results show that SimPRL not only surpasses state-of-the-art methods but also achieves more stable generalization and greater efficiency in terms of runtime and memory usage. Our source code is available athttps://github.com/TXLiao/SimPRL

Mobile signaling data has been used to discover comprehensive and fine-grained patterns of individual travel activities. However, mobile signaling data often exhibits notable data quality issues caused by systematic factors (e.g., weather, construction, congestion) or individual factors (e.g., low battery, poor reception). Improving the quality of signaling data is a crucial yet challenging task due to irregular spatiotemporal intervals and complex inter-correlations among base stations. In this paper, we formalize the augmentation of signaling data as a base station location reconstruction task. We propose a location-aware transformer structure named SigFormer, which employs self-supervised learning to reconstruct the location information of missing base stations based on the sampled signaling sequence. Specifically, we first design a Continuous Signaling Encoder to model irregularly sampled signaling sequences and encode interval information generated by base station transitions. Then, we learn the Base Station Embedding to describe implicit features of base stations and design a Neighbor Region Encoder to incorporate geographic information as an auxiliary for base station representations. Finally, through an attention-based encoder-decoder framework, we aggregate location information and base station features, employing sequence learning to capture spatiotemporal dependencies and reconstruct base station locations. Experiment results on real-world datasets indicate that our method outperforms existing approaches for location reconstruction. The proposed method originates from practical demands in telecom systems, addressing the challenge of missing and abnormal base station locations by modeling spatiotemporal transition patterns in signaling data. Our approach supports various tasks in practical applications, including imputing missing base station locations and correcting anomalous locations, effectively improving the quality of signaling data.

Spatio-temporal forecasting is pivotal in numerous real-world applications, including transportation planning, energy management, and climate monitoring. In this work, we aim to harness the reasoning and generalization abilities of Pre-trained Language Models (PLMs) for more effective spatio-temporal forecasting, particularly in data-scarce scenarios. However, recent studies uncover that PLMs, which are primarily trained on textual data, often falter when tasked with modeling the intricate correlations in numerical time series, thereby limiting their effectiveness in comprehending spatio-temporal data. To bridge the gap, we propose RePST, a semantic-oriented PLM reprogramming framework tailored for spatio-temporal forecasting. Specifically, we first propose a semantic-oriented decomposer that adaptively disentangles spatially correlated time series into interpretable sub-components, which facilitates PLM to understand sophisticated spatio-temporal dynamics via a divide-and-conquer strategy. Moreover, we propose a selective discrete reprogramming scheme, which introduces an expanded spatio-temporal vocabulary space to project spatio-temporal series into discrete representations. This scheme minimizes the information loss during reprogramming and enriches the representations derived by PLMs. Extensive experiments on real-world datasets show that the proposed RePST outperforms twelve state-of-the-art baseline methods, particularly in data-scarce scenarios, highlighting the effectiveness and superior generalization capabilities of PLMs for spatio-temporal forecasting. Codes and Appendix can be found at https://github.com/usail-hkust/REPST.

Large-scale human mobility data contains rich spatial and temporal information for urban sensing, crowd flow modeling, and urban planning. However, it is usually difficult to access wide-coverage, long-term, and consistent-time human mobility data. Most of the publicly available datasets are actually only records of discontinuous trajectories of a very small portion of urban citizens in asynchronous time due to the limited usage of apps for location data collection or the limited number of volunteers. To address this problem and empower open urban research, this paper constructs a high-quality human mobility dataset by generating large-scale citizen trajectories based on massive cellular signaling data. Particularly, we first propose a heatmap diffusion module to generate a probability heatmap that produces plausible trajectories at both the individual and city scales. Then, we propose a masked trajectory AutoEncoder, which can generate individual trajectory embeddings from partially given or empty trajectories. Third, a flexible framework is provided to incorporate the heatmap diffusion module with the masked trajectory embeddings, demonstrating significant flexibility in handling both fully masked trajectories for city-wide analysis and partially masked trajectories for specific locations. We have conducted extensive experiments to validate the utility of the regenerated trajectories at both individual and region levels for various applications. Numerous case studies further illustrate that our model learns not only the distribution of the trajectories but also the semantics of different urban areas.

Bike sharing systems have been widely deployed in urban cities for first- and last-mile transportation. However, because of the geographical and temporal imbalance of bike demand, bikes need to be reallocated system-wide among stations during the night to maintain a high service level while minimizing demand loss due to stockout or overcapacity. Two technical challenges remain in optimizing the static bike rebalancing operations. One challenge is to accurately predict bike pickup and dropoff demand at each station, considering demand substitution effects and subsequently determining the optimal rebalancing quantity for each station. The other is to efficiently optimize the routing of multiple rebalancing vehicles for large-scale bike sharing systems, considering outlier stations with rebalancing quantities exceeding vehicle capacity. To this end, we propose an end-to-end solution to tackle the aforesaid challenges. Specifically, we first develop deep learning-based predictors that capture the time dependencies of station-level demand, the impact of weather conditions, and the demand substitution effect by nearby stations. Based on the demand rate, a sequential simulation-based demand loss estimator is developed to find the optimal rebalancing quantities that lead to the minimum expected demand loss. Then, a mixed integer linear programming model is formulated to optimize the routing problem of rebalancing vehicles. To address the computational challenge, we propose a data-driven decomposition algorithm to support a multivehicle multivisit rebalancing strategy by decomposing the multivehicle routing problem into smaller and tractable single-vehicle routing problems, which can be solved in parallel. Finally, extensive numerical experiments using real-world data from New York City Citi Bike demonstrate the accuracy of the proposed bike demand predictors, the impact of demand substitution, and the efficiency of the data-driven optimization framework. History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Learning. Funding: This work was supported by the National Natural Science Foundation of China [Grant 72201222] and the Hong Kong Research Grants Council [Grants CityU 21500220 and CityU 11504322]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0182 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0182 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Multi-task architecture learning has achieved significant success by learning optimal sharing architectures for different tasks. However, previous works to learn branched architectures for different tasks can sometimes lead to unsatisfying multi-task performance, as not all detailed branches are relevant to a specific task. Task-relevant architectures can be sparse, including only partial channels or layers in the entire architecture (i.e., a sub-network). In addition, most previous works rely on a heuristic architecture selection procedure that could not support continuous architecture optimization. To this end, in this paper, we propose dual-mask , a progressively sparse multi-task architecture learning method. Starting with a task-free architecture, it identifies the informative features along two-level, channels and layers, for each task, while suppressing conflicting or noisy parts in a differentiable manner, so that better task-specific sub-networks are captured. Specifically, the channel and layer selection modules produce respective hybrid binary and real value masks, designed to pick salient channels and layers for each task, respectively. To jointly optimize masks with model parameters, we propose an importance-guided relaxation method for solving the stochastic binary optimization problem , after which the interference or noise parts can be pruned by masks. Additionally, a progressive training strategy with continuation is provided that gradually sparsity the task-specific sub-networks. Experiments show that dual-mask achieves superior performance than SOTA multi-task methods.

Information diffusion prediction, aimed at estimating future interacting users for a given content, is crucial for various applications on online social platforms. Recently, methods based on dynamic graph learning have achieved superior performance. However, these methods often face scalability issues due to their full-neighbor aggregation, which requires loading the whole diffusion graph, making them impractical for large graphs. While improving model scalability through sampling is an immediate approach, it is challenging on the diffusion graph due to various user dependencies (i.e., the temporal and structural correlations of user–item interactions). To address this problem, we propose a new model named ASDIP, which performs adaptive sampling on the diffusion graph. Specifically, ASDIP employs multiple sampling strategies to extract walks from the diffusion graph, each identifying a representative user dependency by sampling walks that satisfy a specific temporal constraint. Next, the walks sampled by different strategies are first mapped into distinct strategy-specific user representations and then merged into a unified user representation, adaptively fusing the information obtained from different strategies. Finally, a cascade representation learning module is proposed to generate cascade representations based on user representations and interaction timestamps. Experimental results validate the effectiveness and scalability of ASDIP.

Knowledge Tracing aims to assess student learning states by predicting their performance in answering questions. Different from the existing research which utilizes fixed-length learning sequence to obtain the student states and regards KT as a static problem, this work is motivated by three dynamical characteristics: 1) The scales of students answering records are constantly growing; 2) The semantics of time intervals between the records vary; 3) The relationships between students, questions and concepts are evolving. The three dynamical characteristics above contain the great potential to revolutionize the existing knowledge tracing methods. Along this line, we propose a Dynamic Graph-based Knowledge Tracing model, namely DyGKT. In particular, a continuous-time dynamic question-answering graph for knowledge tracing is constructed to deal with the infinitely growing answering behaviors, and it is worth mentioning that it is the first time dynamic graph learning technology is used in this field. Then, a dual time encoder is proposed to capture long-term and short-term semantics among the different time intervals. Finally, a multiset indicator is utilized to model the evolving relationships between students, questions, and concepts via the graph structural feature. Numerous experiments are conducted on five real-world datasets, and the results demonstrate the superiority of our model. All the used resources are publicly available at https://github.com/PengLinzhi/DyGKT.

Structure encoding has proven to be the key feature to distinguishing links in a graph. However, Structure encoding in the temporal graph keeps changing as the graph evolves, repeatedly computing such features can be time-consuming due to the high-order subgraph construction. We develop the Co-Neighbor Encoding Schema (CNES) to address this issue. Instead of recomputing the feature by the link, CNES stores information in the memory to avoid redundant calculations. Besides, unlike the existing memory-based dynamic graph learning method that stores node hidden states, we introduce a hashtable-based memory to compress the adjacency matrix for efficient structure feature construction and updating with vector computation in parallel. Furthermore, CNES introduces a Temporal-Diverse Memory to generate long-term and short-term structure encoding for neighbors with different structural information. A dynamic graph learning framework, Co-Neighbor Encoding Network (CNE-N), is proposed using the aforementioned techniques. Extensive experiments on thirteen public datasets verify the effectiveness and efficiency of the proposed method.

Dynamic graph learning has attracted much attention in recent years due to the fact that most of the real-world graphs are dynamic and evolutionary.As a result, many dynamic learning methods have been proposed to cope with the changes of node states over time.Among these studies, a critical issue is how to update the representations of nodes when new temporal events are observed.In this paper, we provide a novel memory structure -Memory Map (MemMap) for this problem.MemMap is an adaptive and evolutionary latent memory space, where each cell corresponds to an evolving "topic" of the dynamic graph.Moreover, the representation of a node is generated from its semantically correlated memory cells, rather than linked neighbors of the node.We have conducted experiments on real-world datasets and compared our method with the SOTA ones.It can be concluded that: 1) By constructing an adaptive and evolving memory structure during the dynamic learning process, our method can capture the dynamic graph changes, and the learned MemMap is actually a compact evolving structure organized according to the latent "topics" of the graph nodes.2) Our research suggests that it is a more effective and efficient way to generate node representations from a latent semantic space (like MemMap in our method) than from directly connected neighbors (like most of the previous graph learning methods).The reason is that the number of memory cells in latent space could be much smaller than the number of nodes in a real-world graph, and the representation learning process could well balance the global and local message passing by leveraging the semantic similarity of graph nodes via the correlated memory cells.

Temporal link prediction, aiming at predicting future interactions among entities based on historical interactions, is crucial for a series of real-world applications. Although previous methods have demonstrated the importance of relative encodings for effective temporal link prediction, computational efficiency remains a major concern in constructing these encodings. Moreover, existing relative encodings are usually constructed based on structural connectivity, where temporal information is seldom considered. To address the aforementioned issues, we first analyze existing relative encodings and unify them as a function of temporal walk matrices. This unification establishes a connection between relative encodings and temporal walk matrices, providing a more principled way for analyzing and designing relative encodings. Based on this analysis, we propose a new temporal graph neural network called TPNet, which introduces a temporal walk matrix that incorporates the time decay effect to simultaneously consider both temporal and structural information. Moreover, TPNet designs a random feature propagation mechanism with theoretical guarantees to implicitly maintain the temporal walk matrices, which improves the computation and storage efficiency. Experimental results on 13 benchmark datasets verify the effectiveness and efficiency of TPNet, where TPNet outperforms other baselines on most datasets and achieves a maximum speedup of $33.3 \times$ compared to the SOTA baseline.

Travel time estimation (TTE) is a fundamental and challenging problem for navigation and travel planning. Though many efforts have been devoted to this task, most of the previous research has focused on extracting useful features of the routes to improve the estimation accuracy. In our opinion, the key issue of TTE is how to handle the rich spatiotemporal information underlying a route and how to model the multi-faceted factors that affect travel time. Along this line, we propose a multi-faceted route representation learning framework that divides a route into three sequences: a trajectory sequence consists of GPS coordinates to describe spatial information, an attribute sequence to encode the features of each road segment, and a semantic sequence consists of the IDs of road segments to capture the context information of routes. Then, we design a sequential learning module and transformer encoder to get the representations of three sequences for each route respectively. Finally, we fuse the multi-faceted route representations together, and provide a self-supervised learning module to improve the generalization of final representation. Experiments on two real-world datasets demonstrate that our method could provide more accurate travel time estimation than baselines, and all the multi-faceted route representations contribute to the improvement of estimation accuracy.

Origin-Destination demand prediction is a fundamental and important task in the urban transportation system. It is more challenging and complex than region demand prediction since it needs to predict traffic demand for each pair of regions rather than a single region, which means$N^{2}$time series need to be predicted given$N$stations. Most existing works are mainly proposed for the region (or station) demand prediction. On the other hand, previous Origin-Destination demand prediction methods only follow a single discrete-time setting while the input data are continuous-time OD orders, which means these methods have not sufficiently leveraged the rich information for demand prediction. To solve these challenges, we propose a novel framework consisting of both continuous-time and discrete-time representation learning modules for Origin-Destination demand prediction. Firstly, we construct memorable representations of all nodes and design a continuous-time learning module to update the nodes’ representations once a time-stamped OD transaction is observed. Then, a discrete-time representation learning module is proposed to generate discrete-time messages containing information across a fixed time interval from a macro perspective. Next, a co-updater module is designed to fuse messages from both continuous-time and discrete-time channels into node memory. Last, a graph attention module is developed, which generates final node embeddings using updated node memory and predicts the forthcoming OD demand matrix based on them. The experimental results on real-world datasets show that our method leads to significant and consistent improvements compared to other methods.

Anomaly detection in multivariate time series is a critical research area, but it is also a challenging one due to its occurrence in various real-world scenarios, such as structural health monitoring and risk management. Traditional approaches for anomaly detection rely on deviating distribution and a static threshold that is set manually. However, static thresholds fail to detect contextual anomalies, leading to a high ratio of false anomalies. Therefore, a self-adaptive thresholding method is required to improve the accuracy of anomaly detection. In this study, we propose HCR-AdaAD, a multivariate anomaly detection framework that combines hierarchical context representation learning with deep learning methods. The core idea is to extract normal time-series patterns by transforming them into images, which can be used to extract spatial features and generate robust representations for normal time series. Next, we adopt Extreme Value Theory (EVT) to set self-adaptive thresholds in streaming time series, which can contribute to the ideal precision for anomaly detection and high interpretability with contextual information. We conducted evaluation experiments on three public datasets, and the results demonstrate the effectiveness and soundness of our proposed model. HCR-AdaAD offers a novel and effective approach to anomaly detection in multivariate time series that outperforms traditional methods, making it a promising solution for real-world applications in various domains.

Numerous spatial-temporal learning methods have been proposed for crowd flow modeling, which is an important problem in Intelligent Transportation Systems (ITS). However, most of the existing methods were designed to use data in one specific form to solve one particular task of crowd flow modeling and the shared patterns among different tasks have been largely ignored. In this paper, we investigate how to learn generic node representations that can simultaneously support various downstream tasks of crowd flow modeling. Along this line, we develop a continuous-time dynamic graph representation learning method based onBootstrapping forCrowdFlow modeling (BootCF). Our approach follows a training procedure with two phases. In the pre-training phase, the continuous-time dynamic encoder converts edges with timestamps into messages to update the representations of the related traffic nodes. Inspired by the recent progress of contrastive learning, a bootstrapping framework for continuous-time dynamic graphs is designed to calculate pre-training loss and update the model in a self-supervised way, and thus enabling the node representation learning to be task-agnostic. Moreover, a context-aware data augmentation on continuous-time dynamic graphs is proposed to generate the augmented view of input data. Once the general node representations are obtained, the second phase can learn an effective model for any downstream task. Experiments on two real-world datasets show that our approach can achieve significant performance gain on four downstream tasks, which demonstrates that the proposed method has the powerful generalization capability for learning task-agnostic node representations.

Given a sequence of sets, where each set has a timestamp and contains an arbitrary number of elements, temporal sets prediction aims to predict the elements in the subsequent set. Previous studies for temporal sets prediction mainly focus on the modelling of elements and implicitly represent each user's preference based on his/her interacted elements. However, user preferences are often continuously evolving and the evolutionary trend cannot be fully captured with the indirect learning paradigm of user preferences. To this end, we propose a continuous-time user preference modelling framework for temporal sets prediction, which explicitly models the evolving preference of each user by maintaining a memory bank to store the states of all the users and elements. Specifically, we first construct a universal sequence by arranging all the user-set interactions in a non-descending temporal order, and then chronologically learn from each user-set interaction. For each interaction, we continuously update the memories of the related user and elements based on their currently encoded messages and past memories. Moreover, we present a personalized user behavior learning module to discover user-specific characteristics based on each user's historical sequence, which aggregates the previously interacted elements from dual perspectives according to the user and elements. Finally, we develop a set-batch algorithm to improve the model efficiency, which can create time-consistent batches in advance and achieve 3.5× and 3.0× speedups in the training and evaluation process on average. Experiments on four real-world datasets demonstrate the superiority of our approach over state-of-the-arts under both transductive and inductive settings. The good interpretability of our method is also shown.

Accurate prediction of what types of patents that companies will apply for in the next period of time can figure out their development strategies and help them discover potential partners or competitors in advance. Although important, this problem has been rarely studied in previous research due to the challenges in modeling companies-continuously evolving preferences and capturing the semantic correlations of classification codes. To fill this gap, we propose an event-based dynamic graph learning framework for patent application trend prediction. In particular, our method is founded on the memorable representations of both companies and patent classification codes. When a new patent is observed, the representations of the related companies and classification codes are updated according to the historical memories and the currently encoded messages. Moreover, a hierarchical message passing mechanism is provided to capture the semantic proximities of patent classification codes by updating their representations along the hierarchical taxonomy. Finally, the patent application trend is predicted by aggregating the representations of the target company and classification codes from static, dynamic and hierarchical perspectives. Experiments on real-world data demonstrate the effectiveness of our approach under various experimental conditions, and also reveal the abilities of our method in learning semantics of classification codes and tracking technology developing trajectories of companies.

Designing new molecules is essential for drug discovery and material science. Recently, deep generative models that aim to model molecule distribution have made promising progress in narrowing down the chemical research space and generating high-fidelity molecules. However, current generative models only focus on modeling 2-D bonding graphs or 3-D geometries, which are two complementary descriptors for molecules. The lack of ability to jointly model them limits the improvement of generation quality and further downstream applications. In this article, we propose a joint 2-D and 3-D graph diffusion model (JODO) that generates geometric graphs representing complete molecules with atom types, formal charges, bond information, and 3-D coordinates. To capture the correlation between 2-D molecular graphs and 3-D geometries in the diffusion process, we develop a diffusion graph transformer (DGT) to parameterize the data prediction model that recovers the original data from noisy data. The DGT uses a relational attention mechanism that enhances the interaction between node and edge representations. This mechanism operates concurrently with the propagation and update of scalar attributes and geometric vectors. Our model can also be extended for inverse molecular design targeting single or multiple quantum properties. In our comprehensive evaluation pipeline for unconditional joint generation, the experimental results show that JODO remarkably outperforms the baselines on the QM9 and GEOM-Drugs datasets. Furthermore, our model excels in few-step fast sampling, as well as in inverse molecule design and molecular graph generation. Our code is provided in https://github.com/GRAPH-0/JODO.

Patent classification aims to assign multiple International Patent Classification (IPC) codes to a given patent. Existing methods for automated patent classification primarily focus on analyzing the text descriptions of patents. However, apart from the textual information, each patent is also associated with some assignees, and the knowledge of their previously applied patents can often be valuable for accurate classification. Furthermore, the hierarchical taxonomy defined by the IPC system provides crucial contextual information and enables models to leverage the correlations between IPC codes for improved classification accuracy. However, existing methods fail to incorporate the above aspects and lead to reduced performance. To address these limitations, we propose an integrated framework that comprehensively considers patent-related information for patent classification. To be specific, we first present an IPC codes correlations learning module to capture both horizontal and vertical information within the IPC codes. This module effectively captures the correlations by adaptively exchanging and aggregating messages among IPC codes at the same level (horizontal information) and from both parent and children codes (vertical information), which allows for a comprehensive integration of knowledge and relationships within the IPC hierarchical taxonomy. Additionally, we design a historical application patterns learning component to incorporate previous patents of the corresponding assignee by aggregating high-order temporal information via a dual-channel graph neural network. Finally, our approach combines the contextual information from patent texts, which encompasses the semantics of IPC codes, with assignees’ sequential preferences to make predictions. Experimental evaluations on real-world datasets demonstrate the superiority of our proposed approach over existing methods. Moreover, we present the model’s ability to capture the temporal patterns of assignees and the semantic dependencies among IPC codes.

Given a sequence of sets, where each set contains an arbitrary number of elements, temporal sets prediction aims to predict which elements will appear in the subsequent set. Existing methods for temporal sets prediction are developed on sophisticated components (e.g., recurrent neural networks, attention or gating mechanisms, and graph neural networks), which inevitably increase the model complexity due to more trainable parameters and higher computational costs. Moreover, the involved nonlinear activation may contribute little or even degrade the performance. In this paper, we present a succinct architecture that is solely built on the Simplified Fully Connected Networks (SFCNs) for temporal sets prediction to bring both effectiveness and efficiency together. In particular, given a user's sequence of sets, we employ SFCNs to derive representations of the user by learning inter-set temporal dependencies, intra-set element relationships, and intra-embedding channel correlations. Two families of general functions are introduced to preserve the permutation-invariant property of each set and the permutation-equivariant property of elements in each set. Moreover, we design a user representations adaptive fusing module to aggregate user representations according to each element for improving the prediction performance. Experiments on four benchmarks show the superiority of our approach over the state-of-the-art under both transductive and inductive settings. We also theoretically and empirically demonstrate that our model has lower space and time complexity than baselines. Codes and datasets are available at https://github.com/yule-BUAA/SFCNTSP.

Many deep spatio-temporal learning methods have been proposed for crowd flow modeling in recent years. However, most of them focus on designing a spatial and temporal convolution mechanism to aggregate information from nearby nodes and historical observations for a pre-defined prediction task. Different from the existing research, this paper aims to provide a generic and dynamic representation learning method for crowd flow modeling. The main idea of our method is to maintain a continuous-time representation for each node, and update the representations of all nodes continuously according to the streaming observed data. Along this line, a particular encoder-decoder architecture is proposed, where the encoder converts the newly happened transactions into a timestamped message, and then the representations of related nodes are updated according to the generated message. The role of the decoder is to guide the representation learning process by reconstructing the observed transactions based on the most recent node representations. Moreover, a number of virtual nodes are added to discover macro-level spatial patterns and also share the representations among spatially-interacted stations. Experiments have been conducted on two real-world datasets for four popular prediction tasks in crowd flow modeling. The result demonstrates that our method could achieve better prediction performance for all the tasks than baseline methods.

Learning the underlying distribution of molecular graphs and generating high-fidelity samples is a fundamental research problem in drug discovery and material science. However, accurately modeling distribution and rapidly generating novel molecular graphs remain crucial and challenging goals. To accomplish these goals, we propose a novel Conditional Diffusion model based on discrete Graph Structures (CDGS) for molecular graph generation. Specifically, we construct a forward graph diffusion process on both graph structures and inherent features through stochastic differential equations (SDE) and derive discrete graph structures as the condition for reverse generative processes. We present a specialized hybrid graph noise prediction model that extracts the global context and the local node-edge dependency from intermediate graph states. We further utilize ordinary differential equation (ODE) solvers for efficient graph sampling, based on the semi-linear structure of the probability flow ODE. We also combine the solvers with gradient guidance from the molecule property predictor for similarity-constrained molecule optimization. Experiments on diverse datasets validate the effectiveness of our framework. Particularly, the proposed method still generates high-quality molecular graphs in a limited number of steps.

The essential task of urban planning is to generate the optimal land-use configuration of a target area. However, traditional urban planning is time-consuming and labor-intensive. Deep generative learning gives us hope that we can automate this planning process and come up with the ideal urban plans. While remarkable achievements have been obtained, they have exhibited limitations in lacking awareness of: 1) the hierarchical dependencies between functional zones and spatial grids; 2) the peer dependencies among functional zones; and 3) human regulations to ensure the usability of generated configurations. To address these limitations, we develop a novel human-instructed deep hierarchical generative model. We rethink the urban planning generative task from a unique functionality perspective, where we summarize planning requirements into different functionality projections for better urban plan generation. To this end, we develop a three-stage generation process from a target area to zones to grids. The first stage is to label the grids of a target area with latent functionalities to discover functional zones. The second stage is to perceive the planning requirements to form urban functionality projections. We propose a novel module: functionalizer to project the embedding of human instructions and geospatial contexts to the zone-level plan to obtain such projections. Each projection includes the information of land-use portfolios and the structural dependencies across spatial grids in terms of a specific urban function. The third stage is to leverage multi-attentions to model the zone-zone peer dependencies of the functionality projections to generate grid-level land-use configurations. Finally, we present extensive experiments to demonstrate the effectiveness of our framework.

Spatiotemporal data mining plays an important role in air quality monitoring, crowd flow modeling, and climate forecasting. However, the originally collected spatiotemporal data in real-world scenarios is usually incomplete due to sensor failures or transmission loss. Spatiotemporal imputation aims to fill the missing values according to the observed values and the underlying spatiotemporal dependence of them. The previous dominant models impute missing values autoregressively and suffer from the problem of error accumulation. As emerging powerful generative models, the diffusion probabilistic models can be adopted to impute missing values conditioned by observations and avoid inferring missing values from inaccurate historical imputation. However, the construction and utilization of conditional information are inevitable challenges when applying diffusion models to spatiotemporal imputation. To address above issues, we propose a conditional diffusion framework for spatiotemporal imputation with enhanced prior modeling, named PriSTI. Our proposed framework provides a conditional feature extraction module first to extract the coarse yet effective spatiotemporal dependencies from conditional information as the global context prior. Then, a noise estimation module transforms random noise to realistic values, with the spatiotemporal attention weights calculated by the conditional feature, as well as the consideration of geographic relationships. PriSTI outperforms existing imputation methods in various missing patterns of different real-world spatiotemporal data, and effectively handles scenarios such as high missing rates and sensor failure. The implementation code is available at https://github.com/LMZZML/PriSTI.

Feature generation aims to generate new and meaningful features to create a discriminative representation space. A generated feature is meaningful when the generated feature is from a feature pair with inherent feature interaction. In the real world, experienced data scientists can identify potentially useful feature-feature interactions, and generate meaningful dimensions from an exponentially large search space in an optimal crossing form over an optimal generation path. But, machines have limited human-like abilities. We generalize such learning tasks as self-optimizing feature generation. Self-optimizing feature generation imposes several under-addressed challenges on existing systems: meaningful, robust, and efficient generation. To tackle these challenges, we propose a principled and generic representation-crossing framework to solve self-optimizing feature generation. To achieve hashing representation, we propose a three-step approach: feature discretization, feature hashing, and descriptive summarization. To achieve reinforcement crossing, we develop a hierarchical reinforcement feature crossing approach. We present extensive experimental results to demonstrate the effectiveness and efficiency of the proposed method. The code is available at https://github.com/yingwangyang/HRC_feature_cross.git.

Information propagation on social networks could be modeled as cascades, and many efforts have been made to predict the future popularity of cascades. However, most of the existing research treats a cascade as an individual sequence. Actually, the cascades might be correlated with each other due to the shared users or similar topics. Moreover, the preferences of users and semantics of a cascade are usually continuously evolving over time. In this paper, we propose a continuous-time graph learning method for cascade popularity prediction, which first connects different cascades via a universal sequence of user-cascade and user-user interactions and then chronologically learns on the sequence by maintaining the dynamic states of users and cascades. Specifically, for each interaction, we present an evolution learning module to continuously update the dynamic states of the related users and cascade based on their currently encoded messages and previous dynamic states. We also devise a cascade representation learning component to embed the temporal information and structural information carried by the cascade. Experiments on real-world datasets demonstrate the superiority and rationality of our approach.

Popularity prediction, which aims to forecast how many users would like to interact with a target item or online content in the future, can help online shopping or social media platforms to identify popular items or digital contents. Many efforts have been made to study how the multi-faceted factors, such as item features, user preferences, and social influence, affect user-item interactions, but little work has focused on the evolutionary dynamics of these factors for individuals or groups. In that light, this paper develops a community-based dynamic graph learning method for popularity prediction. First, a dynamic graph learning framework is proposed to maintain a dynamic representation for each item or user entity and update the representations according to the newly observed user-item interactions. Second, a community detection module is designed to capture the evolving community structures and identify the most influential nodes. More importantly, our framework leverages a community-level message passing during the learning process to balance local and global information propagation. Finally, we predict the popularity of the target item or online content based on the learned representations. Our experimental results based on three real-world datasets demonstrate that the proposed method achieves better performance than the baselines. Our method could not only model the changes in a user's preferences, but also capture how the communities evolve over time.

We propose DyGFormer, a new Transformer-based architecture for dynamic graph learning. DyGFormer is conceptually simple and only needs to learn from nodes' historical first-hop interactions by: (1) a neighbor co-occurrence encoding scheme that explores the correlations of the source node and destination node based on their historical sequences; (2) a patching technique that divides each sequence into multiple patches and feeds them to Transformer, allowing the model to effectively and efficiently benefit from longer histories. We also introduce DyGLib, a unified library with standard training pipelines, extensible coding interfaces, and comprehensive evaluating protocols to promote reproducible, scalable, and credible dynamic graph learning research. By performing exhaustive experiments on thirteen datasets for dynamic link prediction and dynamic node classification tasks, we find that DyGFormer achieves state-of-the-art performance on most of the datasets, demonstrating its effectiveness in capturing nodes' correlations and long-term temporal dependencies. Moreover, some results of baselines are inconsistent with previous reports, which may be caused by their diverse but less rigorous implementations, showing the importance of DyGLib. All the used resources are publicly available at https://github.com/yule-BUAA/DyGLib.

With the explosive accumulation of urban geographic, mobile, and IoT service data, AI-assisted automated urban planning, with a goal of configuring land-uses, has become an emerging interdisciplinary topic for smart cities. Existing literature mostly views urban planning as a generative task from the perspective of generating land-use configuration images. Such perspective is limited by two issues: 1) hierarchical planning dependency across multi scales : there are hierarchical dependencies between region-level urban function configurations and block-level building configurations. 2) sequential planning dependency within a scale : when planning the buildings of a place, planning a shopping mall can impose constraints on planning subsequent Points of Interest (POIs). In response, we propose a new perspective of formulating urban planning as a hierarchical decision process. That is, given a target region with many geographic blocks, a machine planner firstly selects the optimized urban function portfolios, thereafter, sequentially selects the most appropriate POI for each block based on its urban functions and previously-placed POIs over planning steps. We reformulate this decision process into a hierarchical reinforcement learning task and develop a novel hierarchical reinforced urban planning framework. This framework includes two components: 1) In region-level configuration, we present an actor- critic based method to overcome the challenge of weak reward feedback in planning the urban functions of regions. 2) In block-level configuration, we propose a single-agent iterative POI allocation strategy to model dependencies between POIs and urban functions, and between current and previous POIs. Finally, we present extensive experimental results on real-world urban data to demonstrate the enhanced performances of the “planning as hierarchical decision process” perspective and the reinforced planning model.

Forecasting traffic flow is an important task in urban areas, and a large number of methods have been proposed for traffic flow prediction. However, most of the existing methods follow a general technical route to aggregate historical information spatially and temporally. In this paper, we propose a different approach for traffic flow prediction. Our major motivation is to more effectively incorporate various intrinsic patterns in real-world traffic flows, such as fixed spatial distributions, topological correlations, and temporal periodicity. Along this line, we propose a novel autoencoder-based traffic flow prediction method, named Spatio-Temporal AutoEncoder (ST-AE). The core of our method is an autoencoder specially designed to learn the intrinsic patterns from traffic flow data, and encode the current traffic flow information into a low-dimensional representation. The prediction is made by simply projecting the current hidden states to the future hidden states, and then reconstructing the future traffic flows with the trained autoencoder. We have conducted extensive experiments on four real-world data sets. Our method outperforms existing methods in several settings, particularly for long-term traffic flow prediction.

Multivariate time series anomaly detection has great potentials in many practical applications. Extreme unbalanced training set and noise interference make it challenging to accurately capture the distribution of normal data and then detect anomalies. Existing AutoEncoder(AE)-based approaches are lack of effective regularization method specially designed for anomaly detection tasks thus easily overfitting while Generative Adversarial Network(GAN)-based approaches are mostly trained under the hypothesis of pollution-free training set, which means the training set is all composed of normal samples and that is hard to satisfy in practice. To tackle these problems, in this paper we propose a GAN based anomaly detection method for multivariate time series named FGANomaly (letter F is for Filter). The core idea is to filter possible anomalous samples with pseudo-labels before training the discriminator thus to capture the distribution of normal data as precise as possible. In addition, we design a novel training objective for the generator, which leads the generator to concentrate more on plausible normal data and ignore anomalies. We conducted comprehensive experiments on four public datasets, and the experimental results show the superiority of our method over baselines in both performance and robustness.

Traffic flow prediction is a fundamental problem in spatiotemporal data mining. Most of the existing studies focuses on designing statistical models to fit historical traffic data, which are purely data-driven approaches and fail to reveal the underlying mechanisms of urban traffic. To address this issue, we propose the spatiotemporal potential energy field model (ST-PEF+), which applies the field theory for human mobility to interpret the underlying mechanisms of urban traffic, and introduces the theory into data-driven deep learning models. ST-PEF+ consists of a PEF extraction module and a data-driven module. Inspired by the field theory for human mobility, the PEF extraction module adopts an algorithm to decompose the grid-based traffic flow graph into several polytree-based potential energy fields (PEFs), where traffic flows from high potential locations to low potential locations, just as water is driven by the gravity field. We also provide a theoretical analysis to ensure that the polytree decomposition algorithm can decompose any traffic flow graph. In the data-driven module, ST-PEF+ learns a spatiotemporal deep learning model to predict the dynamics of PEFs. The model adopts correlation-adaptive neural network structures, which consists of a temporal component for temporal correlations and a spatial component for spatial correlations. The temporal component employs a GRU and DCN combined structure to capture both short-term autocorrelation and long-term repeating patterns of PEFs. The spatial component extends the GAT using weighted directed attention to model the asymmetric spatial structure in PEFs. The prediction results of traffic flow are finally derived from PEFs that are predicted by the spatiotemporal deep learning model. We conduct extensive evaluations on three real-world traffic datasets. The results show that our model outperforms the state-of-the-art baselines. In addition, case studies confirm that the PEFs learned in our framework can reveal the underlying mechanisms of urban traffic, thus improving the model interpretability.

Representation learning on heterogeneous graphs aims to obtain meaningful node representations to facilitate various downstream tasks. Existing heterogeneous graph learning methods are primarily developed by following the propagation mechanism of node representations. There are few efforts on studying the role of relations for improving the learning of more fine-grained node representations. Indeed, it is important to collaboratively learn the semantic representations of relations and discern node representations with respect to different relation types. In this paper, we propose a novel Relation-aware Heterogeneous Graph Neural Network (R-HGNN), to learn node representations on heterogeneous graphs at a fine-grained level by considering relation-aware characteristics. Specifically, a dedicated graph convolution component is first designed to learn unique node representations from each relation-specific graph separately. Then, a cross-relation message passing module is developed to improve the interactions of node representations across different relations. Also, the relation representations are learned in a layer-wise manner to capture relation semantics, which are used to guide the node representation learning process. Moreover, a semantic fusing module is presented to aggregate relation-aware node representations into a compact representation with the learned relation representations. Experimental results on extensive graph learning tasks demonstrate that our approach could consistently outperform existing methods.

Graph Neural Networks (GNNs) have been widely applied in the semi-supervised node classification task, where a key point lies in how to sufficiently leverage the limited but valuable label information. Most of the classical GNNs solely use the known labels for computing the classification loss at the output. In recent years, several methods have been designed to additionally utilize the labels at the input. One part of the methods augment the node features via concatenating or adding them with the one-hot encodings of labels, while other methods optimize the graph structure by assuming neighboring nodes tend to have the same label. To bring into full play the rich information of labels, in this article we present a label-enhanced learning framework for GNNs, which first models each label as a virtual center for intra-class nodes and then jointly learns the representations of both nodes and labels. Our approach could not only smooth the representations of nodes belonging to the same class, but also explicitly encode the label semantics into the learning process of GNNs. Moreover, a training node selection technique is provided to eliminate the potential label leakage issue and guarantee the model generalization ability. Finally, an adaptive self-training strategy is proposed to iteratively enlarge the training set with more reliable pseudo labels and distinguish the importance of each pseudo-labeled node during the model training process. Experimental results on both real-world and synthetic datasets demonstrate our approach can not only consistently outperform the state-of-the-arts, but also effectively smooth the representations of intra-class nodes.

To improve recommendation performance, many efforts have been made to study how to equip the conventional methods with auxiliary information such as item relations. Meanwhile, a growing body of work has focused on applying graph convolutional networks to recommendation tasks. Thus, it is promising to use graph convolution to model multi-order relations for improving recommendation performance. However, the existing graph convolution-based recommendation methods may suffer from structural design problems: for methods with embedding transformations in graph convolutional layers, the MLP makes the updated embedding dimensions coupled, hurting the embedding expressivity. While for methods based on simplified graph convolution, removing the parameter matrices makes the model attach a same weight to embeddings in different layers, limiting the model expressivity. In this paper, we propose a novel graph convolution-based recommendation method, namely Channel-Independent Graph Convolutional Network (CIGCN). To learn disentangled embeddings, CIGCN uses diagonal parameter matrices as filters in graph convolution, keeping the updated embedding dimensions independent. In addition, with layer-aggregation strategies, the parameters in the diagonal matrices act as trainable weights that attach different importance to the embeddings in each layer and each dimension, enhancing the model expressivity. Results of extensive experiments show the superior performance of our proposed method.

In real-world scenarios, a user's interactions with items could be formalized as a behavior sequence, indicating his/her dynamic and evolutionary preferences. To this end, a series of recent efforts in recommender systems aim at improving recommendation performance by considering the sequential information. However, impacts of sequential behavior on future interactions may vary greatly in different scenarios. Additionally, semantic item relations underlying item attributes have not been well exploited in sequential recommendation models, which could be crucial for measuring item similarities in recommendation. To deal with the above problems, this paper provides a general embedding smoothing framework for sequential recommendation models. Specifically, we first construct a hybrid item graph by fusing sequential item relations derived from user-item interactions with semantic item relations built upon item attributes. Second, we perform graph convolutions on the hybrid item graph to generate smoothed item embedding. Finally, we equip sequential recommendation models with the smoothed item representations to enhance their performances. Experimental results demonstrate that with our embedding smoothing framework, the state-of-the-art sequential recommendation model, SASRec, achieves superior performance to most baseline methods on three real-world datasets. Moreover, the results show that most mainstream sequential recommendation models could benefit from our framework.

Hierarchical text classification aims to leverage label hierarchy in multi-label text classification.Existing methods encode label hierarchy in a global view, where label hierarchy is treated as the static hierarchical structure containing all labels.Since global hierarchy is static and irrelevant to text samples, it makes these methods hard to exploit hierarchical information.Contrary to global hierarchy, local hierarchy as a structured labels hierarchy corresponding to each text sample.It is dynamic and relevant to text samples, which is ignored in previous methods.To exploit global and local hierarchies, we propose Hierarchy-guided BERT with Global and Local hierarchies (HBGL), which utilizes the large-scale parameters and prior language knowledge of BERT to model both global and local hierarchies.Moreover, HBGL avoids the intentional fusion of semantic and hierarchical modules by directly modeling semantic and hierarchical information with BERT.Compared with the state-of-the-art method HGCLR, our method achieves significant improvement on three benchmark datasets.

Graph generative models have broad applications in biology, chemistry and social science. However, modelling and understanding the generative process of graphs is challenging due to the discrete and high-dimensional nature of graphs, as well as permutation invariance to node orderings in underlying graph distributions. Current leading autoregressive models fail to capture the permutation invariance nature of graphs for the reliance on generation ordering and have high time complexity. Here, we propose a continuous-time generative diffusion process for permutation invariant graph generation to mitigate these issues. Specifically, we first construct a forward diffusion process defined by a stochastic differential equation (SDE), which smoothly converts graphs within the complex distribution to random graphs that follow a known edge probability. Solving the corresponding reverse-time SDE, graphs can be generated from newly sampled random graphs. To facilitate the reverse-time SDE, we newly design a position-enhanced graph score network, capturing the evolving structure and position information from perturbed graphs for permutation equivariant score estimation. Under the evaluation of comprehensive metrics, our proposed generative diffusion process achieves competitive performance in graph distribution learning. Experimental results also show that GraphGDP can generate high-quality graphs in only 24 function evaluations, much faster than previous autoregressive models.

Recent years have witnessed a rapid growth of applying deep spatiotemporal methods in traffic forecasting. However, the prediction of origin-destination (OD) demands is still a challenging problem since the number of OD pairs is usually quadratic to the number of stations. In this case, most of the existing spatiotemporal methods fail to handle spatial relations on such a large scale. To address this problem, this paper provides a dynamic graph representation learning framework for OD demands prediction. In particular, a hierarchical memory updater is first proposed to maintain a time-aware representation for each node, and the representations are updated according to the most recently observed OD trips in continuous-time and multiple discrete-time ways. Second, a spatiotemporal propagation mechanism is provided to aggregate representations of neighbor nodes along a random spatiotemporal route which treats origin and destination as two different semantic entities. Last, an objective function is designed to derive the future OD demands according to the most recent node representations, and also to tackle the data sparsity problem in OD prediction. Extensive experiments have been conducted on two real-world datasets, and the experimental results demonstrate the superiority of the proposed method. The code and data are available at https://github.com/Rising0321/HMOD.

Traffic demand forecasting by deep neural networks has attracted widespread interest in both academia and industry society. Among them, the pairwise Origin-Destination (OD) demand prediction is a valuable but challenging problem due to several factors: (i) the large number of possible OD pairs, (ii) implicitness of spatial dependence, and (iii) complexity of traffic states. To address the above issues, this paper proposes a Continuous-time and Multi-level dynamic graph representation learning method for Origin-Destination demand prediction (CMOD). Firstly, a continuous-time dynamic graph representation learning framework is constructed, which maintains a dynamic state vector for each traffic node (metro stations or taxi zones). The state vectors keep historical transaction information and are continuously updated according to the most recently happened transactions. Secondly, a multi-level structure learning module is proposed to model the spatial dependency of station-level nodes. It can not only exploit relations between nodes adaptively from data, but also share messages and representations via cluster-level and area-level virtual nodes. Lastly, a cross-level fusion module is designed to integrate multi-level memories and generate comprehensive node representations for the final prediction. Extensive experiments are conducted on two real-world datasets from Beijing Subway and New York Taxi, and the results demonstrate the superiority of our model against the state-of-the-art approaches.

Recent studies have shown great promise in applying graph neural networks for multivariate time series forecasting, where the interactions of time series are described as a graph structure and the variables are represented as the graph nodes. Along this line, existing methods usually assume that the graph structure (or the adjacency matrix), which determines the aggregation manner of graph neural network, is fixed either by definition or self-learning. However, the interactions of variables can be dynamic and evolutionary in real-world scenarios. Furthermore, the interactions of time series are quite different if they are observed at different time scales. To equip the graph neural network with a flexible and practical graph structure, in this paper, we investigate how to model the evolutionary and multi-scale interactions of time series. In particular, we first provide a hierarchical graph structure cooperated with the dilated convolution to capture the scale-specific correlations among time series. Then, a series of adjacency matrices are constructed under a recurrent manner to represent the evolving correlations at each layer. Moreover, a unified neural network is provided to integrate the components above to get the final prediction. In this way, we can capture the pair-wise correlations and temporal dependency simultaneously. Finally, experiments on both single-step and multi-step forecasting tasks demonstrate the superiority of our method over the state-of-the-art approaches.

Given a sequence of sets with timestamps, where each set includes an arbitrary number of elements, temporal sets prediction aims to predict elements in the consecutive set. Indeed, predicting temporal sets is much more complicated than the conventional predictions of time series and temporal events. Recent studies on temporal sets prediction follow the same pipeline that only learns from each user’s own sequence, which fails to discover the collaborative signals among the sequences of different users. In this paper, we propose a novel element-guided temporal graph neural network to tackle the above issue in temporal sets prediction. Specifically, we first connect sequences of different users via a temporal graph, where nodes contain users and elements, and edges represent user-element interactions with time information. Then, we devise a new message aggregation mechanism to improve the model expressive ability via adaptively learning element-specific representations for each user with the guidance of elements. By performing the element-guided message aggregation among multiple hops, collaborative signals latent in high-order user-element interactions are explicitly encoded. Finally, we present a temporal information utilization module to capture both the semantic and periodic patterns in user sequential behaviors. Experiments on real-world datasets demonstrate that our approach could not only outperform the existing methods with a significant margin but also capture the collaborative signals. Codes and datasets are available at https://github.com/yule-BUAA/ETGNN.

Machine learning models have recently demonstrated the ability to mine structural health monitoring data. While existing machine learning models can provide better performance than the previous univariate time-series model, it is still an open challenge of fully mining the spatial and temporal characteristics of the structural response to increase their accuracy. In addition, the heterogeneous correlation is always missed in traditional models. This article proposes a heterogeneous structural response prediction (HSRP) framework based on the deep learning model to improve the performance. The HSRP framework cannot only make full use of spatial and temporal correlations but also mine the correlation between heterogeneous responses. Motivated by recent studies in machine learning, an attention module is introduced to learn the correlation between different responses and initial the weights of sensors and past response. The convolutional neural network is also implemented to extract the spatial features and the long short-term memory network is used to extract the weekly, daily, and hourly patterns of structural response. A real-world data set collected from a bridge is used to evaluate the performance of the proposed model on single-step prediction and multistep prediction. The experimental results show that the proposed model outperforms several widely used benchmark models. Furthermore, additional experiments and evaluations are implemented to investigate the sensitivity and robustness of the proposed model.

Recent years have witnessed the emerging success of Graph Neural Networks (GNNs) for modeling graphical data. A GNN can model the spatial dependencies of nodes in a graph based on message passing through node aggregation. However, in many application scenarios, these spatial dependencies can change over time, and a basic GNN model cannot capture these changes. In this article, we propose a G raph S eq uence neural network with an A tt ention mechanism (GSeqAtt) for processing graph sequences. More specifically, two attention mechanisms are combined: a horizontal mechanism and a vertical mechanism. GTransformer, which is a horizontal attention mechanism for handling time series, is used to capture the correlations between graphs in the input time sequence. The vertical attention mechanism, a Graph Network (GN) block structure with an attention mechanism (GNAtt), acts within the graph structure in each frame of the time series. Experiments show that our proposed model is able to handle information propagation for graph sequences accurately and efficiently. Moreover, results on real-world data from three road intersections show that our GSeqAtt outperforms state-of-the-art baselines on the traffic speed prediction task.

Travel time estimation of a given path is a crucial task of Intelligent Transportation Systems (ITS). Accurate travel time estimation can benefit multiple downstream applications such as route planning, real-time navigation, and urban construction. However, it is a challenging problem since the travel time is largely affected by multiple complicated factors including spatial factors, temporal factors and external factors, and obtaining informative representations of a given path is not trivial. Most previous works solved this problem in either Euclidean space or non-Euclidean space, which was unilateral to represent the actual traveling path and led to relatively poor performance. To address this, this paper proposes a multi-semantic path representation method to exploit information in Euclidean space and non-Euclidean space simultaneously. First, since the path is composed of several segments, we generate semantic representations of segments in non-Euclidean space by taking both the time information and the historical co-occurrence into consideration. Second, as the path could be equally represented as several travelled intersections, semantic representations of intersection sequences are also extracted to improve the capability of the method by considering information in Euclidean space. Meanwhile, semantic representations from properties, including the length and the type of segments, are also incorporated into the model. Finally, a sequence learning component is added on the top to aggregate the information along the entire path and provides the final estimation. Extensive experiments were conducted on two real-world taxi trajectories datasets, and the experimental results demonstrate the superiority of the proposed method.

Accurate and real-time taxi demand prediction can help managers pre-allocate taxi resources in cities, which assists drivers quickly finding passengers and reduce passengers’ waiting time. Most of the existing studies focus on mining spatial-temporal characteristics of taxi demand distributions, while lacking in modeling the correlations between taxi pick-up demand and the drop-off demand from the perspective of multi-task learning. In this article, we propose a multi-task learning model containing three parallel LSTM layers to co-predict taxi pick-up and drop-off demands, and compare the performance of single demand prediction methodology and that of two demands’ co-prediction methodology. Experimental results on real-world datasets demonstrate that the pick-up demand and the drop-off demand do depend on each other, and the effectiveness of the proposed co-prediction methods.

Predicting treatment medicines is a key task in many intelligent healthcare systems. Prediction of treatment medicines can assist doctors in making informed prescription decisions for patients according to their Electronic Health Records (EHRs). However, predicting treatment medicines is a challenging task due to the following reasons: (1) heterogeneous nature of EHR data that typically includes laboratory results, treatment records, disease conditions, and demographic information; (2) complex correlations among EHR sequences, including inter-correlations between sequences and temporal intra-correlations within each sequence; (3) temporal dynamics of these correlations changing with disease progression. In this paper, we predict treatment medicines for patients with dual adaptive sequential networks (DASNet). Specifically, DASNet is designed with three components. First, a decomposed adaptive long short-term memory network (DA-LSTM) is designed to capture the intra- and inter-correlations in multiple heterogeneous temporal sequences. Then, we develop an attentive meta learning network (AT-MetaNet) to learn dynamic weight parameters for DA-LSTM, thus enabling it to model various correlation structures. Finally, we employ an attentive fusion network (AT-FuNet) to incorporate historical information and collectively fuse representation embeddings of heterogeneous data to predict treatment medicines. Our results on the public MIMIC-III dataset covering 11 medical conditions demonstrate that the proposed end-to-end model can achieve the state-of-the-art prediction performance while providing clinically useful insights.

Extreme multi-label text classification(XMC) is a task for finding the most relevant labels from a large label set. Nowadays deep learning-based methods have shown significant success in XMC. However, the existing methods (e.g., AttentionXML and X-Transformer etc) still suffer from 1) combining several models to train and predict for one dataset, and 2) sampling negative labels statically during the process of training label ranking model, which will harm the performance and accuracy of model. To address the above problems, we propose LightXML, which adopts end-to-end training and dynamical negative labels sampling. In LightXML, we use GAN like networks to recall and rank labels. The label recalling part will generate negative and positive labels, and the label ranking part will distinguish positive labels from these labels. Based on these networks, negative labels are sampled dynamically during label ranking part training. With feeding both label recalling and ranking parts with the same text representation, LightXML can reach high performance. Extensive experiments show that LightXML outperforms state-of-the-art methods in five extreme multi-label datasets with much smaller model size and lower computational complexity. In particular, on the Amazon dataset with 670K labels, LightXML can reduce the model size up to 72% compared to AttentionXML. Our code is available at http://github.com/kongds/LightXML.

Graph Convolutional Network (GCN) has been widely applied in transportation demand prediction due to its excellent ability to capture non-Euclidean spatial dependence among station-level or regional transportation demands. However, in most of the existing research, the graph convolution was implemented on a heuristically generated adjacency matrix, which could neither reflect the real spatial relationships of stations accurately, nor capture the multi-level spatial dependence of demands adaptively. To cope with the above problems, this paper provides a novel graph convolutional network for transportation demand prediction. Firstly, a novel graph convolution architecture is proposed, which has different adjacency matrices in different layers and all the adjacency matrices are self-learned during the training process. Secondly, a layer-wise coupling mechanism is provided, which associates the upper-level adjacency matrix with the lower-level one. It also reduces the scale of parameters in our model. Lastly, a unitary network is constructed to give the final prediction result by integrating the hidden spatial states with gated recurrent unit, which could capture the multi-level spatial dependence and temporal dynamics simultaneously. Experiments have been conducted on two real-world datasets, NYC Citi Bike and NYC Taxi, and the results demonstrate the superiority of our model over the state-of-the-art ones.

Transportation demand prediction is a classic problem in intelligent transportation research. However, most exist studies have been focused on improving the prediction accuracy in a single demand mode, and there is a lack of understanding of the impact of multiple transportation modes. To this paper, we aim to uncover the interactions of multiple transportation modes and develop a co-prediction method for multimodal transportation demand prediction. Specifically, we first propose a self-learned spatial graph construction method, which automatically learns spatial dependencies of both homogeneous and heterogeneous transportation stations, and then constructs a mode-free spatial dependence graph of the studied transportation stations. Then, a spatiotemporal convolution module is provided to update the state of each station spatially and temporally according to its neighbor stations on the self-learned spatial graph. Moreover, we design an output layer to map the hidden state of each station to the demands of multimodal transportation stations. Finally, experimental results on real-world data have not only validated the effectiveness of the proposed method, but also revealed that co-prediction of multimodal transportation demands could always result in higher prediction performances than single-mode prediction methods as it takes the interactions of multiple transportation modes into account.

Urban planning designs land-use configurations and can benefit building livable, sustainable, safe communities. Inspired by image generation, deep urban planning aims to leverage deep learning to generate land-use configurations. However, urban planning is a complex process. Existing studies usually ignore the need of personalized human guidance in planning, and spatial hierarchical structure in planning generation. Moreover, the lack of large-scale land-use configuration samples poses a data sparsity challenge. This paper studies a novel deep human guided urban planning method to jointly solve the above challenges. Specifically, we formulate the problem into a deep conditional variational autoencoder based framework. In this framework, we exploit the deep encoder-decoder design to generate land-use configurations. To capture the spatial hierarchy structure of land uses, we enforce the decoder to generate both the coarse-grained layer of functional zones, and the fine-grained layer of POI distributions. To integrate human guidance, we allow humans to describe what they need as texts and use these texts as a model condition input. To mitigate training data sparsity and improve model robustness, we introduce a variational Gaussian embedding mechanism. It not just allows us to better approximate the embedding space distribution of training data and sample a larger population to overcome sparsity, but also adds more probabilistic randomness into the urban planning generation to improve embedding diversity so as to improve robustness. Finally, we present extensive experiments to validate the enhanced performances of our method.

Dynamic Graph Neural Networks (DGNNs) have become one of the most promising methods for traffic speed forecasting. However, when adapting DGNNs for traffic speed forecasting, existing approaches are usually built on a static adjacency matrix (no matter predefined or self-learned) to learn spatial relationships among different road segments, even if the impact of two road segments can be changeable dynamically during a day. Moreover, the future traffic speed cannot only be related with the current traffic speed, but also be affected by other factors such as traffic volumes. To this end, in this paper, we aim to explore these dynamic and multi-faceted spatio-temporal characteristics inherent in traffic data for further unleashing the power of DGNNs for better traffic speed forecasting. Specifically, we design a dynamic graph construction method to learn the time-specific spatial dependencies of road segments. Then, a dynamic graph convolution module is proposed to aggregate hidden states of neighbor nodes to focal nodes by message passing on the dynamic adjacency matrices. Moreover, a multi-faceted fusion module is provided to incorporate the auxiliary hidden states learned from traffic volumes with the primary hidden states learned from traffic speeds. Finally, experimental results on real-world data demonstrate that our method can not only achieve the state-of-the-art prediction performances, but also obtain the explicit and interpretable dynamic spatial relationships of road segments.

In knowledge graphs, there are usually different types of nodes, multiple heterogeneous relations, and numerous attributes of nodes and edges, which impose the challenges on the task of Node Importance Estimation (NIE). Indeed, existing NIE approaches, such as PageRank (PR) and Node-Degree (ND), are not designed for handling knowledge graphs with the rich information related with these multifarious nodes and edges. To this end, in this paper, we propose a representation learning framework to leverage the rich information inherent in these multifarious nodes and edges for improving node importance estimation in knowledge graphs. Specifically, we provide a Relational Graph Transformer Network (RGTN), where a relational graph transformer is first proposed to propagate node information with the consideration of semantic predicate representations. Here, the assumption is that different predicates may have distinct effects on the transmission of node importance. Then, two separate encoders are designed to capture both the structural and semantic information of nodes respectively, and a co-attention module is developed to fuse the two separate representations of nodes. Next, an attention-based aggregation module is adopted to map the representations of nodes to their importance values. In addition, a learning-to-rank loss is designed to ensure that the learned representations can be aware of the relative ranking information among nodes. Finally, extensive experiments have been conducted on real-world knowledge graphs, and the results illustrate that our model outperforms the existing methods consistently for all the evaluation metrics. The code and the data are available at https://github.com/GRAPH-0/RGTN-NIE.

Precise traffic demand prediction could help government and enterprises make better management and operation decisions by providing them with data-driven insights. However, it is a nontrivial effort to design an effective traffic demand prediction method due to the spatial and temporal characteristics of traffic demand distributions, dynamics of human mobility, and impacts of multiple environmental factors. To handle these problems, a Dynamic Transition Convolutional Neural Network (DTCNN) is proposed for the purpose of precise traffic demand prediction. Particularly, a transition network is first constructed according to the citiwide historical departure and arrival records, where the nodes are virtual stations discovered by a density-peak based clustering algorithm and the edges of two nodes correspond to transition flows of two stations. Then, a dynamic transition convolution unit is designed to model the spatial distributions of the traffic demands, and to capture the evolution of the demand dynamics. Last, a unifying learning framework is provided to incorporate the spatiotemporal states of the traffic demands with environmental factors. Experiments have been conducted on NYC taxi and bike-sharing data, and the results validate the effectiveness of the proposed method.

As a data-driven healthcare service, automatic treatment regimen design has great potential to improve healthcare efficiency and quality. However, it is a nontrivial endeavor to develop such a healthcare service due to two major challenges: 1) the treatment records are complex data objects consisting of various semantic and temporal information, and 2) the treatment outcome usually depends on a large number of internal and external factors. Because of these difficulties, automatic treatment regimen design is still an open research problem nowadays. To fill this research gap, this paper first formulates a treatment sequence as temporal sets, then provides a novel Extended Jaccard Similarity (EJS) measure to quantify the similarities between treatment sequences. We show that the proposed EJS is a general and effective measure to capture the similarity between two complex temporal sets. Further, we develop an efficient clustering algorithm which can achieve reasonable clustering results with only a portion of the pairwise similarities between treatment sequences and then extract a semantic prototype of the treatment regimen from each cluster of treatment sequences. Finally, we adopt a matrix factorization framework to predict the treatment outcomes by integrating multiple internal and external factors. We conduct comprehensive experiments on Electronic Medical Records (EMRs) of more than 28,000 patients from 14 hospitals. The results demonstrate the effectiveness of our approach and its superiority over the state-of-the-art ones.

While Water Treatment Networks (WTNs) are critical infrastructures for local communities and public health, WTNs are vulnerable to cyber attacks. Effective detection of attacks can defend WTNs against discharging contaminated water, denying access, destroying equipment, and causing public fear. While there are extensive studies in WTNs attack detection, they only exploit the data characteristics partially to detect cyber attacks. After preliminary exploring the sensing data of WTNs, we find that integrating spatio-temporal knowledge, representation learning, and detection algorithms can improve attack detection accuracy. To this end, we propose a structured anomaly detection framework to defend WTNs by modeling the spatiotemporal characteristics of cyber attacks in WTNs. In particular, we propose a spatio-temporal representation framework specially tailored to cyber attacks after separating the sensing data of WTNs into a sequence of time segments. This framework has two key components. The first component is a temporal embedding module to preserve temporal patterns within a time segment by projecting the time segment of a sensor into a temporal embedding vector. We then construct Spatio-Temporal Graphs (STGs), where a node is a sensor and an attribute is the temporal embedding vector of the sensor, to describe the state of the WTNs. The second component is a spatial embedding module, which learns the final fused embedding of the WTNs from STGs. In addition, we devise an improved one class-SVM model that utilizes a new designed pairwise kernel to detect cyber attacks. The devised pairwise kernel augments the distance between normal and attack patterns in the fused embedding space. Finally, we conducted extensive experimental evaluations with real-world data to demonstrate the effectiveness of our framework: it achieves an accuracy of 91.65%, with average improvement ratios of 82.78% and 22.96% with respect to F1 and AUC, compared with baseline methods.

Given a sequence of sets, where each set contains an arbitrary number of elements, the problem of temporal sets prediction aims to predict the elements in the subsequent set. In practice, temporal sets prediction is much more complex than predictive modelling of temporal events and time series, and is still an open problem. Many possible existing methods, if adapted for the problem of temporal sets prediction, usually follow a two-step strategy by first projecting temporal sets into latent representations and then learning a predictive model with the latent representations. The two-step approach often leads to information loss and unsatisfactory prediction performance. In this paper, we propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective of our approach is to learn element relationship by constructing set-level co-occurrence graph and then perform graph convolutions on the dynamic relationship graphs. Moreover, we design an attention-based module to adaptively learn the temporal dependency of elements and sets. Finally, we provide a gated updating mechanism to find the hidden shared patterns in different sequences and fuse both static and dynamic information to improve the prediction performance. Experiments on real-world data sets demonstrate that our approach can achieve competitive performances even with a portion of the training data and can outperform existing methods with a significant margin.

Many sequential behaviors such as purchasing items from time to time, selecting courses in different terms, collecting event logs periodically could be formalized as sequential sets of actions or elements, namely temporal sets. Predicting the subsequent set according to historical sequence of sets could help us make better producing, scheduling, or operating decisions. However, most of the existing methods were designed for predicting time series or temporal events, which could not be directly used for temporal sets prediction due to the difficulties of multi-level representations of items and sets, complex temporal dependencies of sets, and evolving dynamics of sequential behaviors. To address these issues, this paper provides a novel sets prediction method, called DSNTSP (Dual Sequential Network for Temporal Sets Prediction). Our model first learns both item-level representations and set-level representations of set sequences separately based on a transformer framework. Then, a co-transformer module is proposed to capture the multiple temporal dependencies of items and sets. Last, a gated neural module is designed to predict the subsequent set by fusing all the multi-level correlations and multiple temporal dependencies of items and sets. The experimental results on real-world data sets show that our methods lead to significant and consistent improvements as compared to other methods.

Summary Being part of public transportations, the bike sharing system plays an important role in solving the “last mile” problem. With more advantages than traditional bike sharing systems relying on fixed docks, dockless bike sharing systems are rapidly expanding, which generally operated under cloud‐based architectures. Emerging Things‐Edge‐Cloud (TEC) architectures are showing many advantages than cloud‐based architectures including greater potential in reducing risks of privacy leakage due to their decentralized computing manners. However, even in the TEC architecture, differential attacks, which aims to mine information of individuals in the aggregated data, still cause major threats. To solve this problem, this article proposes a novel encoding and decoding framework within the TEC architecture for time series predictions in bike sharing systems, with considerations of guaranteeing differential privacy. In particular, we first construct a dynamic autoencoder based on the Long Short Term Memory (LSTM) network, then the collected raw temporal data are encoded into a hidden state in the edge end. In the cloud end, the trained output weights are used to reconstruct the current time series and predict the next‐period time series according to received hidden states. This framework not only improves the computation efficiency of bike sharing system by leveraging computing power at the node end but also provides safer data publications that meet the differential privacy requirements. Experiments are conducted on real‐world datasets, results demonstrate the effectiveness of the proposed framework in providing data services with both high utility in sequence predictions and high safety level for users' privacy.

Taxi and sharing bike bring great convenience to urban transportation. A lot of efforts have been made to improve the efficiency of taxi service or bike sharing system by predicting the next-period pick-up or drop-off demand. Different from the existing research, this paper is motivated by the following two facts: 1) From a micro view, an observed spatial demand at any time slot could be decomposed as a combination of many hidden spatial demand bases; 2) From a macro view, the multiple transportation demands are strongly correlated with each other, both spatially and temporally. Definitely, the above two views have great potential to revolutionize the existing taxi or bike demand prediction methods. Along this line, this paper provides a novel Co-prediction method based on Spatio-Temporal neural Network, namely, CoST-Net. In particular, a deep convolutional neural network is constructed to decompose a spatial demand into a combination of hidden spatial demand bases. The combination weight vector is used as a representation of the decomposed spatial demand. Then, a heterogeneous Long Short-Term Memory (LSTM) is proposed to integrate the states of multiple transportation demands, and also model the dynamics of them mixedly. Last, the environmental features such as humidity and temperature are incorporated with the achieved overall hidden states to predict the multiple demands simultaneously. Experiments have been conducted on real-world taxi and sharing bike demand data, results demonstrate the superiority of the proposed method over both classical and the state-of-the-art transportation demand prediction methods.

Multi-task learning is a successful machine learning framework which improves the performance of prediction models by leveraging knowledge among tasks, e.g., the relationships between different tasks. Most of existing multi-task learning methods focus on guiding learning process by predefined task relationships. In fact, these methods have not fully exploited the associated relationships during the learning process. On the one hand, replacing predefined task relationships by adaptively learned ones may result in higher prediction accuracy as it can avoid the risk of misguiding caused by improperly predefined relationships. On the other hand, apart from the task relationships, feature-task dependence and feature-feature interactions could also be employed to guide the learning process. Along this line, we propose aMultiple Relational Attention Network (MRAN) framework for multi-task learning, in which three types of relationships are considered. Correspondingly, MRAN consists of three attention-based relationship learning modules: 1) a task-task relationship learning module which captures the relationships among tasks automatically and controls the positive and negative knowledge transfer adaptively; 2) a feature-feature interaction learning module that handles the complicated interactions among features; 3) a task-feature dependence learning module, which can associate the related features with target tasks separately. To evaluate the effectiveness of the proposed MARN, experiments are conducted on two public datasets and a real-world dataset crawled from a review hosting site. Experimental results demonstrate the superiority of our method over both classical and the state-of-the-art multi-task learning methods.

Accurate metro ridership prediction can guide passengers in efficiently selecting their departure time and transferring from station to station. An increasing number of deep learning algorithms are being utilized to forecast metro ridership due to the development of computational intelligence. However, limited efforts have been exerted to consider spatiotemporal features, which are important in forecasting ridership through deep learning methods, in large-scale metro networks. To fill this gap, this paper proposes a parallel architecture comprising convolutional neural network (CNN) and bi-directional long short-term memory network (BLSTM) to extract spatial and temporal features, respectively. Metro ridership data are transformed into ridership images and time series. Spatial features can be learned from ridership image data by using CNN, which demonstrates favorable performance in video detection. Time series data are input into the BLSTM which considers the historical and future impacts of ridership in temporal feature extraction. The two networks are concatenated in parallel and prevented from interfering with each other. Joint spatiotemporal features are fed into a fully connected network for metro ridership prediction. The Beijing metro network is used to demonstrate the efficiency of the proposed algorithm. The proposed model outperforms traditional statistical models, deep learning architectures, and sequential structures, and is suitable for ridership prediction in large-scale metro networks. Metro authorities can thus effectively allocate limited resources to overcrowded areas for service improvement.

Recent years have witnessed an opportunity for improving healthcare efficiency and quality by mining Electronic Medical Records (EMRs). This paper is aimed at developing a treatment engine, which learns from historical EMR data and provides a patient with next-period prescriptions based on disease conditions, laboratory results, and treatment records of the patient. Importantly, the engine takes consideration of both treatment records and physical examination sequences which are not only heterogeneous and temporal in nature but also often with different record frequencies and lengths. Moreover, the engine also combines static information (e.g., demographics) with the temporal sequences to provide personalized treatment prescriptions to patients. In this regard, a novel Long Short-Term Memory (LSTM) learning framework is proposed to model inter-correlations of different types of medical sequences by connections between hidden neurons. With this framework, we develop three multifaceted LSTM models: Fully Connected Heterogeneous LSTM, Partially Connected Heterogeneous LSTM, and Decomposed Heterogeneous LSTM. The experiments are conducted on two datasets: one is the public MIMIC-III ICU data, and the other comes from several Chinese hospitals. Experimental results reveal the effectiveness of the framework and the three models. The work is deemed important and meaningful for both academia and practitioners in the realm of medical treatment and prediction, as well as in other fields of applications where intelligent decision support becomes pervasive.

The mobile in-App service analysis, aiming at classifying mobile internet traffic into different types of service usages, has become a challenging and emergent task for mobile service providers due to the increasing adoption of secure protocols for in-App services. While some efforts have been made for the classification of mobile internet traffic, existing methods rely on complex feature construction and large storage cache, which lead to low processing speed, and thus not practical for online real-time scenarios. To this end, we develop an iterative analyzer for classifying encrypted mobile traffic in a real-time way. Specifically, we first select an optimal set of most discriminative features from raw features extracted from traffic packet sequences by a novel Maximizing Inner activity similarity and Minimizing Different activity similarity (MIMD) measurement. To develop the online analyzer, we first represent a traffic flow with a series of time windows, which are described by the optimal feature vector and are updated iteratively at the packet level. Instead of extracting feature elements from a series of raw traffic packets, our feature elements are updated when a new traffic packet is observed and the storage of raw traffic packets is not required. The time windows generated from the same service usage activity are grouped by our proposed method, namely, recursive time continuity constrained KMeans clustering (rCKC). The feature vectors of cluster centers are then fed into a random forest classifier to identify corresponding service usages. Finally, we provide extensive experiments on real-world Internet traffic data from Wechat, Whatsapp, and Facebook to demonstrate the effectiveness and efficiency of our approach. The results show that the proposed analyzer provides high accuracy in real-world scenarios, and has low storage cache requirement as well as fast processing speed.

Bike sharing systems, aiming at providing the missing links in public transportation systems, are becoming popular in urban cities. Many providers of bike sharing systems are ready to expand their bike stations from the existing service area to surrounding regions. A key to success for a bike sharing systems expansion is the bike demand prediction for expansion areas. There are two major challenges in this demand prediction problem: First. the bike transition records are not available for the expansion area and second. station level bike demand have big variances across the urban city. Previous research efforts mainly focus on discovering global features, assuming the station bike demands react equally to the global features, which brings large prediction error when the urban area is large and highly diversified. To address these challenges, in this paper, we develop a hierarchical station bike demand predictor which analyzes bike demands from functional zone level to station level. Specifically, we first divide the studied bike stations into functional zones by a novel Bi-clustering algorithm which is designed to cluster bike stations with similar POI characteristics and close geographical distances together. Then, the hourly bike check-ins and check-outs of functional zones are predicted by integrating three influential factors: distance preference, zone-to-zone preference, and zone characteristics. The station demand is estimated by studying the demand distributions among the stations within the same functional zone. Finally, the extensive experimental results on the NYC Citi Bike system with two expansion stages show the advantages of our approach on station demand and balance prediction for bike sharing system expansions.

We present an approach for improving the performance of complex knowledge-based processes by providing data-driven step-by-step recommendations. Our framework uses the associations between similar historic process performances and contextual information to determine the prototypical way of enacting the process. We introduce a novel similarity metric for grouping traces into clusters that incorporates temporal information about activity performance and handles concurrent activities. Our data-driven recommender system selects the appropriate prototype performance of the process based on user-provided context attributes. Our approach for determining the prototypes discovers the commonly performed activities and their temporal relationships. We tested our system on data from three real-world medical processes and achieved recommendation accuracy up to an F1 score of 0.77 (compared to an F1 score of 0.37 using ZeroR) with 63.2% of recommended enactments being within the first five neighbors of the actual historic enactments in a set of 87 cases. Our framework works as an interactive visual analytic tool for process mining. This work shows the feasibility of data-driven decision support system for complex knowledge-based processes.

A Point of Interest (POI) refers to a specific location that people may find useful or interesting. While a large body of research has been focused on identifying and recommending POIs, there are few studies on characterizing the life cycle of POIs. Indeed, a comprehensive understanding of POI life cycle can be helpful for various tasks, such as urban planning, business site selection, and real estate evaluation. In this paper, we develop a framework, named POLIP, for characterizing the POI life cycle with multiple data sources. Specifically, to investigate the POI evolution process over time, we first formulate a serial classification problem to predict the life status of POIs. The prediction approach is designed to integrate two important perspectives: 1) the spatial-temporal dependencies associated with the prosperity of POIs, and 2) the human mobility dynamics hidden in the citywide taxicab data related to the POIs at multiple granularity levels. In addition, based on the predicted life statuses in successive time windows for a given POI, we design an algorithm to characterize its life cycle. Finally, we performed extensive experiments using large-scale and real-world datasets. The results demonstrate the feasibility in automatic characterizing POI life cycle and shed important light on future research directions.

Bike sharing systems, aiming at providing the missing links in public transportation systems, are becoming popular in urban cities. A key to success for a bike sharing systems is the effectiveness of rebalancing operations, that is, the efforts of restoring the number of bikes in each station to its target value by routing vehicles through pick-up and drop-off operations. There are two major issues for this bike rebalancing problem: the determination of station inventory target level and the large scale multiple capacitated vehicle routing optimization with outlier stations. The key challenges include demand prediction accuracy for inventory target level determination, and an effective optimizer for vehicle routing with hundreds of stations. To this end, in this paper, we develop a Meteorology Similarity Weighted K-Nearest-Neighbor (MSWK) regressor to predict the station pick-up demand based on large-scale historic trip records. Based on further analysis on the station network constructed by station-station connections and the trip duration, we propose an inter station bike transition (ISBT) model to predict the station drop-off demand. Then, we provide a mixed integer nonlinear programming (MINLP) formulation of multiple capacitated bike routing problem with the objective of minimizing total travel distance. To solve it, we propose an Adaptive Capacity Constrained K-centers Clustering (AdaCCKC) algorithm to separate outlier stations (the demands of these stations are very large and make the optimization infeasible) and group the rest stations into clusters within which one vehicle is scheduled to redistribute bikes between stations. In this way, the large scale multiple vehicle routing problem is reduced to inner cluster one vehicle routing problem with guaranteed feasible solutions. Finally, the extensive experimental results on the NYC Citi Bike system show the advantages of our approach for bike demand prediction and large-scale bike rebalancing optimization.

The analysis of large-scale Electrical Medical Records (EMRs) has the potential to develop and optimize clinical treatment regimens. A treatment regimen usually includes a series of doctor orders containing rich temporal and heterogeneous information. However, in many existing studies, a doctor order is simplified as an event code and a treatment record is simplified as a code sequence. Thus, the information inherent in doctor orders is not fully used for in-depth analysis. In this paper, we aim at exploiting the rich information in doctor orders and developing data-driven approaches for improving clinical treatments. To this end, we first propose a novel method to measure the similarities between treatment records with consideration of sequential and multifaceted information in doctor orders. Then, we propose an efficient density-based clustering algorithm to summarize large-scale treatment records, and extract a semantic representation of each treatment cluster. Finally, we develop a unified framework to evaluate the discovered treatment regimens, and find the most effective treatment regimen for new patients. In the empirical study, we validate our methods with EMRs of 27,678 patients from 14 hospitals. The results show that: 1) Our method can successfully extract typical treatment regimens from large-scale treatment records. The extracted treatment regimens are intuitive and provide managerial implications for treatment regimen design and optimization. 2) By recommending the most effective treatment regimens, the total cure rate in our data improves from 19.89% to 21.28%, and the effective rate increases up to 98.29%.

Affinity Propagation (AP) clustering has been successfully used in a lot of clustering problems. However, most of the applications deal with static data. This paper considers how to apply AP in incremental clustering problems. First, we point out the difficulties in Incremental Affinity Propagation (IAP) clustering, and then propose two strategies to solve them. Correspondingly, two IAP clustering algorithms are proposed. They are IAP clustering based on K-Medoids (IAPKM) and IAP clustering based on Nearest Neighbor Assignment (IAPNA). Five popular labeled data sets, real world time series and a video are used to test the performance of IAPKM and IAPNA. Traditional AP clustering is also implemented to provide benchmark performance. Experimental results show that IAPKM and IAPNA can achieve comparable clustering performance with traditional AP clustering on all the data sets. Meanwhile, the time cost is dramatically reduced in IAPKM and IAPNA. Both the effectiveness and the efficiency make IAPKM and IAPNA able to be well used in incremental clustering tasks.