Supply chains are integral to global economic stability, yet disruptions can swiftly propagate through interconnected networks, resulting in substantial economic impacts. Accurate and timely inference of supply chain resilience—the capability to maintain core functions during disruptions—is crucial for proactive risk mitigation and robust network design. However, existing approaches lack effective mechanisms to infer supply chain resilience without explicit system dynamics and struggle to represent the higher-order, multi-entity dependencies inherent in supply chain networks. These limitations motivate the definition of a novel problem and the development of targeted modeling solutions. To address these challenges, we formalize a novel problem: Supply Chain Resilience Inference (SCRI), defined as predicting supply chain resilience using hypergraph topology and observed inventory trajectories without explicit dynamic equations. To solve this problem, we propose the Supply Chain Resilience Inference Hypergraph Network (SC-RIHN), a novel hypergraph-based model leveraging set-based encoding and hypergraph message passing to capture multi-party firm-product interactions. Comprehensive experiments demonstrate that SC-RIHN significantly outperforms traditional MLP, representative graph neural network variants, and ResInf baselines across synthetic benchmarks, underscoring its potential for practical, early-warning risk assessment in complex supply chain systems.

Post-training quantization offers an efficient pathway to deploy super-resolution models, yet existing methods treat weight and activation quantization independently, missing their critical interplay. Through controlled experiments on SwinIR, we uncover a striking asymmetry: weight quantization primarily degrades structural similarity, while activation quantization disproportionately affects pixel-level accuracy. This stems from their distinct roles—weights encode learned restoration priors for textures and edges, whereas activations carry input-specific intensity information. Building on this insight, we propose HarmoQ, a unified framework that harmonizes quantization across components through three synergistic steps: structural residual calibration proactively adjusts weights to compensate for activation-induced detail loss, harmonized scale optimization analytically balances quantization difficulty via closed-form solutions, and adaptive boundary refinement iteratively maintains this balance during optimization. Experiments show HarmoQ achieves substantial gains under aggressive compression, outperforming prior art by 0.46 dB on Set5 at 2-bit while delivering 3.2× speedup and 4× memory reduction on A100 GPUs. This work provides the first systematic analysis of weight-activation coupling in super-resolution quantization and establishes a principled solution for efficient high-quality image restoration.

Disaster has a great impact on human mobility patterns. Clustering mobility patterns by generalizing group level characteristics from diverse individual trajectories plays a vital role in informing post-disaster recovery strategies. However, predefined clustering criteria or supervised labeling alone are insufficient to adequately capture the dynamics of mobility patterns during disaster events. Besides, the existing methods are not enough to capture the sensitive changes in mobility patterns in disaster events and handle the data of high-dimensional. This research investigates an embedded deep learning-based method which can automatically extract the groups' short-term feature of mobility patterns and achieves short-term mobility pattern clustering during disaster. The proposed method employs a Transformer-based temporal encoder to capture intra-day sequence patterns and integrates a VAE component with an embedded latent variable that directly encodes group-level mobility modes. We also design a compactness–separation loss that explicitly encourages within-mode feature compactness and between-mode feature separation. Based on massive mobile data, we conduct mobility pattern clustering on the case of 2011 Fukushima Earthquake. Compared to conventional clustering approaches, the proposed model structure is more discriminative and can capture the more sensitive changes between pre- and post- events. Compared to baselines with different loss functions, proposal methods can make more accurate fitting result and obtain more discrete clusters modes. Additionally, sensitivity analysis is conducted to examine the influence of key hyper parameters within the model. Based on the clustering outcomes, five representative mobility patterns are identified. We further analyze the spatial-temporal characteristics of mobility pattern changes during the disaster events and the recovery period of mobility pattern.

Similarity search is a fundamental research problem with broad applications in various research fields, including data mining, information retrieval, and machine learning. The core idea of similarity search is to find the most similar data sample of given query items, based on a specific similarity metric with the highest similarity score with all the search candidates in a large-scale database. It may suffer from a prohibitive computation cost and storage cost, which motivates us to design effective and fast similarity search algorithms in various scenarios. However, data missing is unavoidable in real-world scenarios, which results in a less accurate similarity score and further leads to an inaccurate similarity matrix. Therefore, obtaining an accurate similarity matrix is non-trivial when there are incomplete observations. To solve this problem, we propose a similarity matrix calibration method to estimate a high-quality similarity matrix and further provide a better similarity search performance. Firstly, we propose an objective function to minimize the difference between the initial inaccurate similarity matrix and the optimal estimated similarity matrix, where the inherent symmetric and Positive Semi-Definiteness (PSD) properties are utilized as the constraint to guide the calibration process. Then, we design an effective algorithm with high efficiency to provide a high-quality similarity matrix that approximates the ground-truth similarity matrix. Theoretical analysis demonstrates the efficiency guarantee of our proposed method, and extensive experimental results on real-world datasets verify the effectiveness and efficiency of the proposed method on the similarity matrix calibration task and the downstream similarity search task.

Trajectory anomaly detection is critical for transportation safety, solving traffic congestion, optimizing routes, and other smart city applications. Existing methods often misclassify reasonable but non-standard trajectories, such as route changes due to traffic congestion, as anomalies. Additionally, many methods require extensive labeled datasets for training, however, in real-world scenarios, trajectory anomaly labeled data are often scarce and costly to obtain. To address the challenges, we investigate a Semi-Supervised Contrastive learning approach for Trajectory Anomaly Detection, namely SSCTAD. Specifically, SSCTAD integrates a transformer based method to process trajectories and utilizes the in-out flow encoder to capture the spatial-temporal patterns from the trajectories, which can help the model capture the traffic conditions. Then, by leveraging a small subset of labeled data along with extensive unlabeled data, SSCTAD utilizes a semi-supervised contrastive learning component enables the model to better distinguish normal and anomalous trajectories by maximizing the similarity of latent representation from the same class and diversifying those from different classes. This method improves the performance of trajectory anomaly detection and effectively reduces the dependence of the model on the labeled dataset. Finally, experiments show that SSCTAD outperforms the other baseline models.

The rapid densification of 5G deployments and the emergence of 6G-era services demand accurate cellular traffic forecasting across heterogeneous communication modalities at fine spatiotemporal resolutions. Existing approaches primarily rely on numerical time-series analysis and often underutilize contextual information related to urban function, temporal routines, and external events. This paper presents MobileLLM, a dual-pathway framework that integrates numerical spatiotemporal signals with semantic contextual conditioning for multimodal cellular traffic prediction. The numerical pathway encodes historical traffic observations into node-level tokens, while the semantic pathway transforms structured contextual descriptions into text embeddings. These pathways are fused through a partially frozen GPT-2 backbone and a cross-modal decoder, enabling parameter-efficient adaptation while preserving useful pre-trained priors. Rather than using language models for raw numerical regression, our framework uses the semantic pathway to provide context-aware conditioning for numerical forecasting. Comprehensive evaluation on three real-world datasets-Milan, Trentino, and Shanghai-demonstrates consistent performance improvements across SMS, voice call, and Internet traffic prediction. MobileLLM achieves MAE improvements ranging from 0.88% to 32.62% over strong baselines, with particularly pronounced gains for Internet traffic where contextual dependence is strongest. These results show that combining numerical modeling with semantic contextual representations provides a practical and effective direction for next-generation cellular traffic forecasting.

The development of the Internet of Things (IoT) has enhanced smart city services for traffic monitoring, leading to numerous schemes for accurate flow prediction based on traffic sensors. However, existing approaches primarily capture spatio-temporal (ST) dependencies from traffic graphs and train their models using randomly ordered data. This overlooks the fact that the modeling difficulty of each sensor/node in the ST traffic graph can vary significantly due to its spatial dependencies and temporal trends, resulting in unreliable and unstable predictions in IoT scenarios. In this context, we argue that a well-designed curriculum with an easy-to-difficult order can improve the training of ST models. Therefore, this paper introduces an ST difficulty measurer to score the node-level difficulty of traffic graph from both spatial and temporal aspects, and then implements a curriculum in the ST model training process. More specifically, based on the tiered ST difficulty score, the ST model training begins with a subgraph consisting of “easy” nodes characterized by relatively consistent spatial relationships and regular temporal patterns. Gradually, more difficult nodes are incorporated into the subgraph and participate in subsequent training stages. Comprehensive experiments and analysis on two real-world traffic flow datasets confirm the effectiveness of our proposed approach.

Generalizable Image Super-Resolution aims to enhance model generalization capabilities under unknown degradations. To achieve this goal, the models are expected to focus only on image content-related features instead of overfitting degradations. Recently, numerous approaches such as Dropout and Feature Alignment have been proposed to suppress models' natural tendency to overfit degradations and yield promising results. Nevertheless, these works have assumed that models overfit to all degradation types (e.g., blur, noise, JPEG), while through careful investigations in this paper, we discover that models predominantly overfit to noise, largely attributable to its distinct degradation pattern compared to other degradation types. In this paper, we propose a targeted feature denoising framework, comprising noise detection and denoising modules. Our approach presents a general solution that can be seamlessly integrated with existing super-resolution models without requiring architectural modifications. Our framework demonstrates superior performance compared to previous regularization-based methods across five traditional benchmarks and datasets, encompassing both synthetic and real-world scenarios.

Real-world data distributions often shift continuously across multiple latent factors such as time, geography, and socioeconomic contexts. However, existing domain generalization approaches typically treat domains as discrete or as evolving along a single axis (e.g., time). This oversimplification fails to capture the complex, multidimensional nature of real-world variation. This paper introduces the task of Continuous Domain Generalization (CDG), which aims to generalize predictive models to unseen domains defined by arbitrary combinations of continuous variations. We present a principled framework grounded in geometric and algebraic theories, showing that optimal model parameters across domains lie on a low-dimensional manifold. To model this structure, we propose a Neural Lie Transport Operator (NeuralLio), which enables structure-preserving parameter transitions by enforcing geometric continuity and algebraic consistency. To handle noisy or incomplete domain variation descriptors, we introduce a gating mechanism to suppress irrelevant dimensions and a local chart-based strategy for robust generalization. Extensive experiments on synthetic and real-world datasets, including remote sensing, scientific documents, and traffic forecasting, demonstrate that our method significantly outperforms existing baselines in both generalization accuracy and robustness.

Abstract Trajectories serve as a cornerstone of intelligent transportation systems, playing an important role in many applications such as traffic flow prediction, route planning, and urban management. However, the availability of such data is limited due to privacy issues, ethical concerns, and the high cost associated with infrastructure deployment. In recent years, rapidly developing generative models such as Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and Diffusion Models (DMs) have demonstrated strong capabilities in learning complex data distributions and generating synthetic data, thereby alleviating the data accessibility issue. In this survey, we systematically review the existing literature on deep generative models that address the problem of trajectory generation. First, we classify the existing literature into two categories: unconditional and conditional trajectory generation. In unconditional generation, trajectories are generated without contextual constraints, whereas conditional generation incorporates several important factors such as road network topology, time of day, and user preferences to guide the trajectory generation process. Then, for each category, we further classify the literature into three methodological types, including VAEs, GANs, and DMs, and analyze how these models address key challenges under different settings. Finally, we discuss promising directions for future research and hope to inspire further advances in trajectory generation.

Road travel time estimation (RTTE) is crucial for route planning, especially in urban connected vehicle environments. However, current methods struggle to extract the latent traffic semantics embedded in trajectory data, limiting their ability to provide fine-grained travel time predictions for individual road segments compared to overall routes. In this study, we introduce DeepTrafficSense, a comprehensive RTTE framework that takes into account the heterogeneity and temporal variability of road segment conditions. It integrates a heterogeneous dynamic graph (HDG) and a trajectory motion transformer (TMT) to model macro traffic dynamics and micro vehicle motion patterns. The HDG captures the temporal evolution of traffic states across road segments using asynchronous memory updates and adaptive feature aggregation, effectively addressing challenges such as irregular trajectory sampling and stale memory states. The TMT, designed for variable-length multivariate sequences, employs self- and cross-attention mechanisms to fuse endogenous (e.g., speed) and exogenous (e.g., acceleration) motion features. Additionally, a type-aware gating mechanism dynamically adjusts the contributions of the HDG and TMT, enhancing prediction accuracy for both intersections and regular road segments. Experiments on large-scale datasets from Chengdu and Xi’an demonstrate that DeepTrafficSense significantly outperforms state-of-the-art models in RTTE tasks.

Friend recommendation is an important real-world application in Location-based Social Networks (LBSN), helping users discover potential friends and enhance their overall happiness. LBSN mainly comprises two distinct data structures: spatio-temporal data for human mobility and graph data for social networks. These two data structures make it challenging to model the complex relationships between them, which are essential for comprehensively understanding users’ lives. Previous studies have either modeled user trajectories and social networks separately or used classical simple graph-based methods, where a simple edge links only two nodes, failing to capture the multiple relationships inherent in LBSN. Furthermore, most studies have relied on Euclidean space to train their graph models, which could result in significant distortion because of tree-like social network data structure. To address these limitations, we propose a novel heterogeneous LBSN hypergraph that represents user check-in records and continuous trajectories—comprising multiple Points of Interest (POI)—as hyperedges, enabling the representation of complex spatio-temporal relationships. This approach enables us to link multiple nodes of different types by hyperedges and use hyperbolic spaces to create more efficient graph representations. Additionally, we devise a new type-specific attention mechanism for our Heterogeneous Hyperbolic Hypergraph Neural Network (H 3 GNN), which is end-to-end trainable and employs supervised contrastive learning to learn hypergraph node embeddings for the subsequent friend recommendation task with the help of hyperbolic space. Finally, our model H 3 GNN achieves better results than existing methods on six real-world city datasets, and our ablation studies demonstrate the effectiveness of each component. Additionally, our experiments indicate that H 3 GNN requires less data storage and training time compared to previous methods.

Analyzing the causal impact of various government-related policies on the epidemic spread is of critical importance. This article aims to investigate the problem of assessing the causal effects of different COVID-19-related policies on the USA epidemic spread in different counties at any given time period, while eliminating biased interference from unobserved confounders (e.g., the vigilance of residents). However, the infection outcome of each region is influenced not only by its own confounding factors but also by policy interventions implemented in neighboring regions. Furthermore, the government policy index may exhibit a time-delay influence on outbreak dynamics. To this end, we implement observational data about different COVID-19-related policies (treatment) and outbreak dynamics (outcome) across different U.S. counties over time and develop a causal framework that learns the representations of time-varying confounders to tackle the aforementioned issues. More specifically, we employ one recurrent structure to capture the accumulative effects stemming from the policy history and then utilize hypergraph neural network to model the interactions among spatial regions. Our experimental results demonstrate the effectiveness of the proposed framework in quantifying the causal impact of different policy types on epidemics. Compared with baseline methods, our assessment provides valuable insights for future policy-making endeavors.

Maritime transportation, vital for nearly 90% of global trade, necessitates precise vessel trajectory prediction for safety and efficiency. Although the Automatic Identification System (AIS) provides a comprehensive data source, how to model these multi-modal and heterogeneous time-varying sequences (such as vessels’ kinetic information and ocean weather factors) poses a formidable challenge. Moreover, most existing approaches are limited by the confined scope of vessel trajectory modeling, making it impossible to consider the unique characteristics of maritime transportation system. To tackle these challenges, we propose a novel framework called AISFuser to i) encode unique maritime traffic network into graphical representations, and ii) introduce the heterogeneity into multi-modal temporal embeddings through Self-Supervised Learning (SSL). Specifically, our AISFuser is constructed by combining an attention-based graph block with a transformer network to encode information across space and time, respectively. In terms of temporal dimension, one SSL auxiliary task is also designed to enhance the heterogeneity of temporal representations and supplement the main vessel prediction task. We validate the effectiveness of the proposed AISFuser on a real-world AIS dataset. Extensive experimental results demonstrate that our method can forecast multiple attributes of vessel trajectory for over 10 hours into the future, outperforming competitive baselines.

Spatiotemporal neural networks have shown great promise in urban scenarios by effectively capturing temporal and spatial correlations. However, urban environments are constantly evolving, and current model evaluations are often limited to traffic scenarios and use data mainly collected only a few weeks after training period to evaluate model performance. The generalization ability of these models remains largely unexplored. To address this, we propose a Spatiotemporal Out-of-Distribution (ST-OOD) benchmark, which comprises six urban scenario: bike-sharing, 311 services, pedestrian counts, traffic speed, traffic flow, ride-hailing demand, and bike-sharing, each with in-distribution (same year) and out-of-distribution (next years) settings. We extensively evaluate state-of-the-art spatiotemporal models and find that their performance degrades significantly in out-of-distribution settings, with most models performing even worse than a simple Multi-Layer Perceptron (MLP). Our findings suggest that current leading methods tend to over-rely on parameters to overfit training data, which may lead to good performance on in-distribution data but often results in poor generalization. We also investigated whether dropout could mitigate the negative effects of overfitting. Our results showed that a slight dropout rate could significantly improve generalization performance on most datasets, with minimal impact on in-distribution performance. However, balancing in-distribution and out-of-distribution performance remains a challenging problem. We hope that the proposed benchmark will encourage further research on this critical issue.

Training models on spatio-temporal (ST) data poses an open problem due to the complicated and diverse nature of the data itself, and it is challenging to ensure the model's performance directly trained on the original ST data. While limiting the variety of training data can make training easier, it can also lead to a lack of knowledge and information for the model, resulting in a decrease in performance. To address this challenge, we presented an innovative paradigm that incorporates three separate forms of curriculum learning specifically targeting from spatial, temporal, and quantile perspectives. Furthermore, our framework incorporates a stacking fusion module to combine diverse information from three types of curriculum learning, resulting in a strong and thorough learning process. We demonstrated the effectiveness of this framework with extensive empirical evaluations, highlighting its better performance in addressing complex ST challenges. We provided thorough ablation studies to investigate the effectiveness of our curriculum and to explain how it contributes to the improvement of learning efficiency on ST data.

In many intelligent transportation systems, predicting the future motion of heterogeneous traffic participants is a fundamental but challenging task due to various factors encompassing the agents’ dynamic states, interactions with neighboring agents and surrounding traffic infrastructures, and their stochastic and multi-modal natural behavior tendencies. However, existing approaches have limitations as they either focus solely on static, pairwise interactions, ignoring interactions of varied granularity, or fail to tackle agents’ heterogeneity. In this paper, instead of focusing solely on pairwise interactions, we propose a Heterogenous Hypergraph Graph Neural Network (HHGNN) based motion prediction model that leverages the nature of hypergraph to encode the groupwise interactions among traffic participants. Moreover, we propose the type-aware two-level hypergraph message passing module (TTHMS) with learnable hyperedge-type embeddings to model the intra-group and inter-group level interactions among heterogeneous traffic agents (e.g., vehicles, pedestrians, and cyclists). Besides, We integrate a scene context fusion layer in TTHMS to incorporate the scene context. Comparison and ablation experiments on the Waymo Open Motion Dataset (WOMD) demonstrate HHGNN’s effectiveness within the motion prediction task.

Spatiotemporal time series forecasting plays a key role in a wide range of real-world applications. While significant progress has been made in this area, fully capturing and leveraging spatiotemporal heterogeneity remains a fundamental challenge. Therefore, we propose a novel Heterogeneity-Informed Meta-Parameter Learning scheme. Specifically, our approach implicitly captures spatiotemporal heterogeneity through learning spatial and temporal embeddings, which can be viewed as a clustering process. Then, a novel spatiotemporal meta-parameter learning paradigm is proposed to learn spatiotemporal-specific parameters from meta-parameter pools, which is informed by the captured heterogeneity. Based on these ideas, we develop a Heterogeneity-Informed Spatiotemporal Meta-Network (HimNet) for spatiotemporal time series forecasting. Extensive experiments on five widely-used benchmarks demonstrate our method achieves state-of-the-art performance while exhibiting superior interpretability. Our code is available at https://github.com/XDZhelheim/HimNet.

Maritime traffic management increasingly relies on vessel position information provided by terrestrial and satellite networks of the Automatic Identification System (AIS). Unfortunately, the problem of missing AIS data can lead to long-term gaps in vessel trajectory, raising corresponding security concerns regarding collision risks and illicit activities. Existing imputation approaches are often constrained by vehicle-based low-sampling trajectories, hindering their ability to address unique characteristics of maritime transportation systems and long-term missing scenarios. To tackle these challenges, we propose a novel generative framework for long-term vessel trajectory imputation. Our framework considers irregular tracks of vessels, which differ from those of cars due to the absence of a structured road network, and ensures the continuity of multi-point imputed trajectories. Specifically, we first utilize a pre-trained trajectory embedding block to capture patterns of vessel movements. Subsequently, we introduce a diffusion-based model for generating missing trajectories, where observed trajectory modeling with transformer encoding architecture and embeddings of both historical vessel trajectory and external factors serve as conditional information. In particular, we design a physics-guided discriminator in the training stage, which imposes kinematic constraints between locations and angles to improve the continuity of the imputed trajectories. Comprehensive experiments and analysis on a real-world AIS dataset confirm the effectiveness of our proposed approach.

Temporal Domain Generalization (TDG) addresses the challenge of training predictive models under temporally varying data distributions. Traditional TDG approaches typically focus on domain data collected at fixed, discrete time intervals, which limits their capability to capture the inherent dynamics within continuous-evolving and irregularly-observed temporal domains. To overcome this, this work formalizes the concept of Continuous Temporal Domain Generalization (CTDG), where domain data are derived from continuous times and are collected at arbitrary times. CTDG tackles critical challenges including: 1) Characterizing the continuous dynamics of both data and models, 2) Learning complex high-dimensional nonlinear dynamics, and 3) Optimizing and controlling the generalization across continuous temporal domains. To address them, we propose a Koopman operator-driven continuous temporal domain generalization (Koodos) framework. We formulate the problem within a continuous dynamic system and leverage the Koopman theory to learn the underlying dynamics; the framework is further enhanced with a comprehensive optimization strategy equipped with analysis and control driven by prior knowledge of the dynamics patterns. Extensive experiments demonstrate the effectiveness and efficiency of our approach. The code can be found at: https://github.com/Zekun-Cai/Koodos.

Long-term time series forecasting (LTSF) represents a critical frontier in time series analysis, characterized by extensive input sequences, as opposed to the shorter spans typical of traditional approaches. While longer sequences inherently offer richer information for enhanced predictive precision, prevailing studies often respond by escalating model complexity. These intricate models can inflate into millions of parameters, resulting in prohibitive parameter scales. Our study demonstrates, through both theoretical and empirical evidence, that decomposition is key to containing excessive model inflation while achieving uniformly superior and robust results across various datasets. Remarkably, by tailoring decomposition to the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks, using over 99\% fewer parameters than the majority of competing methods. Through this work, we aim to unleash the power of a restricted set of parameters by capitalizing on domain characteristics—a timely reminder that in the realm of LTSF, bigger is not invariably better. The code is available at \url{https://anonymous.4open.science/r/SSCNN-321D/}.

Recent achievements in deep learning (DL) have demonstrated its potential in predicting traffic flows. Such predictions are beneficial for understanding the situation and making traffic control decisions. However, most state-of-the-art DL models are considered “black boxes” with little to no transparency of the underlying mechanisms for end users. Some previous studies attempted to “open the black box” and increase the interpretability of generated predictions. However, handling complex models on large-scale spatiotemporal data and discovering salient spatial and temporal patterns that significantly influence traffic flow remain challenging. To overcome these challenges, we present TrafPS , a visual analytics approach for interpreting traffic prediction outcomes to support decision-making in traffic management and urban planning. The measurements region SHAP and trajectory SHAP are proposed to quantify the impact of flow patterns on urban traffic at different levels. Based on the task requirements from domain experts, we employed an interactive visual interface for the multi-aspect exploration and analysis of significant flow patterns. Two real-world case studies demonstrate the effectiveness of TrafPS in identifying key routes and providing decision-making support for urban planning.

Tensor time series (TTS) data, a generalization of one-dimensional time series on a high-dimensional space, is ubiquitous in real-world applications. Compared to modeling time series or multivariate time series, which has received much attention and achieved tremendous progress in recent years, tensor time series has been paid less effort. However, properly coping with the TTS is a much more challenging task, due to its high-dimensional and complex inner structure. In this article, we start by revealing the structure of TTS data from afn statistical view of point. Then, in line with this analysis, we perform T ensor T ime S eries forecasting via a proposed Multi-way Norm alization ( TTS-Norm ), which effectively disentangles multiple heterogeneous low-dimensional substructures from the original high-dimensional structure. Finally, we design a novel objective function for TTS forecasting, accounting for the numerical heterogeneity among different low-dimensional subspaces of TTS. Extensive experiments on two real-world datasets verify the superior performance of our proposed model. 1

Multivariate time-series (MTS) forecasting is a paramount and fundamental problem in many real-world applications. The core issue in MTS forecasting is how to effectively model complex spatial-temporal patterns. In this paper, we develop a adaptive, interpretable and scalable forecasting framework, which seeks to individually model each component of the spatial-temporal patterns. We name this framework SCNN, as an acronym ofStructuredComponent-basedNeuralNetwork. SCNN works with a pre-defined generative process of MTS, which arithmetically characterizes the latent structure of the spatial-temporal patterns. In line with its reverse process, SCNN decouples MTS data into structured and heterogeneous components and then respectively extrapolates the evolution of these components, the dynamics of which are more traceable and predictable than the original MTS. Extensive experiments are conducted to demonstrate that SCNN can achieve superior performance over state-of-the-art models on three real-world datasets. Additionally, we examine SCNN with different configurations and perform in-depth analyses of the properties of SCNN.

Perceiving and modeling urban crowd movements are of great importance to smart city-related fields. Governments and public service operators can benefit from such efforts as they can be applied to crowd management, resource scheduling, and early emergency warning. However, most prior research on urban crowd modeling has failed to describe the dynamics and continuity of human mobility, leading to inconsistent and irrelevant results when they tackle multiple homogeneous forecasting tasks as they can only be modeled independently. To overcome this drawback, we propose to model human mobility from a new perspective, which uses the citywide crowd transition process constituted by a series of transition matrices from low order to high order, to help us understand how the crowd dynamics evolve step-by-step. We further propose a Deep Transition Process Network to process and predict such new high-dimensional data, where novel grid embedding with Graph Convolutional Network, parameter-shared Convolutional LSTM, and High-Dimensional Attention mechanism are designed to learn the complicated dependencies in terms of spatial, temporal, and ordinal features. We conduct experiments on two datasets generated by a large amount of GPS data collected from a real-world smartphone application. The experiment results demonstrate the superior performance of our proposed methodology over existing approaches.

Enhancing the energy efficiency of buildings significantly relies on monitoring indoor ambient temperature. The potential limitations of conventional temperature measurement techniques, together with the omnipresence of smartphones, have redirected researchers' attention towards the exploration of phone-based ambient temperature estimation methods. However, existing phone-based methods face challenges such as insufficient privacy protection, difficulty in adapting models to various phones, and hurdles in obtaining enough labeled training data. In this study, we propose a distributed phone-based ambient temperature estimation system which enables collaboration among multiple phones to accurately measure the ambient temperature in different areas of an indoor space. This system also provides an efficient, cost-effective approach with a few-shot meta-learning module and an automated label generation module. It shows that with just 5 new training data points, the temperature estimation model can adapt to a new phone and reach a good performance. Moreover, the system uses crowdsourcing to generate accurate labels for all newly collected training data, significantly reducing costs. Additionally, we highlight the potential of incorporating federated learning into our system to enhance privacy protection. We believe this study can advance the practical application of phone-based ambient temperature measurement, facilitating energy-saving efforts in buildings.

Monitoring the crowd in urban hot spot has been an important research topic in the field of urban management and has high social impact. It can allow more flexible allocation of public resources such as public transportation schedule adjustment and arrangement of police force. After 2020, because of the epidemic of COVID-19 virus, the public mobility pattern is deeply affected by the situation of epidemic as the physical close contact is the dominant way of infection. In this study, we propose a confirmed case-driven time-series prediction of crowd in urban hot spot named MobCovid. The model is a deviation of Informer, a popular time-serial prediction model proposed in 2021. The model takes both the number of nighttime staying people in downtown and confirmed cases of COVID-19 as input and predicts both the targets. In the current period of COVID, many areas and countries have relaxed the lockdown measures on public mobility. The outdoor travel of public is based on individual decision. Report of large amount of confirmed cases would restrict the public visitation of crowded downtown. But, still, government would publish some policies to try to intervene in the public mobility and control the spread of virus. For example, in Japan, there are no compulsory measures to force people to stay at home, but measures to persuade people to stay away from downtown area. Therefore, we also merge the encoding of policies on measures of mobility restriction made by government in the model to improve the precision. We use historical data of nighttime staying people in crowded downtown and confirmed cases of Tokyo and Osaka area as study case. Multiple times of comparison with other baselines including the original Informer model prove the effectiveness of our proposed method. We believe our work can make contribution to the current knowledge on forecasting the number of crowd in urban downtown during the Covid epidemic.

Traffic forecasting as a canonical task of multivariate time series forecasting has been a significant research topic in AI community. To address the spatio-temporal heterogeneity and non-stationarity implied in the traffic stream, in this study, we propose Spatio-Temporal Meta-Graph Learning as a novel Graph Structure Learning mechanism on spatio-temporal data. Specifically, we implement this idea into Meta-Graph Convolutional Recurrent Network (MegaCRN) by plugging the Meta-Graph Learner powered by a Meta-Node Bank into GCRN encoder-decoder. We conduct a comprehensive evaluation on two benchmark datasets (i.e., METR-LA and PEMS-BAY) and a new large-scale traffic speed dataset called EXPY-TKY that covers 1843 expressway road links in Tokyo. Our model outperformed the state-of-the-arts on all three datasets. Besides, through a series of qualitative evaluations, we demonstrate that our model can explicitly disentangle the road links and time slots with different patterns and be robustly adaptive to any anomalous traffic situations. Codes and datasets are available at https://github.com/deepkashiwa20/MegaCRN.

Spatial-temporal (ST) graph modeling, such as traffic speed forecasting and taxi demand prediction, is an important task in deep learning area. However, for the nodes in the graph, their ST patterns can vary greatly in difficulties for modeling, owning to the heterogeneous nature of ST data. We argue that unveiling the nodes to the model in a meaningful order, from easy to complex, can provide performance improvements over traditional training procedure. The idea has its root in Curriculum Learning, which suggests in the early stage of training models can be sensitive to noise and difficult samples. In this paper, we propose ST-Curriculum Dropout, a novel and easy-to-implement strategy for spatial-temporal graph modeling. Specifically, we evaluate the learning difficulty of each node in high-level feature space and drop those difficult ones out to ensure the model only needs to handle fundamental ST relations at the beginning, before gradually moving to hard ones. Our strategy can be applied to any canonical deep learning architecture without extra trainable parameters, and extensive experiments on a wide range of datasets are conducted to illustrate that, by controlling the difficulty level of ST relations as the training progresses, the model is able to capture better representation of the data and thus yields better generalization.

Conventional reinforcement learning (RL) needs an environment to collect fresh data, which is impractical when online interactions are costly. Offline RL provides an alternative solution by directly learning from the previously collected dataset. However, it will yield unsatisfactory performance if the quality of the offline datasets is poor. In this paper, we consider an offline-to-online setting where the agent is first learned from the offline dataset and then trained online, and propose a framework called Adaptive Policy Learning for effectively taking advantage of offline and online data. Specifically, we explicitly consider the difference between the online and offline data and apply an adaptive update scheme accordingly, that is, a pessimistic update strategy for the offline dataset and an optimistic/greedy update scheme for the online dataset. Such a simple and effective method provides a way to mix the offline and online RL and achieve the best of both worlds. We further provide two detailed algorithms for implementing the framework through embedding value or policy-based RL algorithms into it. Finally, we conduct extensive experiments on popular continuous control tasks, and results show that our algorithm can learn the expert policy with high sample efficiency even when the quality of offline dataset is poor, e.g., random dataset.

Wave Function Collapse (WFC) is a widely used tile-based algorithm in procedural content generation, including textures, objects, and scenes. However, the current WFC algorithm and related research lack the ability to generate commercialized large-scale or infinite content due to constraint conflict and time complexity costs. This paper proposes a Nested WFC (N-WFC) algorithm framework to reduce time complexity. To avoid conflict and backtracking problems, we offer a complete and sub-complete tileset preparation strategy, which requires only a small number of tiles to generate aperiodic and deterministic infinite content. We also introduce the weight-brush system that combines N-WFC and sub-complete tileset, proving its suitability for game design. Our contribution addresses WFC’s challenge in massive content generation and provides a theoretical basis for implementing concrete games.

Urban time series data forecasting featuring significant contributions to sustainable development is widely studied as an essential task of the smart city. However, with the dramatic and rapid changes in the world environment, the assumption that data obey Independent Identically Distribution is undermined by the subsequent changes in data distribution, known as concept drift, leading to weak replicability and transferability of the model over unseen data. To address the issue, previous approaches typically retrain the model, forcing it to fit the most recent observed data. However, retraining is problematic in that it leads to model lag, consumption of resources, and model re-invalidation, causing the drift problem to be not well solved in realistic scenarios. In this study, we propose a new urban time series prediction model for the concept drift problem, which encodes the drift by considering the periodicity in the data and makes on-the-fly adjustments to the model based on the drift using a meta-dynamic network. Experiments on real-world datasets show that our design significantly outperforms state-of-the-art methods and can be well generalized to existing prediction backbones by reducing their sensitivity to distribution changes.

With the rapid development of the Intelligent Transportation System (ITS), accurate traffic forecasting has emerged as a critical challenge. The key bottleneck lies in capturing the intricate spatio-temporal traffic patterns. In recent years, numerous neural networks with complicated architectures have been proposed to address this issue. However, the advancements in network architectures have encountered diminishing performance gains. In this study, we present a novel component called spatio-temporal adaptive embedding that can yield outstanding results with vanilla transformers. Our proposed Spatio-Temporal Adaptive Embedding transformer (STAEformer) achieves state-of-the-art performance on five real-world traffic forecasting datasets. Further experiments demonstrate that spatio-temporal adaptive embedding plays a crucial role in traffic forecasting by effectively capturing intrinsic spatio-temporal relations and chronological information in traffic time series.

To understand human mobility following the typhoon, analyzing the causal impact of extreme typhoon weather on human mobility is important for disaster emergency management. However, the unobserved confounders (e.g., the characteristic of each region) correlate with the strength of typhoon weather and also affect human mobility during typhoon, which may generate biased influences on the causal analysis process. Besides, these confounders may be time-varying following the dynamic movements of typhoon. In this work, we develop a neural network-based continuous causal effect estimation framework to mitigate the interference from (unobserved) confounders and assess the continuous causal responses of typhoon-related weather (treatment) on several types of human mobility (outcome) across different counties at any given period. To this end, we integrate the big data from two huge typhoons in Japan (i.e., Typhoon Faxai and Hagibis) and leverage multiple sources of covariates (i.e., residents' vigilance and basic mobility patterns) from different counties to learn the representations of time-varying confounders. The experimental results indicate the effectiveness of our proposed framework in capturing the confounders for quantifying the causal impact of extreme weather during the typhoon process, compared with several existing causal studies.

Machine unlearning allows data owners to erase certain data and its impact from learning models for the right to be forgotten. However, privacy risks during the unlearning process have been identified. Earlier studies have used differences in model outputs before and after unlearning to conduct membership inference attacks. Nevertheless, the current attacks on machine unlearning are limited to inference and cannot reconstruct data without access to the victim's dataset. In this paper, we propose a reconstruction attack towards machine unlearning (RAU), which can reconstruct the unlearned data by exploiting the privacy leakage from the two models. To improve reconstruction quality, we propose a Conditional Matching Generative Adversarial Network (CMGAN), a novel variant of generative adversarial networks which introduces a reconstructive loss. Our work demonstrates the possible privacy leakage of current machine unlearning scenarios. Experimental results on MNIST and Fashion-MNIST show that the proposed attack achieves high label recovery accuracy and good data recovery performance.

Tensor time series (TTS) data, a generalization of one-dimensional time series on a high-dimensional space, is ubiquitous in real-world scenarios, especially in monitoring systems involving multi-source spatio-temporal data (e.g., transportation demands and air pollutants). Compared to modeling time series or multivariate time series, which has received much attention and achieved tremendous progress in recent years, tensor time series has been paid less effort. Properly coping with the tensor time series is a much more challenging task, due to its high-dimensional and complex inner structure. In this paper, we develop a novel TTS forecasting framework, which seeks to individually model each heterogeneity component implied in the time, the location, and the source variables. We name this framework as GMRL, short for Gaussian Mixture Representation Learning. Experiment results on two real-world TTS datasets verify the superiority of our approach compared with the state-of-the-art baselines. Code and data are published on https://github.com/beginner-sketch/GMRL.

Recent years have witnessed the great potential of attention mechanism in graph representation learning. However, while variants of attention-based GNNs are setting new benchmarks for numerous real-world datasets, recent works have pointed out that their induced attentions are less robust and generalizable against noisy graphs due to lack of direct supervision. In this paper, we present a new framework which utilizes the tool of causality to provide a powerful supervision signal for the learning process of attention functions. Specifically, we estimate the direct causal effect of attention to the final prediction, and then maximize such effect to guide attention attending to more meaningful neighbors. Our method can serve as a plug-and-play module for any canonical attention-based GNNs in an end-to-end fashion. Extensive experiments on a wide range of benchmark datasets illustrated that, by directly supervising attention functions, the model is able to converge faster with a clearer decision boundary, and thus yields better performances.

Today, thanks to the rapid developing mobile and sensor networks in IoT (Internet of Things) systems, spatio-temporal big data are being constantly generated. They have brought us a data-driven possibility to sense and understand crowd mobility on a city scale. A fundamental task towards the next-generation mobility services, such as Intelligent Transportation Systems (ITS), Mobility-as-a-Service (MaaS), is spatio-temporal predictive modeling of the geo-sensory signals. There is a recent line of research leveraging deep learning techniques to boost the forecasting performance on such tasks. While simulating the regularity of mobility behaviors (e.g., routines, periodicity) in a more sophisticated way, the existing studies ignore an important part of urban activities, i.e., events. Including holidays, extreme weathers, pandemic, accidents, various urban events happen from time to time and cause non-stationary phenomena, which by nature make the spatio-temporal forecasting task challenging. We thereby envision an event-aware urban mobility prediction model that is capable of fast adapting and making reliable predictions in different scenarios, which is crucial to decision making towards emergency response and urban resilience.

Human mobility nowcasting is a fundamental research problem for intelligent transportation planning, disaster responses and management, etc. In particular, human mobility under big disasters such as hurricanes and pandemics deviates from its daily routine to a large extent, which makes the task more challenging. Existing works mainly focus on traffic or crowd flow prediction in normal situations. To tackle this problem, in this study, disaster-related Twitter data is incorporated as a covariate to understand the public awareness and attention about the disaster events and thus perceive their impacts on the human mobility. Accordingly, we propose a Meta-knowledge-Memorizable Spatio-Temporal Network (MemeSTN), which leverages memory network and meta-learning to fuse social media and human mobility data. Extensive experiments over three real-world disasters including Japan 2019 typhoon season, Japan 2020 COVID-19 pandemic, and US 2019 hurricane season were conducted to illustrate the effectiveness of our proposed solution. Compared to the state-of-the-art spatio-temporal deep models and multivariate-time-series deep models, our model can achieve superior performance for nowcasting human mobility in disaster situations at both country level and state level.

In recent years, the use of WiFi fingerprints for indoor positioning has grown in popularity, largely due to the widespread availability of WiFi and the proliferation of mobile communication devices. However, many existing methods for constructing fingerprint data sets rely on labor-intensive and time-consuming processes of collecting large amounts of data. Additionally, these methods often focus on ideal laboratory environments, rather than considering the practical challenges of large multifloor buildings. To address these issues, we present a novel WiFi domain adversarial graph convolutional network model that can be trained using a small number of labeled site survey data and large amounts of unlabeled crowdsensed WiFi fingerprints. By constructing heterogeneous graphs based on received signal strength indicators (RSSIs) between waypoints and WiFi access points (APs), our model is able to effectively capture the topological structure of the data. We also incorporate graph convolutional networks (GCNs) to extract graph-level embeddings, a feature that has been largely overlooked in previous WiFi indoor localization studies. To deal with the challenges of large amounts of unlabeled data and multiple data domains, we employ a semisupervised domain adversarial training scheme to effectively utilize unlabeled data and align the data distributions across domains. Our system is evaluated using a public indoor localization data set that includes multiple buildings, and the results show that it performs competitively in terms of localization accuracy in large buildings.

As trajectory plays an important role in intelligent transportation systems, achieving trajectory recovery from gradients is of great value. Existing research has shown that federated learning is vulnerable to attacks that recover the original training data from shared gradients. Still, we find that gradients attack is difficult to succeed with trajectory data. Most existing attacks have minimal effectiveness when facing models with higher nonlinearity and temporal-related characteristics, such as destination prediction models. In this article, we propose deep leakage from gradients with mobility prior (DLGMP) algorithm to solve these problems. The proposed DLGMP algorithm leverages the spatiotemporal structural information as the prior mobility knowledge, narrowing down the initial search space sharply and decreasing the difficulty of recovery. We further improve our algorithm with an easily extensible regularization term and an adversarial loss of Wasserstein GAN (WGAN) to help recover accurate and reasonable trajectories. Experiments on three mainstream data sets show good performance of our DLGMP algorithm. We also discuss several possible solutions to prevent gradients attack.

Activity pattern prediction is a critical part of urban computing, urban planning, intelligent transportation, and so on. Based on a dataset with more than 10 million GPS trajectory records collected by mobile sensors, this research proposed a CNN-BiLSTM-VAE-ATT-based encoder-decoder model for fine-grained individual activity sequence prediction. The model combines the long-term and short-term dependencies crosswise and also considers randomness, diversity, and uncertainty of individual activity patterns. The proposed results show higher accuracy compared to the ten baselines. The model can generate high diversity results while approximating the original activity patterns distribution. Moreover, the model also has interpretability in revealing the time dependency importance of the activity pattern prediction.

Life pattern clustering is essential for abstracting the groups' characteristics of daily life patterns and activity regularity. Based on millions of GPS records, this research proposes a framework on the life pattern clustering which can efficiently identify the groups that have similar life patterns. The proposed method can retain original features of individual life pattern data without aggregation. Metagraph-based data structure is proposed for presenting the diverse life pattern. Spatial-temporal similarity includes significant places semantics, time-sequential properties and frequency are integrated into this data structure, which captures the uncertainty of an individual and the diversities between individuals. Non-negative-factorization-based method is utilized for reducing the dimension. The results show that our proposed method can effectively identify the groups that have similar life pattern in long term and takes advantage in computation efficiency and representational capacity compared with the traditional methods. We reveal the representative life pattern groups and analyze the group characteristics of human life patterns during different periods and different regions. We believe our work helps in future infrastructure planning, services improvement and policy making related to urban and transportation, thus promoting a humanized and sustainable city.

Human mobility similarity comparison plays a critical role in modeling city dynamics, which exerts an enormous impact on developing intelligent transportation system. By expanding origin-destination matrix, we propose a mobility expression named mobility tableau and corresponding similarity measurement approach. Compared with traditional Origin-Destination matrix-based mobility comparison, mobility tableau comparison provides multi-dimensional similarity information, including volume similarity, spatial similarity, mass inclusiveness and structure similarity. The robustness of the measure is supported through several sensitive analysis based on real Global Positioning System dataset. The better performance of our proposed approach compared with traditional methods in two case studies including Call Detail Record based mobility tableau validation and different cities’ mobility comparison also demonstrates the practicality and superiority of our method.

The fast growing of e-commerce makes the express delivery an important component of transportation system. Absent delivery as one of the main reasons of failed delivery, brings serious waste of resources every year. Although with the development of Internet of Things technology, positioning system of recipients’ mobile devices can provide location information to help delivery routing optimization solving absent delivery, the lack of security makes it not practical. This paper proposes an IoT positioning technology based intelligent delivery system by introducing blockchain system and location information encryption, which overcomes three fatal problems of traditional IoT positioning technology based system: different party shares all information, location information interaction is too exposed and the system is unattractive to recipients. A set of analysis with real case experiment about efficiency improvement, incentive effect and security are conducted to validate the robustness of the system. Compared with previous research, the proposed system not only has high accuracy by utilizing real-time location information instead of unreliable prediction results, but also possesses high security.

How to effectively extract identifiable information from human mobility data and distinguish different agents is a significant topic for location-based services and intelligent transportation systems, which is described as the Human Mobility Signature Identification problem. A deeper understanding of the identifiable information underlain in human mobility can help us lay the foundation for applications such as irregular user behavior detection and privacy protection. However, human mobility comprises a mixture of different mobility patterns, traditional methods usually pay more attention to spatial-temporal features, while pattern dimension feature is usually ignored, which makes the result very dependent on the population agglomeration degree. To bridge the research gap, in this paper, we propose a novel Life-Travel pattern-based learning module (LTP-Net), in which spatial-temporal-pattern dimension features are embedded together to provide more comprehensive information for individual identification. A real-world mobile phone location dataset is utilized to evaluate the performance of the proposed LTP-Net and traditional methods. Several case studies are also conducted to analyze the model performance, including the abnormal behavior detection for the east Japan earthquake.

How to generate road conditions from urban GPS trajectory is an important problem in transportation systems. However, this computation process usually suffers from serious missing value problem due to the observation uncertainty or limited reports from crowdsourcing systems. Conventional tensor factorization approaches learn the spatio-temporal dependencies in a collaborative filtering way, which ignores the complex road network structure information and temporal heterogeneity. In this study, we propose a multi-view model with multiple aspects of prior knowledge to impute traffic state computed from a real-world trajectory dataset. More specifically, in the spatial view, rather than focusing on a specific type of road segment, we take the heterogeneity of road network into consideration and model the multiple relations of adjacent road segments. Meanwhile, the temporal pattern is also viewed as a heterogeneous graphical structure that discriminates the weekly/hourly adjacency in the temporal view. Finally, we fuse the above spatio-temporal features to provide a robust estimation under different sparse conditions. Intensive experiments on two types of missing scenarios (i.e., random and non-random) demonstrate that the proposed imputation method outperforms all the other state-of-the-art approaches. In addition, our model represents interpretable patterns for spatio-temporal graph analysis.

Multi-variate time series (MTS) data is a ubiquitous class of data abstraction in the real world. Any instance of MTS is generated from a hybrid dynamical system and their specific dynamics are usually unknown. The hybrid nature of such a dynamical system is a result of complex external attributes, such as geographic location and time of day, each of which can be categorized into either spatial attributes or temporal attributes. Therefore, there are two fundamental views which can be used to analyze MTS data, namely the spatial view and the temporal view. Moreover, from each of these two views, we can partition the set of data samples of MTS into disjoint forecasting tasks in accordance with their associated attribute values. Then, samples of the same task will manifest similar forthcoming pattern, which is less sophisticated to be predicted in comparison with the original single-view setting. Considering this insight, we propose a novel multi-view multi-task (MVMT) learning framework for MTS forecasting. Instead of being explicitly presented in most scenarios, MVMT information is deeply concealed in the MTS data, which severely hinders the model from capturing it naturally. To this end, we develop two kinds of basic operations, namely task-wise affine transformation and task-wise normalization, respectively. Applying these two operations with prior knowledge on the spatial and temporal view allows the model to adaptively extract MVMT information while predicting. Extensive experiments on three datasets are conducted to illustrate that canonical architectures can be greatly enhanced by the MVMT learning framework in terms of both effectiveness and efficiency. In addition, we design rich case studies to reveal the properties of representations produced at different phases in the entire prediction procedure.

Predicting the density and flow of the crowd or traffic at a citywide level becomes possible by using the big data and cutting-edge AI technologies. It has been a very significant research topic with high social impact, which can be widely applied to emergency management, traffic regulation, and urban planning. In particular, by meshing a large urban area to a number of fine-grained mesh-grids, citywide crowd and traffic information in a continuous time period can be represented with 4D tensor (Timestep, Height, Width, Channel). Based on this idea, a series of methods have been proposed to address grid-based prediction for citywide crowd and traffic. In this study, we revisit the density and in-out flow prediction problem and publish a new aggregated human mobility dataset generated from a real-world smartphone application. Comparing with the existing ones, our dataset holds several advantages including large mesh-grid number, fine-grained mesh size, and high user sample. Towards this large-scale crowd dataset, we propose a novel deep learning model called DeepCrowd by designing pyramid architectures and high-dimensional attention mechanism based on Convolutional LSTM. Lastly, thorough and comprehensive performance evaluations are conducted to demonstrate the superiority of the proposed DeepCrowd comparing to multiple state-of-the-art methods.

Recently, forecasting the crowd flows has become an important research topic, and plentiful technologies have achieved good performances. As we all know, the flow at a citywide level is in a mixed state with several basic patterns (e.g., commuting, working, and commercial) caused by the city area functional distributions (e.g., developed commercial areas, educational areas and parks). However, existing technologies have been criticized for their lack of considering the differences in the flow patterns among regions since they want to build only one comprehensive model to learn the mixed flow tensors. Recognizing this limitation, we present a new perspective on flow prediction and propose an explainable framework named ST-ExpertNet, which can adopt every spatial-temporal model and train a set of functional experts devoted to specific flow patterns. Technically, we train a bunch of experts based on the Mixture of Experts (MoE), which guides each expert to specialize in different kinds of flow patterns in sample spaces by using the gating network. We define several criteria, including comprehensiveness, sparsity, and preciseness, to construct the experts for better interpretability and performances. We conduct experiments on a wide range of real-world taxi and bike datasets in Beijing and NYC. The visualizations of the expert's intermediate results demonstrate that our ST-ExpertNet successfully disentangles the city's mixed flow tensors along with the city layout, e.g., the urban ring road structure. Different network architectures, such as ST-ResNet, ConvLSTM, and CNN, have been adopted into our ST-ExpertNet framework for experiments and the results demonstrates the superiority of our framework in both interpretability and performances.

Travel time estimation from GPS trips is of great importance to order duration, ridesharing, taxi dispatching, etc. However, the dense trajectory is not always available due to the limitation of data privacy and acquisition, while the origin-destination (OD) type of data, such as NYC taxi data, NYC bike data, and Capital Bikeshare data, is more accessible. To address this issue, this paper starts to estimate the OD trips travel time combined with the road network. Subsequently, aMulti-taskWeaklySupervisedLearning Framework forTravelTimeEstimation (MWSL-TTE) has been proposed to infer transition probability between roads segments, and the travel time on road segments and intersection simultaneously. Technically, given an OD pair, the transition probability intends to recover the most possible route. And then, the output of travel time is equal to the summation of all segments’ and intersections’ travel time in this route. A novel route recovery function has been proposed to iteratively maximize the current routes’ co-occurrence probability, and minimize the discrepancy between routes’ probability distribution and the inverse distribution of routes’ estimation loss. Moreover, the expected log-likelihood function based on a weakly-supervised framework has been deployed in optimizing the travel time from road segments and intersections concurrently. We conduct experiments on a wide range of real-world taxi datasets in Xi’an and Chengdu and demonstrate our method's effectiveness on route recovery and travel time estimation.

Aggregating crowd density in grids from big mobile datasets is a basic but critical work in urban computing and mobile computing. The error of position estimation in raw mobile data, including spatial deviation and temporal deviation, is inevitable and directly impacts the accuracy of aggregated crowd density results. In this case, a key modifiable areal unit problem is raised to understand the relationship among the crowd density accuracy, raw mobile data error, grid shape, and size, but few studies focused on it. This paper analyzes this modifiable areal unit problem of the error in crowd density estimation from big mobility data. By regarding the error as the result of a convolution operation, an optimization model based restoration method was proposed to fix the error of the estimated result, and we analyzed the restoration effect under different circumstances by several simulation experiments. A real application for grided population distribution map construction and restoration from Call Detail Record was conducted to prove the reliability of the whole analysis, which demonstrates the restoration method can reduce the error by nearly 40% under certain conditions.

The outbreak of coronavirus disease (COVID-19) has swept across more than 180 countries and territories since late January 2020. As a worldwide emergency response, governments have implemented various measures and policies, such as self-quarantine, travel restrictions, work from home, and regional lockdown, to control the spread of the epidemic. These countermeasures seek to restrict human mobility because COVID-19 is a highly contagious disease that is spread by human-to-human transmission. Medical experts and policymakers have expressed the urgency to effectively evaluate the outcome of human restriction policies with the aid of big data and information technology. Thus, based on big human mobility data and city POI data, an interactive visual analytics system called Epidemic Mobility (EpiMob) was designed in this study. The system interactively simulates the changes in human mobility and infection status in response to the implementation of a certain restriction policy or a combination of policies (e.g., regional lockdown, telecommuting, screening). Users can conveniently designate the spatial and temporal ranges for different mobility restriction policies. Then, the results reflecting the infection situation under different policies are dynamically displayed and can be flexibly compared and analyzed in depth. Multiple case studies consisting of interviews with domain experts were conducted in the largest metropolitan area of Japan (i.e., Greater Tokyo Area) to demonstrate that the system can provide insight into the effects of different human mobility restriction policies for epidemic control, through measurements and comparisons.

As a decisive part in the success of Mobility-as-a-Service (MaaS), spatio-temporal predictive modeling for crowd movements is a challenging task particularly considering scenarios where societal events drive mobility behavior deviated from the normality. While tremendous progress has been made to model high-level spatio-temporal regularities with deep learning, most, if not all of the existing methods are neither aware of the dynamic interactions among multiple transport modes nor adaptive to unprecedented volatility brought by potential societal events. In this paper, we are therefore motivated to improve the canonical spatio-temporal network (ST-Net) from two perspectives: (1) design a heterogeneous mobility information network (HMIN) to explicitly represent intermodality in multimodal mobility; (2) propose a memory-augmented dynamic filter generator (MDFG) to generate sequence-specific parameters in an on-the-fly fashion for various scenarios. The enhanced event-aware spatio-temporal network, namely EAST-Net, is evaluated on several real-world datasets with a wide variety and coverage of societal events. Both quantitative and qualitative experimental results verify the superiority of our approach compared with the state-of-the-art baselines. Code and data are published on https://github.com/underdoc-wang/EAST-Net.

A core component of the Intelligent Transportation System (ITS) is road network, which forms the most basic transport infrastructure, and becomes widely applied in many traffic applications. In most traffic models, the spatial representation of road network is learned only through static graph connection while dynamic driver preference and traffic conditions in the real world are ignored. Therefore, in this paper, a novel trajectory-based road network representation is proposed. By mining vehicle trajectories, our proposed method can learn dynamic route choice through embeddings of each road in a next-hop prediction model. Then road correlations are calculated by the embeddings to build a latent correlation graph that can be applied in various traffic-related applications. Extensive experiment results prove the effectiveness and rationality of our proposed approach.

COVID-19 has dramatically changed people’s mobility patterns. This report aims to analyze the impact of COVID-19 on people’s mobility through statistics and comparing the visits of POIs (Point-Of-Interests) in New York State in 2019 and 2020. The report uses data from SafeGraph, which is a data company. The raw data contains POI visits across the United States in 2019 and 2020. Considering the analysis size and difficulty of the data, POI visits from New York State are extracted for analysis, and POI locations are classified according to the tags provided by the source data. The scale of analysis is from macro to micro, and they are the total POI visits data of New York State based on different ways in 2019 and 2020, the POI visits of CBG (Census Block Group) division in New York City, and three representative POI samples to do individual analysis. The analysis methods are: (1) use line plot and bar plot statistics to compare the trends of POI visits data from 2019 to 2020, and (2) make the spatial visualization comparison, which includes grid map, scatter map, heatmap, and OD map, between the first peak of epidemic impact in the first full week of April 2019 and April 2020, and the scope is narrowed to New York City. Wherein the OD maps are drawn based on the CBG division. Compared to related work, the analysis object includes CBG, categories, and individual POI. In addition, the analysis method combines statistical graphs and spatial visualizations and explores the policy impact of the New York City government. This report adopts more multidimensional analysis methods and objects to improve the comprehensiveness and reliability of the analysis content.

Citywide crowd prediction can be of great importance for emergency management, traffic regulation, and urban planning. By meshing a large urban area to a number of fine-grained mesh-grids as illustrated in Fig. 1 , citywide crowd in a continuous time period can be represented with a four-dimensional tensor ${\mathbb{R}^{Timestep{\text{ }}p \times {\text{ }}Height{\text{ }} \times {\text{ }}Width{\text{ }} \times {\text{ }}Channel{\text{ }}}}$ in an analogous manner to video data, where each Timestep can be seen as one video frame, Height , Width is two-dimensional index for mesh-grids, and each Channel stores an aggregated scalar value for each mesh-grid. Specifically, given historical observations of crowd density and in-out flow x d = d 1 ,…,d t , xf = f 1 ,…, f t at timestamp t , we aim to build prediction models for the next-step density and in-out flow y d = d t +1, yf = f t +1, where y d means how many people will be in each mesh-grid at the next timestamp, and yf means how many people will flow into or out from each mesh-grid in next time interval. Al-though many deep models [1] – [6] have been proposed to address such tasks, their actual effects are still not well validated on large-scale and high-quality datasets. The datasets used in most of the works so far are originally generated based on taxi or bicycle trip data, which don’t cover and reflect the citywide crowd density and flow. Thus, we first publish new crowd flow data called BousaiTYO and BousaiOSA [7] . These new datasets are created using the GPS log data collected from a popular smartphone app of Yahoo! Japan Corporation, which can well reflect the real-world crowd flow in Tokyo and Osaka. As shown by Table 1 , our dataset has: (1) larger spatial area; (2) finer mesh size; (3) higher user sample.

Traffic state prediction is essential in Intelligent Transportation Systems for surveillance, management, and daily commuting. For developing high-accuracy prediction models, real-world traffic state datasets are necessary for training model parameters and evaluating prediction results. However, limited by the existing traffic collection devices, most of the current open datasets for traffic state prediction cannot obtain accurate traffic flow information. In contrast, some datasets directly use detection devices in freeway systems, so they cannot reflect complex urban traffic states. Therefore, a dataset from advanced devices that can record the flow from point to point on an urban road network attracts more attention and drives the progress of research on traffic state prediction models. To deal with the above issues, we introduce a Suburban Traffic Flow dataset using Geomagnetic sensors, or STF-G dataset, constructed for traffic flow prediction. The STF-G dataset consists of 2.5 billion vehicle driving scenarios and 319 corresponding geomagnetic sensors. The data was collected over 20 months and processed with two regional road graphs. We also do the Benchmark experiments in STF-G for analyzing and evaluating the performance of graph neural network models in traffic flow prediction and compare them to the other datasets with the same baseline.

Friend recommendation from human mobility is a vital real-world application of location-based social networks (LBSN). It is necessary to recognize patterns from human mobility to assist friend recommendation because previous works have shown complex relations between them. However, most of previous works either modelled social networks and user trajectories separately, or only used classical simple graph-based methods with an edge linking two nodes that cannot fully model the complex data structure of LBSN. Inspired by the fact that hyperedges can connect multiple nodes of different types, we model user trajectories and check-in records as hyperedges in a novel heterogeneous LBSN hypergraph to represent complex spatio-temporal information. And then, we design a type-specific attention mechanism for an end-to-end trainable heterogeneous hypergraph neural network (HHGNN) with supervised contrastive learning, which can learn hypergraph node embedding for the next friend recommendation task. At last, our model HHGNN outperforms the state-of-the-art methods on four real-world city datasets, while ablation studies also confirm the effectiveness of each model part.

Finding out similar regions between cities is important to a variety of real-world applications, such as point-of-interest recommendations, site selection, and travel guidance. With the help of the increasing number of location-based social network users, we can measure the intercity similarity from a new perspective of spatiotemporal characteristics of human mobility. In this paper, we developed an interactive intercity regional similarity explorer (IRSE) that 1) visualizes regional spatiotemporal human mobility features, 2) searches similar region candidates in the target city, and 3) explores the regional similarity from different views in a quantitative and illustrative way. In this paper, we show how our system can be useful in exploring regional similarity across cities in the world by use cases, which will interest various users from different countries in the demonstration session. Demo available at: https://bit.ly/3OjwnGu

Knowing "what is happening" and "what will happen" of the mobility in a city is the building block of a data-driven smart city system. In recent years, mobility digital twin that makes a virtual replication of human mobility and predicting or simulating the fine-grained movements of the subjects in a virtual space at a metropolitan scale in near real-time has shown its great potential in modern urban intelligent systems. However, few studies have provided practical solutions. The main difficulties are four-folds: 1) the daily variation of human mobility is hard to model and predict; 2) the transportation network enforces a complex constraints on human mobility; 3) generating a rational fine-grained human trajectory is challenging for existing machine learning models; and 4) making a fine-grained prediction incurs high computational costs, which is challenging for an online system. Bearing these difficulties in mind, in this paper we propose a two-stage human mobility predictor that stratifies the coarse and fine-grained level predictions. In the first stage, to encode the daily variation of human mobility at a metropolitan level, we automatically extract citywide mobility trends as crowd contexts and predict long-term and long-distance movements at a coarse level. In the second stage, the coarse predictions are resolved to a fine-grained level via a probabilistic trajectory retrieval method, which offloads most of the heavy computations to the offline phase. We tested our method using a real-world mobile phone GPS dataset in the Kanto area in Japan, and achieved good prediction accuracy and a time efficiency of about 2 min in predicting future 1h movements of about 220K mobile phone users on a single machine to support more higher-level analysis of mobility prediction.

Predicting the density and flow of the crowd at a citywide level is significant for city management. By meshing a large urban area to a number of fine-grained mesh-grids, citywide crowd and traffic information in a continuous time period can be represented with 4D tensor (Timestep, Height, Width, Channel). Based on this, we revisit the density and in-out flow prediction problem and publish a new aggregated human mobility dataset generated from a real-world smartphone application. Compared with the existing ones, our dataset has larger mesh-grid number, finer-grained mesh size, and higher user sample. Towards such kind of large-scale crowd dataset, we propose a novel deep learning model called DeepCrowd by designing pyramid architectures and high-dimensional attention mechanism based on Convolutional LSTM. Both the datasets and codes are made available at https://github.com/deepkashiwa20/DeepCrowd.

Reinforcement learning (RL) is inefficient on long-horizon tasks due to sparse rewards and its policy can be fragile to slightly perturbed environments. We address these challenges via a curriculum of tasks with coupled environments, generated by two policies trained jointly with RL: (1) a co-operative planning policy recursively decomposing a hard task into a coarse-to-fine sub-task tree; and (2) an adversarial policy modifying the environment in each sub-task. They are complementary to acquire more informative feedback for RL: (1) provides dense reward of easier sub-tasks while (2) modifies sub-tasks’ environments to be more challenging and diverse. Conversely, they are trained by RL’s dense feedback on sub-tasks so their generated curriculum keeps adaptive to RL’s progress. The sub-task tree enables an easy-to-hard curriculum for every policy: its top-down construction gradually increases sub-tasks the planner needs to generate, while the adversarial training between the environment and RL follows a bottom-up traversal that starts from a dense sequence of easier sub-tasks allowing more frequent environment changes. We compare EAT-C with RL/planning targeting similar problems and methods with environment generators or adversarial agents. Extensive experiments on diverse tasks demonstrate the advantages of our method on improving RL’s efficiency and generalization.

Estimating the travel time of a path is an essential topic for the intelligent transportation system. It serves as the foundation for real-world applications, such as traffic monitoring, route planning, and taxi dispatching. However, building a model for such a data-driven task requires a large amount of users’ travel information, which closely relates to their privacy and, thus, is less likely to be shared. The not independent and identically distributed (Non-IID) trajectory data across data owners also make a predictive model extremely challenging to be personalized if we directly apply federated learning. Finally, previous work on travel time estimation (TTE) does not consider the real-time traffic state of roads, which we argue, can significantly influence the prediction. To address the above challenges, we introduce GOF-TTE for the mobile user group, generative online federated learning framework for TTE, which 1) utilizes the federated learning approach, allowing private data to be kept on client devices while training, and designs the global model as an online generative model shared by all clients to infer the real-time road traffic state and 2) apart from sharing a base model at the server, adapts a fine-tuned personalized model for every client to study their personal driving habits, making up for the residual error made by localized global model prediction. We also employ a simple privacy attack to our framework and implement the differential privacy mechanism to guarantee privacy safety further. Finally, we conduct experiments on two real-world public taxi data sets of DiDi Chengdu and Xi’an. The experimental results demonstrate the effectiveness of our proposed framework.

Recent decades have seen a significant increase in the frequency, intensity, and impact of natural disasters and other emergencies, forcing the governments around the world to make emergency response and disaster management national priorities. The growth of extremely large and complex datasets—commonly referred to asbig data—and various advances in information and communications technology and computing now support more effective approaches to humanitarian relief, logistical coordination, overall disaster management, and long-term recovery in connection with natural disasters and emergency events. Leveraging big data and technological advances for emergency management has attracted considerable attention in the research community. However, the desired merging ofbig data and emergency management(BDEM) requires coordinated efforts to align and define interdisciplinary terminologies and methodologies. To date, the key concepts and technologies in this emerging research area have not been coherently discussed in a sufficiently broad and multidisciplinary manner. In this article, an international team presents an overview of the BDEM domain, highlighting a general framework and discussing key challenges from several perspectives. We introduce and summarize typical technologies and applications, organized into the six broad categories of remote sensing, resilient communication networks, mobile communication networks, human mobility and urban sensing, social network analysis, and knowledge graphs. Finally, we outline several directions of future research.

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g., police) and public service operators (e.g., subway/bus operator) to protect people’s safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the “deep” trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent, which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

Nowadays, with the rapid development of IoT (Internet of Things) and CPS (Cyber-Physical Systems) technologies, big spatiotemporal data are being generated from mobile phones, car navigation systems, and traffic sensors. By leveraging state-of-the-art deep learning technologies on such data, urban traffic prediction has drawn a lot of attention in AI and Intelligent Transportation System community. The problem can be uniformly modeled with a 3D tensor (T, N, C), where T denotes the total time steps, N denotes the size of the spatial domain (i.e., mesh-grids or graph-nodes), and C denotes the channels of information. According to the specific modeling strategy, the state-of-the-art deep learning models can be divided into three categories: grid-based, graph-based, and multivariate time-series models. In this study, we first synthetically review the deep traffic models as well as the widely used datasets, then build a standard benchmark to comprehensively evaluate their performances with the same settings and metrics. Our study named DL-Traff is implemented with two most popular deep learning frameworks, i.e., TensorFlow and PyTorch, which is already publicly available as two GitHub repositories https://github.com/deepkashiwa20/DL-Traff-Grid and https://github.com/deepkashiwa20/DL-Traff-Graph. With DL-Traff, we hope to deliver a useful resource to researchers who are interested in spatiotemporal data analysis.

Forecasting incident occurrences (e.g. crime, EMS, traffic accident) is a crucial task for emergency service providers and transportation agencies in performing response time optimization and dynamic fleet management. However, such events are by nature rare and sparse, which causes the label imbalance problem and inferior performance of models relying on data sufficiency. The existing studies circumvent, instead of truly solving, this issue by defining the incident prediction problem in a coarse-grained temporal (e.g. daily) setting, which leaves the proposed models unrobust to fine-grained dynamics and trivial for the real-world decision making. In this paper, we tackle the temporally fine-grained incident prediction problem in a sparse setting by explicitly exploiting the behind-the-scene chainlike triggering mechanism. Moreover, this chain effect roots in multiple domains (i.e. spatial, categorical), which further entangles with the temporal dimension and happens to be time-variant. To be specific, we propose a novel deep learning framework, namely Spatio-Temporal-Categorical Graph Neural Networks (STC-GNN), to handle the multidimensional and dynamic chain effect for performing fine-grained multi-incident co-prediction. Extensive experiments on three real-world city-level incident datasets verify the insightfulness of our perspective and effectiveness of the proposed model.

Forecasting regional ambulance demand plays a fundamental part in dynamic fleet allocation and redeployment. This topic has been gaining increasing significance, as virtually every country is experiencing an aging population, with generally higher level of vulnerability and demand for the emergency medical service (EMS). Although exploring the spatial and temporal correlations in EMS historical records, the existing methods principally consider the former time-invariant, which does not necessarily hold in reality. Moreover, this assumption ignores the fact that the behind-the-scenes dynamics are people, whose demographic profiles and activity patterns could be determinants of regional EMS demands. In this paper, we are therefore motivated to mine the collective daily routines in human mobility, to further represent the evolving spatial correlations. Particularly, we model profiled mobility groups as multiple random walkers and propose a novel bicomponent neural network, including a heterogeneous multi-graph convolution layer and spatio-temporal interlacing attention module, to perform the prediction task. Experimental results on the real-world data verify the effectiveness of introducing dynamic human mobility and the advantage of our approach over the state-of-the-art models.

Multi-variate time series (MTS) data is a ubiquitous class of data abstraction in the real world. Any instance of MTS is generated from a hybrid dynamical system with their specific dynamics normally unknown. The hybrid nature of such a dynamical system is a result of complex external impacts, which can be summarized as high-frequency and low-frequency from the temporal view, or global and local if we take the spatial view. These impacts also determine the forthcoming development of MTS making them paramount to capture in a time series forecasting task. However, conventional methods face intrinsic difficulties in disentangling the components yielded by each kind of impact from the raw data. To this end, we propose two kinds of normalization modules -- temporal and spatial normalization -- which separately refine the high-frequency component and the local component underlying the raw data. Moreover, both modules can be readily integrated into canonical deep learning architectures such as Wavenet and Transformer. Extensive experiments on three datasets are conducted to illustrate that, with additional normalization modules, the performance of the canonical architectures can be enhanced by a large margin in the application of MTS and achieves state-of-the-art results compared with existing MTS models.

Transportation demand forecasting is a topic of large practical value. However, the model that fits the demand of one transportation by only considering the historical data of its own could be vulnerable since random fluctuations could easily impact the modeling. On the other hand, common factors like time and region attribute, drive the evolution demand of different transportation, leading to a co-evolving intrinsic property between different kinds of transportation. In this work, we focus on exploring the co-evolution between different modes of transport, e.g., taxi demand and shared-bike demand. Two significant challenges impede the discovery of the co-evolving pattern: (1) diversity of the co-evolving correlation, which varies from region to region and time to time. (2) Multi-modal data fusion. Taxi demand and shared-bike demand are time-series data, which have different representations with the external factors. Moreover, the distribution of taxi demand and bike demand are not identical. To overcome these challenges, we propose a novel method, known as co-evolving spatial temporal neural network (CEST). CEST learns a multi-view demand representation for each mode of transport, extracts the co-evolving pattern, then predicts the demand for the target transportation based on multi-scale representation, which includes fine-scale demand information and coarse-scale pattern information. We conduct extensive experiments to validate the superiority of our model over the state-of-art models.

Nowadays, GPS devices have increased explosively and produced huge amounts of trajectory data related to people's outgoing. Through those big location data, many researches aim to analyze human mobility for urban development, such as human movement prediction/modeling, POI (Point-Of-Interest) recommendation. However, trajectory data only contains timestamp and location information. The intention of human movement is not explicit so that it is hard to understand why people go to somewhere. The intention prior to the activity could be of great significance for analyzing and predicting human mobility, which has not been taken into consideration by the existing researches until the present. Thus, in this study, we propose a brand-new concept called human intentional mobility, aiming to employ intention information to predict people's outgoing. We carefully utilize user's search query to sense his intention as well as the intensity. For instance, if a user searches a certain POI for many times in a short period, it will represent a relatively high intention to go there. Then, to fully utilize this intention representation for predicting whether user will visit searched POI or not, we specially design Dual Sequential Interaction Network (DualSIN) as a novel and unique deep-learning model, which can effectively capture the sophisticated interactions among two kinds of sequential information (i.e., search sequence and mobility sequence) and typical categorical information (i.e., user attributes). Last, we evaluate our model on real-world dataset collected from Yahoo! Japan portal application, and demonstrate that it can achieve superior satisfactory performances to the-state-of-the-art models on multiple POI search queries.

Extracting identifiable information from human trajectories is a fundamental task in many location-based services (LBS). However, various mobility patterns underlain in human trajectories are difficult to model by existing models. Moreover, we could hardly define a clear user set for user identification because the set of users are dynamic and changing everyday. Bearing these in mind, we apply a deep multiple instance learning method to handle the multimodal mobility patterns in a weak-supervised learning way, and address the dynamic user set problems via a pairwise loss with negative sampling. We utilize a multi-head attention mechanism to automatically extract multiple aspects and match the corresponding information between query trajectories and historical trajectories. Our method shows a good identification accuracy on three human GPS trajectory data sets comparing with baseline methods.

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g. police) and public service operators (e.g. subway/bus operator) to protect people's safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the deep trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly-complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

Big human mobility data are being continuously generated through a variety of sources, some of which can be treated and used as streaming data for understanding and predicting urban dynamics. With such streaming mobility data, the online prediction of short-term human mobility at the city level can be of great significance for transportation scheduling, urban regulation, and emergency management. In particular, when big rare events or disasters happen, such as large earthquakes or severe traffic accidents, people change their behaviors from their routine activities. This means people's movements will almost be uncorrelated with their past movements. Therefore, in this study, we build an online system called DeepUrbanMomentum to conduct the next short-term mobility predictions by using (the limited steps of) currently observed human mobility data. A deep-learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data for a huge urban area. Experimental results demonstrate the superior performance of our proposed model as compared to the existing approaches. Lastly, we apply our system to a real emergency scenario and demonstrate that our system is applicable in the real world.

The four papers in this special section examine the use of multimedia Big Data for leveraging big events such as natural disasters. Leveraging technological advances and using Big Data approaches for information management and processing in extreme events have attracted much research attention. Multimedia is increasingly becoming the most important and valuable source for insight and information on extreme events. It covers from everyone’s experiences to everything happening in the world. There will be lots of multimedia big data in extreme events—surveillance video, disaster images, social media, voice and video, to name a few, only if their volumes grow to the extent that the traditional multimedia processing and analysis systems may not handle effectively. As such, multimedia big data in extreme events are spurring on tremendous amounts of research and development of related technologies and applications. The papers in this issue is intended to provide a premier forum for researchers working on multimedia big data for extreme events to present their recent research results. It also provides an important opportunity for multidisciplinary studies connecting multimedia computing to disaster information management.

Globalization leads to more and more outbound behavior. Understanding these behavior is of great significance to both outbound and inbound countries in a lot of fields. However, the previous research on outbound behavior mainly unitizes questionnaires or purely analyzes the statistical data, these data are either too difficult to acquire or unable to directly analyze public opinion. This paper aim to analyze outbound behavior through large volume of text and location information published on social network services (SNS) and look into a case study of Chinese people in Japan with the data acquired from Chinese microblog website, Sina Weibo. For text analysis, we utilize an adapted latent Dirichlet allocation model to extract the topics in text and discuss the spatial and temporal distribution of the topics. The experiment proves that the topics represent outbound behavior of tourists and foreign visitors can be found and explained in the experiment dataset, which indicates that SNS data is valid for analyzing outbound behavior.

In recent decades, the frequency and intensity of natural disasters has increased significantly, and this trend is expected to continue. Therefore, understanding and predicting human behavior and mobility during a disaster will play a vital role in planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, such research is very difficult to perform owing to the uniqueness of various disasters and the unavailability of reliable and large-scale human mobility data. In this study, we collect big and heterogeneous data (e.g., GPS records of 1.6 million users 1 over 3 years, data on earthquakes that have occurred in Japan over 4 years, news report data, and transportation network data) to study human mobility following natural disasters. An empirical analysis is conducted to explore the basic laws governing human mobility following disasters, and an effective human mobility model is developed to predict and simulate population movements. The experimental results demonstrate the efficiency of our model, and they suggest that human mobility following disasters can be significantly more predictable and be more easily simulated than previously thought.

The frequency and intensity of natural disasters has increased significantly in recent decades, and this trend is expected to continue. Hence, understanding and predicting human evacuation behavior and mobility will play a vital role in planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, existing models are shallow models, and it is difficult to apply them for understanding the “deep knowledge” of human mobility. Therefore, in this study, we collect big and heterogeneous data (e.g., GPS records of 1.6 million users over 3 years, data on earthquakes that have occurred in Japan over 4 years, news report data, and transportation network data), and we build an intelligent system, namely, DeepMob, for understanding and predicting human evacuation behavior and mobility following different types of natural disasters. The key component of DeepMob is based on a deep learning architecture that aims to understand the basic laws that govern human behavior and mobility following natural disasters, from big and heterogeneous data. Furthermore, based on the deep learning model, DeepMob can accurately predict or simulate a person’s future evacuation behaviors or evacuation routes under different disaster conditions. Experimental results and validations demonstrate the efficiency and superior performance of our system, and suggest that human mobility following disasters may be predicted and simulated more easily than previously thought.

With the rapid development of urbanization and public transportation system, the number of traffic accidents have significantly increased globally over the past decades and become a big problem for human society. Facing these possible and unexpected traffic accidents, understanding what causes traffic accident and early alarms for some possible ones will play a critical role on planning effective traffic management. However, due to the lack of supported sensing data, research is very limited on the field of updating traffic accident risk in real-time. Therefore, in this paper, we collect big and heterogeneous data (7 months traffic accident data and 1.6 million users' GPS records) to understand how human mobility will affect traffic accident risk. By mining these data, we develop a deep model of Stack denoise Autoencoder to learn hierarchical feature representation of human mobility. And these features are used for efficient prediction of traffic accident risk level. Once the model has been trained, our model can simulate corresponding traffic accident risk map with given real-time input of human mobility. The experimental results demonstrate the efficiency of our model and suggest that traffic accident risk can be significantly more predictable through human mobility.

Real-time estimation of human mobility following a massive disaster will play a crucial role in disaster relief. Because human mobility in massive disasters is quite different from their usual mobility, real-time human location data is necessary for precise estimation. Due to privacy concerns, real-time data is anonymized and a popular form of anonymization is population distribution. In this paper, we aim to estimate human mobility following an unprecedented disaster using such population distribution data. To overcome technical obstacles including high dimensionality, we propose novel particle filter by devising proposal distribution. Our proposal distribution provides states considering both prediction model and acquired observation. Therefore, particles maintain high likelihood. In the experiments, our methods realized more accurate estimation than the baselines, and its estimated mobility was consistent with the survey researches. The computational cost is significantly low enough for real-time operations. The GPS data collected on the day of the Great East Japan Earthquake is used for the evaluation.

There are two broad categories of citywide human mobility, routine, composed of daily or periodic travel, and rare, which occurs during events such as the Olympic Games or natural disasters. State-of-the-art studies have shown that routine mobility patterns can be modeled stochastically, while rare human mobility modeling, essential to a variety of urban computing scenarios, such as emergency management and traffic regulation, is a much more challenging and understudied problem. Instead of training a rare-event-specific human mobility model, which suffers from the particularity of the rare events, in this paper we provide a new insight into rare events and propose a novel algorithm, CityCoupling, which establishes an intercity spatial mapping that uses human mobility in one city as input and reproduces human mobility in another city. More intuitively, we attempt to answer the question "What if this rare event happened in another city?". To find the optimal intercity spatial mapping, we utilize an expectation-maximization algorithm to estimate a probabilistic geographical correspondence matrix by regarding intercity trajectory matching as latent variables. Thereafter, a Gibbs sampling-based multiple hidden Markov model generates simulated trajectories. We apply our approach to a large mobile phone GPS dataset in Japan and determine the spatial mapping between Tokyo and Osaka to transfer the human mobility at the Great Eastern Japan Earthquake in Tokyo, which was heavily affected, to simulate what might have occurred if Osaka had been struck by the earthquake. We conduct the evaluation by assuming that New Year's Countdown is a rare event that occurs simultaneously in both Tokyo and Osaka, and thus we quantitatively compare our simulation with the ground truth in Osaka.

We categorize this research in terms of its contribution to both graph theory and computer vision. From the theoretical perspective, this study can be considered as the first attempt to formulate the idea of mining maximal frequent subgraphs in the challenging domain of messy visual data, and as a conceptual extension to the unsupervised learning of graph matching. We define a soft attributed pattern (SAP) to represent the common subgraph pattern among a set of attributed relational graphs (ARGs), considering both their structure and attributes. Regarding the differences between ARGs with fuzzy attributes and conventional labeled graphs, we propose a new mining strategy that directly extracts the SAP with the maximal graph size without applying node enumeration. Given an initial graph template and a number of ARGs, we develop an unsupervised method to modify the graph template into the maximal-size SAP. From a practical perspective, this research develops a general platform for learning the category model (i.e., the SAP) from cluttered visual data (i.e., the ARGs) without labeling "what is where," thereby opening the possibility for a series of applications in the era of big visual data. Experiments demonstrate the superior performance of the proposed method on RGB/RGB-D images and videos.

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, understanding and simulating of human emergency mobility following disasters will becomethe critical issue for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, due to the uniquenessof various disasters and the unavailability of reliable and large scale human mobility data, such kind of research is very difficult to be performed. Hence, in this paper,we collect big and heterogeneous data (e.g. 1.6 million users' GPS records in three years, 17520 times of Japan earthquake data in four years, news reporting data, transportation network data and etc.) to capture and analyze human emergency mobility following different disasters. By mining these big data, we aim to understand what basic laws govern human mobility following disasters, and develop a general model of human emergency mobility for generating and simulating large amount of human emergency movements. The experimental results and validations demonstrate the efficiency of our simulation model, and suggest that human mobility following disasters may be significantly morepredictable and can be easier simulated than previously thought.

Human movements are difficult to predict, especially, when we consider rare behaviors that deviate from normal daily routines. By tracing the behavior of a person over a long period, we can model their daily routines and predict periodical behaviors, whereas rare behaviors, such as participating in the New Year's Eve countdown, can hardly be predicted readily and thus they have usually been treated as outliers of the daily routines in most existing studies. However, for scenarios such as emergency management or intelligent traffic regulation, we are more interested in rare behaviors than daily routines. Using human mobility Big Data, the rare behavior of each individual in a social crowd is no longer rare and thus it may be predicted when we analyze the crowd behavior at a citywide level. Therefore in this study, instead of predicting movement based on daily routines, we make short-term predictions based on the recent movement observations. We propose a novel model called CityMomentum as a predicting-by-clustering framework for sampling future movement using a mixture of multiple random Markov chains, each of which is a Naive Movement Predictive model trained with the movements of the subjects that belong to each cluster. We apply our approach to a big mobile phone GPS log dataset and predict the short-term future movements, especially during the Comiket 80 and New Year's Eve celebration. We evaluate our prediction by a Earth Mover Distance (EMD) based metric, and show our approach accurately predicts the crowd behavior during the rare crowd events, which makes an early crowd event warning and regulation possible in the emergent situations.

Mining object-level knowledge, that is, building a comprehensive category model base, from a large set of cluttered scenes presents a considerable challenge to the field of artificial intelligence. How to initiate model learning with the least human supervision (i.e., manual labeling) and how to encode the structural knowledge are two elements of this challenge, as they largely determine the scalability and applicability of any solution. In this article, we propose a model-learning method that starts from a single-labeled object for each category, and mines further model knowledge from a number of informally captured, cluttered scenes. However, in these scenes, target objects are relatively small and have large variations in texture, scale, and rotation. Thus, to reduce the model bias normally associated with less supervised learning methods, we use the robust 3D shape in RGB-D images to guide our model learning, then apply the properly trained category models to both object detection and recognition in more conventional RGB images. In addition to model training for their own categories, the knowledge extracted from the RGB-D images can also be transferred to guide model learning for a new category, in which only RGB images without depth information in the new category are provided for training. Preliminary testing shows that the proposed method performs as well as fully supervised learning methods.

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, urban emergency management has become the especially important issue for the whole governments around the world.In this paper, we present a novel intelligent system for urban emergency management during the large-scale disasters. The proposed systemstores and manages the global positioning system (GPS) records from mobile devices used by approximately 1.6 million people throughout Japan over one year. By mining and analyzing population movements after the Great East Japan Earthquake, our system can automatically learn a probabilistic model to better understand and simulate human mobility during the emergency situations. Based on the learning model, population mobility in various urban areas impacted by the earthquake throughout Japan can be automatically simulated or predicted. On the basis of such kind of system, it is easy for us to find some new features or population mobility patterns after the recent and unprecedented composite disasters, which are likely to provide valuable experience and play a vital role for future disaster management worldwide.

3D reconstruction from a single image is a classical problem in computer vision. However, it still poses great challenges for the reconstruction of daily-use objects with irregular shapes. In this paper, we propose to learn 3D reconstruction knowledge from informally captured RGB-D images, which will probably be ubiquitously used in daily life. The learning of 3D reconstruction is defined as a category modeling problem, in which a model for each category is trained to encode category-specific knowledge for 3D reconstruction. The category model estimates the pixel-level 3D structure of an object from its 2D appearance, by taking into account considerable variations in rotation, 3D structure, and texture. Learning 3D reconstruction from ubiquitous RGB-D images creates a new set of challenges. Experimental results have demonstrated the effectiveness of the proposed approach.

Graph matching and graph mining are two typical areas in artificial intelligence. In this paper, we define the soft attributed pattern (SAP) to describe the common subgraph pattern among a set of attributed relational graphs (ARGs), considering both the graphical structure and graph attributes. We propose a direct solution to extract the SAP with the maximal graph size without node enumeration. Given an initial graph template and a number of ARGs, we modify the graph template into the maximal SAP among the ARGs in an unsupervised fashion. The maximal SAP extraction is equivalent to learning a graphical model (i.e. an object model) from large ARGs (i.e. cluttered RGB/RGB-D images) for graph matching, which extends the concept of "unsupervised learning for graph matching." Furthermore, this study can be also regarded as the first known approach to formulating "maximal graph mining" in the graph domain of ARGs. Our method exhibits superior performance on RGB and RGB-D images.

People flow at a citywide level is in a mixed state with several basic patterns (e.g. commuting, working, commercial), and it is therefore difficult to extract useful information from such a mixture of patterns directly. In this paper, we proposed a novel tensor factorization approach to modeling city dynamics in a basic life pattern space (CitySpectral Space). To obtain the CitySpectrum, we utilized Non-negative Tensor Factorization (NTF) to decompose a people flow tensor into basic life pattern tensors, described by three bases i.e. the intensity variation among different regions, the time-of-day and the sample days. We apply our approach to a big mobile phone GPS log dataset (containing 1.6 million users) to model the fluctuation in people flow before and after the Great East Japan Earthquake from a CitySpectral perspective. In addition, our framework is extensible to a variety of auxiliary spatial-temporal data. We parametrize a people flow with a spatial distribution of the Points of Interest (POIs) to quantitatively analyze the relationship between human mobility and POI distribution. Based on the parametric people flow, we propose a spectral approach for a site-selection recommendation and people flow simulation in another similar area using POI distribution.

A large category model base can provide object-level knowledge for various perception tasks of the intelligent vehicle system. The automatic and efficient construction of such a model base is highly desirable but challenging. This paper presents a novel semi-supervised approach to discover possible prototype models of 3D object structures from the point cloud of a large and complex environment, given a limited number of seeds in an object category. Our method incrementally trains the models while simultaneously collecting object samples. Considering the bias problem of model learning caused by bias accumulation in a sample collection, we propose to gradually differentiate the standard category model into several sub-category models to represent different intra-category structural styles. Thus, new sub-categories are discovered and modeled, old models are improved, and redundant models for similar structures are deleted iteratively during the learning process. This multiple-model strategy provides several interactive options for the category boundary to deal with the bias problem. Experimental results demonstrate the effectiveness and high efficiency of our approach to model mining from “big point cloud data”.

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, accurately predicting human emergency behavior and their mobility will become the critical issue for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. In this paper, we build up a large human mobility database (GPS records of 1.6 million users over one year) and several different datasets to capture and analyze human emergency behavior and their mobility following the Great East Japan Earthquake and Fukushima nuclear accident. Based on our empirical analysis through these data, we find that human behavior and their mobility following large-scale disaster sometimes correlate with their mobility patterns during normal times, and are also highly impacted by their social relationship, intensity of disaster, damage level, government appointed shelters, news reporting, large population flow and etc. On the basis of these findings, we develop a model of human behavior that takes into account these factors for accurately predicting human emergency behavior and their mobility following large-scale disaster. The experimental results and validations demonstrate the efficiency of our behavior model, and suggest that human behavior and their movements during disasters may be significantly more predictable than previously thought.

An object model base that covers a large number of object categories is of great value for many computer vision tasks. As artifacts are usually designed to have various textures, their structure is the primary distinguishing feature between different categories. Thus, how to encode this structural information and how to start the model learning with a minimum of human labeling become two key challenges for the construction of the model base. We design a graphical model that uses object edges to represent object structures, and this paper aims to incrementally learn this category model from one labeled object and a number of casually captured scenes. However, the incremental model learning may be biased due to the limited human labeling. Therefore, we propose a new strategy that uses the depth information in RGBD images to guide the model learning for object detection in ordinary RGB images. In experiments, the proposed method achieves superior performance as good as the supervised methods that require the labeling of all target objects.

Although graph matching is a fundamental problem in pattern recognition, and has drawn broad interest from many fields, the problem of learning graph matching has not received much attention. In this paper, we redefine the learning of graph matching as a model learning problem. In addition to conventional training of matching parameters, our approach modifies the graph structure and attributes to generate a graphical model. In this way, the model learning is oriented toward both matching and recognition performance, and can proceed in an unsupervised fashion. Experiments demonstrate that our approach outperforms conventional methods for learning graph matching.

The building of an object-level knowledge base is the foundation of a new methodology for many perception tasks in artificial intelligence, and is an area that has received increasing attention in recent years. In this paper, we propose, for the first time, to mine category shape patterns directly from a large urban environment, thus constructing a category structure base. Conventionally, category patterns are learned from a large collection of object samples, but automatic object collection requires prior knowledge of category structures. To solve this chicken-and-egg problem, we learn shape patterns from raw segmentations, and then refine these segmentations based on the pattern knowledge. In the process, we solve two challenging problems of knowledge mining. First, as some categories have large intra-category structure variations, we design an entropy-based method to determine the structure variation for each category, in order to establish the correct range of sample collection. Second, because incorrect segmentation is unavoidable without prior knowledge, we propose a novel unsupervised method that uses a pattern competition strategy to identify and subtract shape patterns formed by incorrectly segmented objects. This ensures that shape patterns are meaningful at the object level. Experimental results demonstrated the effectiveness of the proposed method for category structure mining in a large urban environment.

The Great East Japan Earthquake and the Fukushima nuclear accident cause large human population movements and evacuations. Understanding and predicting these movements is critical for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. In this paper, we construct a large human mobility database that stores and manages GPS records from mobile devices used by approximately 1.6 million people throughout Japan from 1 August 2010 to 31 July 2011. By mining this enormous set of Auto-GPS mobile sensor data, the short-term and long-term evacuation behaviors for individuals throughout Japan during this disaster are able to be automatically discovered. To better understand and simulate human mobility during the disasters, we develop a probabilistic model that is able to be effectively trained by the discovered evacuations via machine learning technique. Based on our training model, population mobility in various cities impacted by the disasters throughout the country is able to be automatically simulated or predicted. On the basis of the whole database, developed model, and experimental results, it is easy for us to find some new features or population mobility patterns after the recent severe earthquake, tsunami and release of radioactivity in Japan, which are likely to play a vital role in future disaster relief and management worldwide.

For reasons of public security, an intelligent surveillance system that can cover a large, crowded public area has become an urgent need. In this article, we propose a novel laser-based system that can simultaneously perform tracking, semantic scene learning, and abnormality detection in a fully online and unsupervised way. Furthermore, these three tasks cooperate with each other in one framework to improve their respective performances. The proposed system has the following key advantages over previous ones: (1) It can cover quite a large area (more than 60×35m), and simultaneously perform robust tracking, semantic scene learning, and abnormality detection in a high-density situation. (2) The overall system can vary with time, incrementally learn the structure of the scene, and perform fully online abnormal activity detection and tracking. This feature makes our system suitable for real-time applications. (3) The surveillance tasks are carried out in a fully unsupervised manner, so that there is no need for manual labeling and the construction of huge training datasets. We successfully apply the proposed system to the JR subway station in Tokyo, and demonstrate that it can cover an area of 60×35m, robustly track more than 150 targets at the same time, and simultaneously perform online semantic scene learning and abnormality detection with no human intervention.

Multitarget tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This article presents a promising online system to deal with these problems. The novelty of this system is that laser and vision are integrated with tracking and online learning to complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers online for “possible interacting targets”. When the targets are in close proximity, the classifiers learned online are used alongside visual information to assist in tracking. Therefore, this mode of cooperation not only deals with various tough problems encountered in tracking, but also ensures that the entire process can be completely online and automatic. Experimental results demonstrate that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a good trade-off between tracking accuracy and the time-cost factor.

Abnormal activity detection plays a crucial role in surveillance applications, and a surveillance system that can perform robustly in the extremely crowded area has become an urgent need for public security. In this paper, we propose a novel laser-based system which can simultaneously perform the tracking, semantic scene learning and abnormality detection in the large and crowded environment. In our system, a novel abnormality detection model is proposed, and it considers and combines various factors that will influence human activity. Moreover, this model intensively investigate the relationship between pedestrians' social behaviors and their walking scenarios. We successfully applied the proposed system to the JR subway station of Tokyo, which can cover a 60×35m area, robustly track more than 180 targets at the same time and simultaneously perform the online semantic scene learning and abnormality detection with no human intervention.

Abnormal activity detection plays a crucial role in surveillance applications, and such system has become an urgent need for public security. In this paper, we propose a novel laser-based system, which can perform the online detection of abnormal activity with an unsupervised way. The proposed system has the following key features that make it advantageous over previous ones: (1) It can cover quite a large and crowded area, such as subway station, public square, intersection and etc. (2) The overall system can vary with time period, incrementally learn the behavior pattern of pedestrians and perform the fully online detection of abnormal activity. This feature makes our system be quite suitable for the real-time applications. (3) The abnormal activity detection is carried out with a fully unsupervised way, there is no need for manual labelling and constructing the huge training datasets. We successfully applied the proposed system into the JR subway station of Tokyo, which can cover a 60×35m area, track more 150 targets at the same time and simultaneously perform the robust detection of abnormal activity with no human intervention.

Learning the knowledge of scene structure and tracking a large number of targets are both active topics of computer vision in recent years, which plays a crucial role in surveillance, activity analysis, object classification and etc. In this paper, we propose a novel system which simultaneously performs the Learning-Semantic-Scene and Tracking, and makes them supplement each other in one framework. The trajectories obtained by the tracking are utilized to continually learn and update the scene knowledge via an online un-supervised learning. On the other hand, the learned knowledge of scene in turn is utilized to supervise and improve the tracking results. Therefore, this “adaptive learning-tracking loop” can not only perform the robust tracking in high density crowd scene, dynamically update the knowledge of scene structure and output semantic words, but also ensures that the entire process is completely automatic and online. We successfully applied the proposed system into the JR subway station of Tokyo, which can dynamically obtain the semantic scene structure and robustly track more than 150 targets at the same time.

Multi-target tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This paper presents a promising tracking system to deal with these problems. The novelty of this system is that laser and vision, tracking and learning are integrated and can complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers for the “possible interacting targets”. When the targets are in close proximity, the learned classifiers and visual information are used to assist in tracking. Therefore, this mode of co-operation between them not only deals with various tough problems encountered in the tracking, but also ensures that the entire process can be completely on-line and automatic. Experimental results demonstrated that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a perfect trade-off between tracking accuracy and time-cost.

In this paper, we present a Probabilistic Detection-based Particle Filter (PD-PF) for tracking a variable number of interacting targets. When the objects do not interact with each other, our method performs like the deterministic detection-base methods. When the objects are in close proximity, the interactions and occlusions are modelled by a mixed proposal constructed by probabilistic detections and information from dynamic models. Specially, prior of detection-reliability minimizes the influence of non-detection or false alarm in the tracking. Moreover, we run independent PD-PF for each target, such that particles are sampled in a small state space, thus our method not only obtains a better approximation of posterior than joint particle filter or independent particle filter when interactions occur, but also has an acceptable computational complexity. Different evaluations demonstrate the validity and efficiency of the proposed method. 1

Successful multi-target tracking requires locating the targets and labeling their identities. For the laser based tracking system, the latter becomes significantly more challenging when the targets frequently interact with each other. This paper presents a novel on-line supervised learning based method for tracking interacting targets with laser scanner. When the targets do not interact with each other, we collect samples and train a classifier for each target. When the targets are in close proximity, we use these classifiers to assist in tracking. Different evaluations demonstrate that this method has a better tracking performance than previous methods when interactions occur, and can maintain correct tracking under various complex tracking situations.