Similarity search is a fundamental research problem with broad applications in various research fields, including data mining, information retrieval, and machine learning. The core idea of similarity search is to find the most similar data sample of given query items, based on a specific similarity metric with the highest similarity score with all the search candidates in a large-scale database. It may suffer from a prohibitive computation cost and storage cost, which motivates us to design effective and fast similarity search algorithms in various scenarios. However, data missing is unavoidable in real-world scenarios, which results in a less accurate similarity score and further leads to an inaccurate similarity matrix. Therefore, obtaining an accurate similarity matrix is non-trivial when there are incomplete observations. To solve this problem, we propose a similarity matrix calibration method to estimate a high-quality similarity matrix and further provide a better similarity search performance. Firstly, we propose an objective function to minimize the difference between the initial inaccurate similarity matrix and the optimal estimated similarity matrix, where the inherent symmetric and Positive Semi-Definiteness (PSD) properties are utilized as the constraint to guide the calibration process. Then, we design an effective algorithm with high efficiency to provide a high-quality similarity matrix that approximates the ground-truth similarity matrix. Theoretical analysis demonstrates the efficiency guarantee of our proposed method, and extensive experimental results on real-world datasets verify the effectiveness and efficiency of the proposed method on the similarity matrix calibration task and the downstream similarity search task.

Abstract Trajectories serve as a cornerstone of intelligent transportation systems, playing an important role in many applications such as traffic flow prediction, route planning, and urban management. However, the availability of such data is limited due to privacy issues, ethical concerns, and the high cost associated with infrastructure deployment. In recent years, rapidly developing generative models such as Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and Diffusion Models (DMs) have demonstrated strong capabilities in learning complex data distributions and generating synthetic data, thereby alleviating the data accessibility issue. In this survey, we systematically review the existing literature on deep generative models that address the problem of trajectory generation. First, we classify the existing literature into two categories: unconditional and conditional trajectory generation. In unconditional generation, trajectories are generated without contextual constraints, whereas conditional generation incorporates several important factors such as road network topology, time of day, and user preferences to guide the trajectory generation process. Then, for each category, we further classify the literature into three methodological types, including VAEs, GANs, and DMs, and analyze how these models address key challenges under different settings. Finally, we discuss promising directions for future research and hope to inspire further advances in trajectory generation.

Friend recommendation is an important real-world application in Location-based Social Networks (LBSN), helping users discover potential friends and enhance their overall happiness. LBSN mainly comprises two distinct data structures: spatio-temporal data for human mobility and graph data for social networks. These two data structures make it challenging to model the complex relationships between them, which are essential for comprehensively understanding users’ lives. Previous studies have either modeled user trajectories and social networks separately or used classical simple graph-based methods, where a simple edge links only two nodes, failing to capture the multiple relationships inherent in LBSN. Furthermore, most studies have relied on Euclidean space to train their graph models, which could result in significant distortion because of tree-like social network data structure. To address these limitations, we propose a novel heterogeneous LBSN hypergraph that represents user check-in records and continuous trajectories—comprising multiple Points of Interest (POI)—as hyperedges, enabling the representation of complex spatio-temporal relationships. This approach enables us to link multiple nodes of different types by hyperedges and use hyperbolic spaces to create more efficient graph representations. Additionally, we devise a new type-specific attention mechanism for our Heterogeneous Hyperbolic Hypergraph Neural Network (H 3 GNN), which is end-to-end trainable and employs supervised contrastive learning to learn hypergraph node embeddings for the subsequent friend recommendation task with the help of hyperbolic space. Finally, our model H 3 GNN achieves better results than existing methods on six real-world city datasets, and our ablation studies demonstrate the effectiveness of each component. Additionally, our experiments indicate that H 3 GNN requires less data storage and training time compared to previous methods.

Analyzing the causal impact of various government-related policies on the epidemic spread is of critical importance. This article aims to investigate the problem of assessing the causal effects of different COVID-19-related policies on the USA epidemic spread in different counties at any given time period, while eliminating biased interference from unobserved confounders (e.g., the vigilance of residents). However, the infection outcome of each region is influenced not only by its own confounding factors but also by policy interventions implemented in neighboring regions. Furthermore, the government policy index may exhibit a time-delay influence on outbreak dynamics. To this end, we implement observational data about different COVID-19-related policies (treatment) and outbreak dynamics (outcome) across different U.S. counties over time and develop a causal framework that learns the representations of time-varying confounders to tackle the aforementioned issues. More specifically, we employ one recurrent structure to capture the accumulative effects stemming from the policy history and then utilize hypergraph neural network to model the interactions among spatial regions. Our experimental results demonstrate the effectiveness of the proposed framework in quantifying the causal impact of different policy types on epidemics. Compared with baseline methods, our assessment provides valuable insights for future policy-making endeavors.

Maritime transportation, vital for nearly 90% of global trade, necessitates precise vessel trajectory prediction for safety and efficiency. Although the Automatic Identification System (AIS) provides a comprehensive data source, how to model these multi-modal and heterogeneous time-varying sequences (such as vessels’ kinetic information and ocean weather factors) poses a formidable challenge. Moreover, most existing approaches are limited by the confined scope of vessel trajectory modeling, making it impossible to consider the unique characteristics of maritime transportation system. To tackle these challenges, we propose a novel framework called AISFuser to i) encode unique maritime traffic network into graphical representations, and ii) introduce the heterogeneity into multi-modal temporal embeddings through Self-Supervised Learning (SSL). Specifically, our AISFuser is constructed by combining an attention-based graph block with a transformer network to encode information across space and time, respectively. In terms of temporal dimension, one SSL auxiliary task is also designed to enhance the heterogeneity of temporal representations and supplement the main vessel prediction task. We validate the effectiveness of the proposed AISFuser on a real-world AIS dataset. Extensive experimental results demonstrate that our method can forecast multiple attributes of vessel trajectory for over 10 hours into the future, outperforming competitive baselines.

Training models on spatio-temporal (ST) data poses an open problem due to the complicated and diverse nature of the data itself, and it is challenging to ensure the model's performance directly trained on the original ST data. While limiting the variety of training data can make training easier, it can also lead to a lack of knowledge and information for the model, resulting in a decrease in performance. To address this challenge, we presented an innovative paradigm that incorporates three separate forms of curriculum learning specifically targeting from spatial, temporal, and quantile perspectives. Furthermore, our framework incorporates a stacking fusion module to combine diverse information from three types of curriculum learning, resulting in a strong and thorough learning process. We demonstrated the effectiveness of this framework with extensive empirical evaluations, highlighting its better performance in addressing complex ST challenges. We provided thorough ablation studies to investigate the effectiveness of our curriculum and to explain how it contributes to the improvement of learning efficiency on ST data.

Spatiotemporal time series forecasting plays a key role in a wide range of real-world applications. While significant progress has been made in this area, fully capturing and leveraging spatiotemporal heterogeneity remains a fundamental challenge. Therefore, we propose a novel Heterogeneity-Informed Meta-Parameter Learning scheme. Specifically, our approach implicitly captures spatiotemporal heterogeneity through learning spatial and temporal embeddings, which can be viewed as a clustering process. Then, a novel spatiotemporal meta-parameter learning paradigm is proposed to learn spatiotemporal-specific parameters from meta-parameter pools, which is informed by the captured heterogeneity. Based on these ideas, we develop a Heterogeneity-Informed Spatiotemporal Meta-Network (HimNet) for spatiotemporal time series forecasting. Extensive experiments on five widely-used benchmarks demonstrate our method achieves state-of-the-art performance while exhibiting superior interpretability. Our code is available at https://github.com/XDZhelheim/HimNet.

Maritime traffic management increasingly relies on vessel position information provided by terrestrial and satellite networks of the Automatic Identification System (AIS). Unfortunately, the problem of missing AIS data can lead to long-term gaps in vessel trajectory, raising corresponding security concerns regarding collision risks and illicit activities. Existing imputation approaches are often constrained by vehicle-based low-sampling trajectories, hindering their ability to address unique characteristics of maritime transportation systems and long-term missing scenarios. To tackle these challenges, we propose a novel generative framework for long-term vessel trajectory imputation. Our framework considers irregular tracks of vessels, which differ from those of cars due to the absence of a structured road network, and ensures the continuity of multi-point imputed trajectories. Specifically, we first utilize a pre-trained trajectory embedding block to capture patterns of vessel movements. Subsequently, we introduce a diffusion-based model for generating missing trajectories, where observed trajectory modeling with transformer encoding architecture and embeddings of both historical vessel trajectory and external factors serve as conditional information. In particular, we design a physics-guided discriminator in the training stage, which imposes kinematic constraints between locations and angles to improve the continuity of the imputed trajectories. Comprehensive experiments and analysis on a real-world AIS dataset confirm the effectiveness of our proposed approach.

Tensor time series (TTS) data, a generalization of one-dimensional time series on a high-dimensional space, is ubiquitous in real-world applications. Compared to modeling time series or multivariate time series, which has received much attention and achieved tremendous progress in recent years, tensor time series has been paid less effort. However, properly coping with the TTS is a much more challenging task, due to its high-dimensional and complex inner structure. In this article, we start by revealing the structure of TTS data from afn statistical view of point. Then, in line with this analysis, we perform T ensor T ime S eries forecasting via a proposed Multi-way Norm alization ( TTS-Norm ), which effectively disentangles multiple heterogeneous low-dimensional substructures from the original high-dimensional structure. Finally, we design a novel objective function for TTS forecasting, accounting for the numerical heterogeneity among different low-dimensional subspaces of TTS. Extensive experiments on two real-world datasets verify the superior performance of our proposed model. 1

Multivariate time-series (MTS) forecasting is a paramount and fundamental problem in many real-world applications. The core issue in MTS forecasting is how to effectively model complex spatial-temporal patterns. In this paper, we develop a adaptive, interpretable and scalable forecasting framework, which seeks to individually model each component of the spatial-temporal patterns. We name this framework SCNN, as an acronym ofStructuredComponent-basedNeuralNetwork. SCNN works with a pre-defined generative process of MTS, which arithmetically characterizes the latent structure of the spatial-temporal patterns. In line with its reverse process, SCNN decouples MTS data into structured and heterogeneous components and then respectively extrapolates the evolution of these components, the dynamics of which are more traceable and predictable than the original MTS. Extensive experiments are conducted to demonstrate that SCNN can achieve superior performance over state-of-the-art models on three real-world datasets. Additionally, we examine SCNN with different configurations and perform in-depth analyses of the properties of SCNN.

Urban time series data forecasting featuring significant contributions to sustainable development is widely studied as an essential task of the smart city. However, with the dramatic and rapid changes in the world environment, the assumption that data obey Independent Identically Distribution is undermined by the subsequent changes in data distribution, known as concept drift, leading to weak replicability and transferability of the model over unseen data. To address the issue, previous approaches typically retrain the model, forcing it to fit the most recent observed data. However, retraining is problematic in that it leads to model lag, consumption of resources, and model re-invalidation, causing the drift problem to be not well solved in realistic scenarios. In this study, we propose a new urban time series prediction model for the concept drift problem, which encodes the drift by considering the periodicity in the data and makes on-the-fly adjustments to the model based on the drift using a meta-dynamic network. Experiments on real-world datasets show that our design significantly outperforms state-of-the-art methods and can be well generalized to existing prediction backbones by reducing their sensitivity to distribution changes.

With the rapid development of the Intelligent Transportation System (ITS), accurate traffic forecasting has emerged as a critical challenge. The key bottleneck lies in capturing the intricate spatio-temporal traffic patterns. In recent years, numerous neural networks with complicated architectures have been proposed to address this issue. However, the advancements in network architectures have encountered diminishing performance gains. In this study, we present a novel component called spatio-temporal adaptive embedding that can yield outstanding results with vanilla transformers. Our proposed Spatio-Temporal Adaptive Embedding transformer (STAEformer) achieves state-of-the-art performance on five real-world traffic forecasting datasets. Further experiments demonstrate that spatio-temporal adaptive embedding plays a crucial role in traffic forecasting by effectively capturing intrinsic spatio-temporal relations and chronological information in traffic time series.

To understand human mobility following the typhoon, analyzing the causal impact of extreme typhoon weather on human mobility is important for disaster emergency management. However, the unobserved confounders (e.g., the characteristic of each region) correlate with the strength of typhoon weather and also affect human mobility during typhoon, which may generate biased influences on the causal analysis process. Besides, these confounders may be time-varying following the dynamic movements of typhoon. In this work, we develop a neural network-based continuous causal effect estimation framework to mitigate the interference from (unobserved) confounders and assess the continuous causal responses of typhoon-related weather (treatment) on several types of human mobility (outcome) across different counties at any given period. To this end, we integrate the big data from two huge typhoons in Japan (i.e., Typhoon Faxai and Hagibis) and leverage multiple sources of covariates (i.e., residents' vigilance and basic mobility patterns) from different counties to learn the representations of time-varying confounders. The experimental results indicate the effectiveness of our proposed framework in capturing the confounders for quantifying the causal impact of extreme weather during the typhoon process, compared with several existing causal studies.

Today, thanks to the rapid developing mobile and sensor networks in IoT (Internet of Things) systems, spatio-temporal big data are being constantly generated. They have brought us a data-driven possibility to sense and understand crowd mobility on a city scale. A fundamental task towards the next-generation mobility services, such as Intelligent Transportation Systems (ITS), Mobility-as-a-Service (MaaS), is spatio-temporal predictive modeling of the geo-sensory signals. There is a recent line of research leveraging deep learning techniques to boost the forecasting performance on such tasks. While simulating the regularity of mobility behaviors (e.g., routines, periodicity) in a more sophisticated way, the existing studies ignore an important part of urban activities, i.e., events. Including holidays, extreme weathers, pandemic, accidents, various urban events happen from time to time and cause non-stationary phenomena, which by nature make the spatio-temporal forecasting task challenging. We thereby envision an event-aware urban mobility prediction model that is capable of fast adapting and making reliable predictions in different scenarios, which is crucial to decision making towards emergency response and urban resilience.

Human mobility nowcasting is a fundamental research problem for intelligent transportation planning, disaster responses and management, etc. In particular, human mobility under big disasters such as hurricanes and pandemics deviates from its daily routine to a large extent, which makes the task more challenging. Existing works mainly focus on traffic or crowd flow prediction in normal situations. To tackle this problem, in this study, disaster-related Twitter data is incorporated as a covariate to understand the public awareness and attention about the disaster events and thus perceive their impacts on the human mobility. Accordingly, we propose a Meta-knowledge-Memorizable Spatio-Temporal Network (MemeSTN), which leverages memory network and meta-learning to fuse social media and human mobility data. Extensive experiments over three real-world disasters including Japan 2019 typhoon season, Japan 2020 COVID-19 pandemic, and US 2019 hurricane season were conducted to illustrate the effectiveness of our proposed solution. Compared to the state-of-the-art spatio-temporal deep models and multivariate-time-series deep models, our model can achieve superior performance for nowcasting human mobility in disaster situations at both country level and state level.

Multi-variate time series (MTS) data is a ubiquitous class of data abstraction in the real world. Any instance of MTS is generated from a hybrid dynamical system and their specific dynamics are usually unknown. The hybrid nature of such a dynamical system is a result of complex external attributes, such as geographic location and time of day, each of which can be categorized into either spatial attributes or temporal attributes. Therefore, there are two fundamental views which can be used to analyze MTS data, namely the spatial view and the temporal view. Moreover, from each of these two views, we can partition the set of data samples of MTS into disjoint forecasting tasks in accordance with their associated attribute values. Then, samples of the same task will manifest similar forthcoming pattern, which is less sophisticated to be predicted in comparison with the original single-view setting. Considering this insight, we propose a novel multi-view multi-task (MVMT) learning framework for MTS forecasting. Instead of being explicitly presented in most scenarios, MVMT information is deeply concealed in the MTS data, which severely hinders the model from capturing it naturally. To this end, we develop two kinds of basic operations, namely task-wise affine transformation and task-wise normalization, respectively. Applying these two operations with prior knowledge on the spatial and temporal view allows the model to adaptively extract MVMT information while predicting. Extensive experiments on three datasets are conducted to illustrate that canonical architectures can be greatly enhanced by the MVMT learning framework in terms of both effectiveness and efficiency. In addition, we design rich case studies to reveal the properties of representations produced at different phases in the entire prediction procedure.

Predicting the density and flow of the crowd or traffic at a citywide level becomes possible by using the big data and cutting-edge AI technologies. It has been a very significant research topic with high social impact, which can be widely applied to emergency management, traffic regulation, and urban planning. In particular, by meshing a large urban area to a number of fine-grained mesh-grids, citywide crowd and traffic information in a continuous time period can be represented with 4D tensor (Timestep, Height, Width, Channel). Based on this idea, a series of methods have been proposed to address grid-based prediction for citywide crowd and traffic. In this study, we revisit the density and in-out flow prediction problem and publish a new aggregated human mobility dataset generated from a real-world smartphone application. Comparing with the existing ones, our dataset holds several advantages including large mesh-grid number, fine-grained mesh size, and high user sample. Towards this large-scale crowd dataset, we propose a novel deep learning model called DeepCrowd by designing pyramid architectures and high-dimensional attention mechanism based on Convolutional LSTM. Lastly, thorough and comprehensive performance evaluations are conducted to demonstrate the superiority of the proposed DeepCrowd comparing to multiple state-of-the-art methods.

Recently, forecasting the crowd flows has become an important research topic, and plentiful technologies have achieved good performances. As we all know, the flow at a citywide level is in a mixed state with several basic patterns (e.g., commuting, working, and commercial) caused by the city area functional distributions (e.g., developed commercial areas, educational areas and parks). However, existing technologies have been criticized for their lack of considering the differences in the flow patterns among regions since they want to build only one comprehensive model to learn the mixed flow tensors. Recognizing this limitation, we present a new perspective on flow prediction and propose an explainable framework named ST-ExpertNet, which can adopt every spatial-temporal model and train a set of functional experts devoted to specific flow patterns. Technically, we train a bunch of experts based on the Mixture of Experts (MoE), which guides each expert to specialize in different kinds of flow patterns in sample spaces by using the gating network. We define several criteria, including comprehensiveness, sparsity, and preciseness, to construct the experts for better interpretability and performances. We conduct experiments on a wide range of real-world taxi and bike datasets in Beijing and NYC. The visualizations of the expert's intermediate results demonstrate that our ST-ExpertNet successfully disentangles the city's mixed flow tensors along with the city layout, e.g., the urban ring road structure. Different network architectures, such as ST-ResNet, ConvLSTM, and CNN, have been adopted into our ST-ExpertNet framework for experiments and the results demonstrates the superiority of our framework in both interpretability and performances.

Travel time estimation from GPS trips is of great importance to order duration, ridesharing, taxi dispatching, etc. However, the dense trajectory is not always available due to the limitation of data privacy and acquisition, while the origin-destination (OD) type of data, such as NYC taxi data, NYC bike data, and Capital Bikeshare data, is more accessible. To address this issue, this paper starts to estimate the OD trips travel time combined with the road network. Subsequently, aMulti-taskWeaklySupervisedLearning Framework forTravelTimeEstimation (MWSL-TTE) has been proposed to infer transition probability between roads segments, and the travel time on road segments and intersection simultaneously. Technically, given an OD pair, the transition probability intends to recover the most possible route. And then, the output of travel time is equal to the summation of all segments’ and intersections’ travel time in this route. A novel route recovery function has been proposed to iteratively maximize the current routes’ co-occurrence probability, and minimize the discrepancy between routes’ probability distribution and the inverse distribution of routes’ estimation loss. Moreover, the expected log-likelihood function based on a weakly-supervised framework has been deployed in optimizing the travel time from road segments and intersections concurrently. We conduct experiments on a wide range of real-world taxi datasets in Xi’an and Chengdu and demonstrate our method's effectiveness on route recovery and travel time estimation.

A core component of the Intelligent Transportation System (ITS) is road network, which forms the most basic transport infrastructure, and becomes widely applied in many traffic applications. In most traffic models, the spatial representation of road network is learned only through static graph connection while dynamic driver preference and traffic conditions in the real world are ignored. Therefore, in this paper, a novel trajectory-based road network representation is proposed. By mining vehicle trajectories, our proposed method can learn dynamic route choice through embeddings of each road in a next-hop prediction model. Then road correlations are calculated by the embeddings to build a latent correlation graph that can be applied in various traffic-related applications. Extensive experiment results prove the effectiveness and rationality of our proposed approach.

COVID-19 has dramatically changed people’s mobility patterns. This report aims to analyze the impact of COVID-19 on people’s mobility through statistics and comparing the visits of POIs (Point-Of-Interests) in New York State in 2019 and 2020. The report uses data from SafeGraph, which is a data company. The raw data contains POI visits across the United States in 2019 and 2020. Considering the analysis size and difficulty of the data, POI visits from New York State are extracted for analysis, and POI locations are classified according to the tags provided by the source data. The scale of analysis is from macro to micro, and they are the total POI visits data of New York State based on different ways in 2019 and 2020, the POI visits of CBG (Census Block Group) division in New York City, and three representative POI samples to do individual analysis. The analysis methods are: (1) use line plot and bar plot statistics to compare the trends of POI visits data from 2019 to 2020, and (2) make the spatial visualization comparison, which includes grid map, scatter map, heatmap, and OD map, between the first peak of epidemic impact in the first full week of April 2019 and April 2020, and the scope is narrowed to New York City. Wherein the OD maps are drawn based on the CBG division. Compared to related work, the analysis object includes CBG, categories, and individual POI. In addition, the analysis method combines statistical graphs and spatial visualizations and explores the policy impact of the New York City government. This report adopts more multidimensional analysis methods and objects to improve the comprehensiveness and reliability of the analysis content.

Citywide crowd prediction can be of great importance for emergency management, traffic regulation, and urban planning. By meshing a large urban area to a number of fine-grained mesh-grids as illustrated in Fig. 1 , citywide crowd in a continuous time period can be represented with a four-dimensional tensor ${\mathbb{R}^{Timestep{\text{ }}p \times {\text{ }}Height{\text{ }} \times {\text{ }}Width{\text{ }} \times {\text{ }}Channel{\text{ }}}}$ in an analogous manner to video data, where each Timestep can be seen as one video frame, Height , Width is two-dimensional index for mesh-grids, and each Channel stores an aggregated scalar value for each mesh-grid. Specifically, given historical observations of crowd density and in-out flow x d = d 1 ,…,d t , xf = f 1 ,…, f t at timestamp t , we aim to build prediction models for the next-step density and in-out flow y d = d t +1, yf = f t +1, where y d means how many people will be in each mesh-grid at the next timestamp, and yf means how many people will flow into or out from each mesh-grid in next time interval. Al-though many deep models [1] – [6] have been proposed to address such tasks, their actual effects are still not well validated on large-scale and high-quality datasets. The datasets used in most of the works so far are originally generated based on taxi or bicycle trip data, which don’t cover and reflect the citywide crowd density and flow. Thus, we first publish new crowd flow data called BousaiTYO and BousaiOSA [7] . These new datasets are created using the GPS log data collected from a popular smartphone app of Yahoo! Japan Corporation, which can well reflect the real-world crowd flow in Tokyo and Osaka. As shown by Table 1 , our dataset has: (1) larger spatial area; (2) finer mesh size; (3) higher user sample.

Traffic state prediction is essential in Intelligent Transportation Systems for surveillance, management, and daily commuting. For developing high-accuracy prediction models, real-world traffic state datasets are necessary for training model parameters and evaluating prediction results. However, limited by the existing traffic collection devices, most of the current open datasets for traffic state prediction cannot obtain accurate traffic flow information. In contrast, some datasets directly use detection devices in freeway systems, so they cannot reflect complex urban traffic states. Therefore, a dataset from advanced devices that can record the flow from point to point on an urban road network attracts more attention and drives the progress of research on traffic state prediction models. To deal with the above issues, we introduce a Suburban Traffic Flow dataset using Geomagnetic sensors, or STF-G dataset, constructed for traffic flow prediction. The STF-G dataset consists of 2.5 billion vehicle driving scenarios and 319 corresponding geomagnetic sensors. The data was collected over 20 months and processed with two regional road graphs. We also do the Benchmark experiments in STF-G for analyzing and evaluating the performance of graph neural network models in traffic flow prediction and compare them to the other datasets with the same baseline.

Friend recommendation from human mobility is a vital real-world application of location-based social networks (LBSN). It is necessary to recognize patterns from human mobility to assist friend recommendation because previous works have shown complex relations between them. However, most of previous works either modelled social networks and user trajectories separately, or only used classical simple graph-based methods with an edge linking two nodes that cannot fully model the complex data structure of LBSN. Inspired by the fact that hyperedges can connect multiple nodes of different types, we model user trajectories and check-in records as hyperedges in a novel heterogeneous LBSN hypergraph to represent complex spatio-temporal information. And then, we design a type-specific attention mechanism for an end-to-end trainable heterogeneous hypergraph neural network (HHGNN) with supervised contrastive learning, which can learn hypergraph node embedding for the next friend recommendation task. At last, our model HHGNN outperforms the state-of-the-art methods on four real-world city datasets, while ablation studies also confirm the effectiveness of each model part.

Finding out similar regions between cities is important to a variety of real-world applications, such as point-of-interest recommendations, site selection, and travel guidance. With the help of the increasing number of location-based social network users, we can measure the intercity similarity from a new perspective of spatiotemporal characteristics of human mobility. In this paper, we developed an interactive intercity regional similarity explorer (IRSE) that 1) visualizes regional spatiotemporal human mobility features, 2) searches similar region candidates in the target city, and 3) explores the regional similarity from different views in a quantitative and illustrative way. In this paper, we show how our system can be useful in exploring regional similarity across cities in the world by use cases, which will interest various users from different countries in the demonstration session. Demo available at: https://bit.ly/3OjwnGu

Knowing "what is happening" and "what will happen" of the mobility in a city is the building block of a data-driven smart city system. In recent years, mobility digital twin that makes a virtual replication of human mobility and predicting or simulating the fine-grained movements of the subjects in a virtual space at a metropolitan scale in near real-time has shown its great potential in modern urban intelligent systems. However, few studies have provided practical solutions. The main difficulties are four-folds: 1) the daily variation of human mobility is hard to model and predict; 2) the transportation network enforces a complex constraints on human mobility; 3) generating a rational fine-grained human trajectory is challenging for existing machine learning models; and 4) making a fine-grained prediction incurs high computational costs, which is challenging for an online system. Bearing these difficulties in mind, in this paper we propose a two-stage human mobility predictor that stratifies the coarse and fine-grained level predictions. In the first stage, to encode the daily variation of human mobility at a metropolitan level, we automatically extract citywide mobility trends as crowd contexts and predict long-term and long-distance movements at a coarse level. In the second stage, the coarse predictions are resolved to a fine-grained level via a probabilistic trajectory retrieval method, which offloads most of the heavy computations to the offline phase. We tested our method using a real-world mobile phone GPS dataset in the Kanto area in Japan, and achieved good prediction accuracy and a time efficiency of about 2 min in predicting future 1h movements of about 220K mobile phone users on a single machine to support more higher-level analysis of mobility prediction.

Predicting the density and flow of the crowd at a citywide level is significant for city management. By meshing a large urban area to a number of fine-grained mesh-grids, citywide crowd and traffic information in a continuous time period can be represented with 4D tensor (Timestep, Height, Width, Channel). Based on this, we revisit the density and in-out flow prediction problem and publish a new aggregated human mobility dataset generated from a real-world smartphone application. Compared with the existing ones, our dataset has larger mesh-grid number, finer-grained mesh size, and higher user sample. Towards such kind of large-scale crowd dataset, we propose a novel deep learning model called DeepCrowd by designing pyramid architectures and high-dimensional attention mechanism based on Convolutional LSTM. Both the datasets and codes are made available at https://github.com/deepkashiwa20/DeepCrowd.

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g., police) and public service operators (e.g., subway/bus operator) to protect people’s safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the “deep” trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent, which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

Nowadays, with the rapid development of IoT (Internet of Things) and CPS (Cyber-Physical Systems) technologies, big spatiotemporal data are being generated from mobile phones, car navigation systems, and traffic sensors. By leveraging state-of-the-art deep learning technologies on such data, urban traffic prediction has drawn a lot of attention in AI and Intelligent Transportation System community. The problem can be uniformly modeled with a 3D tensor (T, N, C), where T denotes the total time steps, N denotes the size of the spatial domain (i.e., mesh-grids or graph-nodes), and C denotes the channels of information. According to the specific modeling strategy, the state-of-the-art deep learning models can be divided into three categories: grid-based, graph-based, and multivariate time-series models. In this study, we first synthetically review the deep traffic models as well as the widely used datasets, then build a standard benchmark to comprehensively evaluate their performances with the same settings and metrics. Our study named DL-Traff is implemented with two most popular deep learning frameworks, i.e., TensorFlow and PyTorch, which is already publicly available as two GitHub repositories https://github.com/deepkashiwa20/DL-Traff-Grid and https://github.com/deepkashiwa20/DL-Traff-Graph. With DL-Traff, we hope to deliver a useful resource to researchers who are interested in spatiotemporal data analysis.

Forecasting incident occurrences (e.g. crime, EMS, traffic accident) is a crucial task for emergency service providers and transportation agencies in performing response time optimization and dynamic fleet management. However, such events are by nature rare and sparse, which causes the label imbalance problem and inferior performance of models relying on data sufficiency. The existing studies circumvent, instead of truly solving, this issue by defining the incident prediction problem in a coarse-grained temporal (e.g. daily) setting, which leaves the proposed models unrobust to fine-grained dynamics and trivial for the real-world decision making. In this paper, we tackle the temporally fine-grained incident prediction problem in a sparse setting by explicitly exploiting the behind-the-scene chainlike triggering mechanism. Moreover, this chain effect roots in multiple domains (i.e. spatial, categorical), which further entangles with the temporal dimension and happens to be time-variant. To be specific, we propose a novel deep learning framework, namely Spatio-Temporal-Categorical Graph Neural Networks (STC-GNN), to handle the multidimensional and dynamic chain effect for performing fine-grained multi-incident co-prediction. Extensive experiments on three real-world city-level incident datasets verify the insightfulness of our perspective and effectiveness of the proposed model.

Forecasting regional ambulance demand plays a fundamental part in dynamic fleet allocation and redeployment. This topic has been gaining increasing significance, as virtually every country is experiencing an aging population, with generally higher level of vulnerability and demand for the emergency medical service (EMS). Although exploring the spatial and temporal correlations in EMS historical records, the existing methods principally consider the former time-invariant, which does not necessarily hold in reality. Moreover, this assumption ignores the fact that the behind-the-scenes dynamics are people, whose demographic profiles and activity patterns could be determinants of regional EMS demands. In this paper, we are therefore motivated to mine the collective daily routines in human mobility, to further represent the evolving spatial correlations. Particularly, we model profiled mobility groups as multiple random walkers and propose a novel bicomponent neural network, including a heterogeneous multi-graph convolution layer and spatio-temporal interlacing attention module, to perform the prediction task. Experimental results on the real-world data verify the effectiveness of introducing dynamic human mobility and the advantage of our approach over the state-of-the-art models.

Multi-variate time series (MTS) data is a ubiquitous class of data abstraction in the real world. Any instance of MTS is generated from a hybrid dynamical system with their specific dynamics normally unknown. The hybrid nature of such a dynamical system is a result of complex external impacts, which can be summarized as high-frequency and low-frequency from the temporal view, or global and local if we take the spatial view. These impacts also determine the forthcoming development of MTS making them paramount to capture in a time series forecasting task. However, conventional methods face intrinsic difficulties in disentangling the components yielded by each kind of impact from the raw data. To this end, we propose two kinds of normalization modules -- temporal and spatial normalization -- which separately refine the high-frequency component and the local component underlying the raw data. Moreover, both modules can be readily integrated into canonical deep learning architectures such as Wavenet and Transformer. Extensive experiments on three datasets are conducted to illustrate that, with additional normalization modules, the performance of the canonical architectures can be enhanced by a large margin in the application of MTS and achieves state-of-the-art results compared with existing MTS models.

Transportation demand forecasting is a topic of large practical value. However, the model that fits the demand of one transportation by only considering the historical data of its own could be vulnerable since random fluctuations could easily impact the modeling. On the other hand, common factors like time and region attribute, drive the evolution demand of different transportation, leading to a co-evolving intrinsic property between different kinds of transportation. In this work, we focus on exploring the co-evolution between different modes of transport, e.g., taxi demand and shared-bike demand. Two significant challenges impede the discovery of the co-evolving pattern: (1) diversity of the co-evolving correlation, which varies from region to region and time to time. (2) Multi-modal data fusion. Taxi demand and shared-bike demand are time-series data, which have different representations with the external factors. Moreover, the distribution of taxi demand and bike demand are not identical. To overcome these challenges, we propose a novel method, known as co-evolving spatial temporal neural network (CEST). CEST learns a multi-view demand representation for each mode of transport, extracts the co-evolving pattern, then predicts the demand for the target transportation based on multi-scale representation, which includes fine-scale demand information and coarse-scale pattern information. We conduct extensive experiments to validate the superiority of our model over the state-of-art models.

Nowadays, GPS devices have increased explosively and produced huge amounts of trajectory data related to people's outgoing. Through those big location data, many researches aim to analyze human mobility for urban development, such as human movement prediction/modeling, POI (Point-Of-Interest) recommendation. However, trajectory data only contains timestamp and location information. The intention of human movement is not explicit so that it is hard to understand why people go to somewhere. The intention prior to the activity could be of great significance for analyzing and predicting human mobility, which has not been taken into consideration by the existing researches until the present. Thus, in this study, we propose a brand-new concept called human intentional mobility, aiming to employ intention information to predict people's outgoing. We carefully utilize user's search query to sense his intention as well as the intensity. For instance, if a user searches a certain POI for many times in a short period, it will represent a relatively high intention to go there. Then, to fully utilize this intention representation for predicting whether user will visit searched POI or not, we specially design Dual Sequential Interaction Network (DualSIN) as a novel and unique deep-learning model, which can effectively capture the sophisticated interactions among two kinds of sequential information (i.e., search sequence and mobility sequence) and typical categorical information (i.e., user attributes). Last, we evaluate our model on real-world dataset collected from Yahoo! Japan portal application, and demonstrate that it can achieve superior satisfactory performances to the-state-of-the-art models on multiple POI search queries.

Extracting identifiable information from human trajectories is a fundamental task in many location-based services (LBS). However, various mobility patterns underlain in human trajectories are difficult to model by existing models. Moreover, we could hardly define a clear user set for user identification because the set of users are dynamic and changing everyday. Bearing these in mind, we apply a deep multiple instance learning method to handle the multimodal mobility patterns in a weak-supervised learning way, and address the dynamic user set problems via a pairwise loss with negative sampling. We utilize a multi-head attention mechanism to automatically extract multiple aspects and match the corresponding information between query trajectories and historical trajectories. Our method shows a good identification accuracy on three human GPS trajectory data sets comparing with baseline methods.

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g. police) and public service operators (e.g. subway/bus operator) to protect people's safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the deep trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly-complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

Globalization leads to more and more outbound behavior. Understanding these behavior is of great significance to both outbound and inbound countries in a lot of fields. However, the previous research on outbound behavior mainly unitizes questionnaires or purely analyzes the statistical data, these data are either too difficult to acquire or unable to directly analyze public opinion. This paper aim to analyze outbound behavior through large volume of text and location information published on social network services (SNS) and look into a case study of Chinese people in Japan with the data acquired from Chinese microblog website, Sina Weibo. For text analysis, we utilize an adapted latent Dirichlet allocation model to extract the topics in text and discuss the spatial and temporal distribution of the topics. The experiment proves that the topics represent outbound behavior of tourists and foreign visitors can be found and explained in the experiment dataset, which indicates that SNS data is valid for analyzing outbound behavior.

In recent decades, the frequency and intensity of natural disasters has increased significantly, and this trend is expected to continue. Therefore, understanding and predicting human behavior and mobility during a disaster will play a vital role in planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, such research is very difficult to perform owing to the uniqueness of various disasters and the unavailability of reliable and large-scale human mobility data. In this study, we collect big and heterogeneous data (e.g., GPS records of 1.6 million users 1 over 3 years, data on earthquakes that have occurred in Japan over 4 years, news report data, and transportation network data) to study human mobility following natural disasters. An empirical analysis is conducted to explore the basic laws governing human mobility following disasters, and an effective human mobility model is developed to predict and simulate population movements. The experimental results demonstrate the efficiency of our model, and they suggest that human mobility following disasters can be significantly more predictable and be more easily simulated than previously thought.

The frequency and intensity of natural disasters has increased significantly in recent decades, and this trend is expected to continue. Hence, understanding and predicting human evacuation behavior and mobility will play a vital role in planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, existing models are shallow models, and it is difficult to apply them for understanding the “deep knowledge” of human mobility. Therefore, in this study, we collect big and heterogeneous data (e.g., GPS records of 1.6 million users over 3 years, data on earthquakes that have occurred in Japan over 4 years, news report data, and transportation network data), and we build an intelligent system, namely, DeepMob, for understanding and predicting human evacuation behavior and mobility following different types of natural disasters. The key component of DeepMob is based on a deep learning architecture that aims to understand the basic laws that govern human behavior and mobility following natural disasters, from big and heterogeneous data. Furthermore, based on the deep learning model, DeepMob can accurately predict or simulate a person’s future evacuation behaviors or evacuation routes under different disaster conditions. Experimental results and validations demonstrate the efficiency and superior performance of our system, and suggest that human mobility following disasters may be predicted and simulated more easily than previously thought.

Real-time estimation of human mobility following a massive disaster will play a crucial role in disaster relief. Because human mobility in massive disasters is quite different from their usual mobility, real-time human location data is necessary for precise estimation. Due to privacy concerns, real-time data is anonymized and a popular form of anonymization is population distribution. In this paper, we aim to estimate human mobility following an unprecedented disaster using such population distribution data. To overcome technical obstacles including high dimensionality, we propose novel particle filter by devising proposal distribution. Our proposal distribution provides states considering both prediction model and acquired observation. Therefore, particles maintain high likelihood. In the experiments, our methods realized more accurate estimation than the baselines, and its estimated mobility was consistent with the survey researches. The computational cost is significantly low enough for real-time operations. The GPS data collected on the day of the Great East Japan Earthquake is used for the evaluation.

Mining object-level knowledge, that is, building a comprehensive category model base, from a large set of cluttered scenes presents a considerable challenge to the field of artificial intelligence. How to initiate model learning with the least human supervision (i.e., manual labeling) and how to encode the structural knowledge are two elements of this challenge, as they largely determine the scalability and applicability of any solution. In this article, we propose a model-learning method that starts from a single-labeled object for each category, and mines further model knowledge from a number of informally captured, cluttered scenes. However, in these scenes, target objects are relatively small and have large variations in texture, scale, and rotation. Thus, to reduce the model bias normally associated with less supervised learning methods, we use the robust 3D shape in RGB-D images to guide our model learning, then apply the properly trained category models to both object detection and recognition in more conventional RGB images. In addition to model training for their own categories, the knowledge extracted from the RGB-D images can also be transferred to guide model learning for a new category, in which only RGB images without depth information in the new category are provided for training. Preliminary testing shows that the proposed method performs as well as fully supervised learning methods.

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, accurately predicting human emergency behavior and their mobility will become the critical issue for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. In this paper, we build up a large human mobility database (GPS records of 1.6 million users over one year) and several different datasets to capture and analyze human emergency behavior and their mobility following the Great East Japan Earthquake and Fukushima nuclear accident. Based on our empirical analysis through these data, we find that human behavior and their mobility following large-scale disaster sometimes correlate with their mobility patterns during normal times, and are also highly impacted by their social relationship, intensity of disaster, damage level, government appointed shelters, news reporting, large population flow and etc. On the basis of these findings, we develop a model of human behavior that takes into account these factors for accurately predicting human emergency behavior and their mobility following large-scale disaster. The experimental results and validations demonstrate the efficiency of our behavior model, and suggest that human behavior and their movements during disasters may be significantly more predictable than previously thought.

The Great East Japan Earthquake and the Fukushima nuclear accident cause large human population movements and evacuations. Understanding and predicting these movements is critical for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. In this paper, we construct a large human mobility database that stores and manages GPS records from mobile devices used by approximately 1.6 million people throughout Japan from 1 August 2010 to 31 July 2011. By mining this enormous set of Auto-GPS mobile sensor data, the short-term and long-term evacuation behaviors for individuals throughout Japan during this disaster are able to be automatically discovered. To better understand and simulate human mobility during the disasters, we develop a probabilistic model that is able to be effectively trained by the discovered evacuations via machine learning technique. Based on our training model, population mobility in various cities impacted by the disasters throughout the country is able to be automatically simulated or predicted. On the basis of the whole database, developed model, and experimental results, it is easy for us to find some new features or population mobility patterns after the recent severe earthquake, tsunami and release of radioactivity in Japan, which are likely to play a vital role in future disaster relief and management worldwide.

For reasons of public security, an intelligent surveillance system that can cover a large, crowded public area has become an urgent need. In this article, we propose a novel laser-based system that can simultaneously perform tracking, semantic scene learning, and abnormality detection in a fully online and unsupervised way. Furthermore, these three tasks cooperate with each other in one framework to improve their respective performances. The proposed system has the following key advantages over previous ones: (1) It can cover quite a large area (more than 60×35m), and simultaneously perform robust tracking, semantic scene learning, and abnormality detection in a high-density situation. (2) The overall system can vary with time, incrementally learn the structure of the scene, and perform fully online abnormal activity detection and tracking. This feature makes our system suitable for real-time applications. (3) The surveillance tasks are carried out in a fully unsupervised manner, so that there is no need for manual labeling and the construction of huge training datasets. We successfully apply the proposed system to the JR subway station in Tokyo, and demonstrate that it can cover an area of 60×35m, robustly track more than 150 targets at the same time, and simultaneously perform online semantic scene learning and abnormality detection with no human intervention.

Multitarget tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This article presents a promising online system to deal with these problems. The novelty of this system is that laser and vision are integrated with tracking and online learning to complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers online for “possible interacting targets”. When the targets are in close proximity, the classifiers learned online are used alongside visual information to assist in tracking. Therefore, this mode of cooperation not only deals with various tough problems encountered in tracking, but also ensures that the entire process can be completely online and automatic. Experimental results demonstrate that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a good trade-off between tracking accuracy and the time-cost factor.