Recovering the fine-grained working process of couriers is becoming one of the essential problems for improving the express delivery systems because knowing the detailed process of how couriers accomplish their daily work facilitates the analyzing, understanding, and optimizing of the working procedure. Although coarse-grained courier trajectories and waybill delivery time data can be collected, this problem is still challenging due to noisy data with spatio-temporal biases, lacking ground truth of couriers’ fine-grained behaviors, and complex correlations between behaviors. Existing works typically focus on a single dimension of the process such as inferring the delivery time and can only yield results of low spatio-temporal resolution, which cannot address the problem well. To bridge the gap, we propose a digital-twin-based iterative calibration system (DTRec) for fine-grained courier working process recovery. We first propose a spatio-temporal bias correction algorithm, which systematically improves existing methods in correcting waybill addresses and trajectory stay points. Second, to model the complex correlations among behaviors and inherent physical constraints, we propose an agent-based model to build the digital twin of couriers. Third, to further improve recovery performance, we design a digital-twin-based iterative calibration framework, which leverages the inconsistency between the deduction results of the digital twin and the recovery results from real-world data to improve both the agent-based model and the recovery results. Experiments show that DTRec outperforms state-of-the-art baselines by 10.8% in terms of fine-grained accuracy on real-world datasets. The system is deployed in the industrial practices in JD Logistics with promising applications. The code is available at https://github.com/tsinghua-fib-lab/Courier-DTRec .

Generally speaking, we can easily specify many causal relationships in the prediction tasks of ubiquitous computing, such as human activity prediction, mobility prediction, and health prediction. However, most of the existing methods in these fields failed to take advantage of this prior causal knowledge. They typically make predictions only based on correlations in the data, which hinders the prediction performance in real-world scenarios, because a distribution shift between training data and testing data generally exists. To fill in this gap, we proposed a Generative Adversarial Network (GAN)-based Causal Information Learning prediction framework, which can effectively leverage causal information to improve the prediction performance of existing ubiquitous computing deep learning models. Specifically, faced with a unique challenge that the treatment variable, referring to the intervention that influences the target in a causal relationship, is generally continuous in ubiquitous computing, the framework employs a representation learning approach with a GAN-based deep learning model. By projecting all variables except the treatment into a latent space, it effectively minimizes confounding bias and leverages the learned latent representation for accurate predictions. In this way, it deals with the continuous treatment challenge, and in the meantime, it can be easily integrated with existing deep learning models to lift their prediction performance in practical scenarios with causal information. Extensive experiments on two large-scale real-world datasets demonstrate its superior performance over multiple state-of-the-art baselines. We also propose an analytical framework together with extensive experiments to empirically show that our framework achieves better performance gain under two conditions: when the distribution differences between the training data and the testing data are more significant and when the treatment effects are larger. Overall, this work suggests that learning causal information is a promising way to improve the prediction performance of ubiquitous computing tasks. We open both our dataset and code 1 and call for more research attention in this area.

For walking pedestrians, when they are blocked by obstacles or other pedestrians, they adjust their speeds and directions to avoid colliding with them, which is called collision avoidance behavior. This behavior is the most complex part of pedestrians' walking processes and its modeling and simulation are the keys to realistic crowd simulation, which serves as the foundation for various applications. However, most existing methods either lack the representation power to accurately model the complex collision behavior or do not model it explicitly, which leads to a poor level of realism of the simulation. To realize realistic crowd simulation, we propose to analyze, understand, and model the collision avoidance behavior in a data-driven way. First, to automatically detect collision avoidance behavior for further analysis, we propose a domain transformation algorithm that detects it by transforming the trajectories in the spatial domain into a new domain where the behavior is much more apparent and is thus easier to detect. The new domain also provides a new perspective for understanding collision avoidance behavior. Second, since there are no mature metrics to evaluate the level of realism, we propose a new evaluation metric based on the least-effort theory, which evaluates the realism of collision avoidance behavior by its physical and mental consumption. This evaluation metric also provides the foundation of modeling. Third, for realistic crowd simulation, we design a reinforcement learning model. It trains agents with our proposed reward function that models pedestrians' intrinsic needs of "reducing effort consumption'' and thus can guide agents to behave realistically when avoiding collisions. Extensive experiments show our model is 55.9% and 52.5% more realistic in collision avoidance behavior than the best baselines on two real-world datasets. We release our codes at https://github.com/tsinghua-fib-lab/TECRL.

The two-dimensional bin packing problem (2DBP) is a critical optimization problem in the furniture production and glass cutting industries, where the objective is to cut smaller-sized items from a minimum number of large standard-sized raw materials. In practice, factories manufacture hundreds of customer orders (sets of items) every day, and to relieve pressure in management, a common practice is to group the orders into batches for production, ensuring that items from one order are in the same batch instead of scattered across the production line. In this work, we formulate this problem as the grouped 2D bin packing problem, a bi-level problem where the upper level partitions orders into groups and the lower level solves 2DBP for items in each group. The main challenges are (1) the coupled optimization of upper and lower levels and (2) the high computational efficiency required for practical application. To tackle these challenges, we propose an iteration-based hierarchical reinforcement learning framework, which can learn to solve the optimization problem in a data-driven way and provide fast online performance after offline training. Extensive experiments demonstrate that our method not only achieves the best performance compared to all baselines but is also robust to changes in dataset distribution and problem constraints. Finally, we deployed our method in the ARROW Home factory in China, resulting in a 4.1% reduction in raw material costs. We have released the source code and datasets to facilitate future research.

Dynamic link prediction is essential for a wide range of domains, including social networks, bioinformatics, knowledge bases, and recommender systems. Existing works have demonstrated that structural information and temporal information are two of the most important information for this problem. However, existing works either focus on modeling them independently or modeling the temporal dynamics of a single structural scale, neglecting the complex correlations among them. This paper proposes to model the inherent correlations among the evolving dynamics of different structural scales for dynamic link prediction. Following this idea, we propose an Attentional Multi-scale Co-evolving Network (AMCNet). Specifically, We model multi-scale structural information by a motif-based graph neural network with multi-scale pooling. Then, we design a hierarchical attention-based sequence-to-sequence model for learning the complex correlations among the evolution dynamics of different structural scales. Extensive experiments on four real-world datasets with different characteristics demonstrate that AMCNet significantly outperforms the state-of-the-art in both single-step and multi-step dynamic link prediction tasks.

Street view data is one of the most common data sources for urban prediction tasks, such as estimating socioeconomic status, sensing physical urban changes, and identifying urban villages. Typical research in this field consists of two steps: acquiring a dataset with a street view image sampling algorithm and designing a prediction algorithm for urban prediction tasks. However, most of the previous research focuses on the prediction algorithms, leaving the sampling algorithms underexplored. To fill this gap, we set out to investigate how different street view image sampling algorithms affect the performance of the follow-up tasks and develop an effective street view image sampling algorithm for urban prediction. Through a comprehensive analysis of the performance of different sampling algorithms in three of the most common urban prediction tasks, including commercial activeness prediction, urban liveliness prediction, and urban population prediction, we provide solid empirical evidence that the sampling algorithm significantly affects the performance of the prediction model. Specifically, the performance differences of different sampling algorithms can reach over 25%. Further, we revealed that the sampling step size and the sampling quality are two important factors that affect the performance of a sampling algorithm, while the sampling angle has little influence. Inspired by our analysis results, we propose an effective street view image sampling algorithm, DAS, which contains a denoising module and an adaptive sampling module. It can dynamically adjust the sampling step size to adapt to the optimal size for each region and get rid of the impact of noise images in the meantime. Experiments on three large-scale datasets demonstrate its superior performance over multiple state-of-the-art baselines, and further ablation study shows the effectiveness of each module. Finally, through a thorough discussion of our findings and experimental results, we provide insights into the street view image sampling algorithm design, and we call for more researches in this blank area.

In recent years, due to the prevalence of online textual advertisements, increasing businesses recognize their huge potential in product promotion. The high-quality textual content has been empirically shown to have a substantial impact on consumers’ attitudes and decisions. As a result, persuasive tactics play an essential role in online textual advertisements, which are employed to increase the attractiveness, and sequentially increase the conversion rate and sales volume. As the context of persuasion, product attributes, e.g., category and price, also greatly influence the persuasion outcomes. However, they are largely overlooked by existing works. In this paper, we propose a novel framework to study context-aware persuasion by designing a multi-task learning model and performing extensive causal analysis. First, the prediction model recognizes the persuasive tactics employed in an advertising text and predicts their promotion effectiveness. Specifically, we design a disentangled representation learning algorithm to capture the persuasive tactics, and then develop a novel context-aware attention module to model the relationships between persuasive tactics and product attributes. Experiments on a large-scale real-world dataset demonstrate the superior performance of our proposed model over state-of-the-art baselines. Then we show its great practical value by conducting an in-depth causal analysis of context-aware results that our model learns, which offers insightful interpretations and guidelines for marketers to employ persuasive tactics in textual advertisements.

Predicting future origin-destination (OD) flow is essential for urban planning since it provides feedback for planning adjustment and reference for road planning. However, OD prediction for urban planning scenarios is unique as it typically lacks training data. A common practice is to refer to data from other cities, which causes the out-of-distribution (OOD) problem. A promising solution is to leverage causal information in the data. However, there are two challenges in utilizing causal information in urban planning scenarios: (a) Urban system has numerous factors, and only part of them indicate causal information. (b) The planned city development correlates with original city characteristics, therefore bringing confounding bias to the causal modelling process. In this paper, we propose designs to solve both challenges. Specifically, we first design a causal disentangled representation module to identify causal factors in attributes. Second, we adopt a variational sample re-weighting module to reduce the confounding bias. Our proposed model outperforms seven state-of-the-art baselines on three real-world datasets, achieving an average improvement of 9.59% in the MAE metric. Further in-depth analysis shows our method's robustness across different urban planning scenarios and outstanding performance in predicting extremely large OD flows, which corroborates the contribution of our designs to the urban planning field.

Large-scale vehicle trajectories bring great benefits in understanding urban mobility, and can be used to promote a wide range of applications in building intelligent transportation systems. Traditional approaches cannot recover the trajectories of all the vehicles on the roads since they are based on partial trajectory data. To address it, we study the all-vehicle trajectory recovery based on traffic camera video data. However, there are two challenges in this study. First, the quality of the images captured by traffic cameras is unbalanced, so it is hard to identify the same vehicles. Second, the traffic camera observation data are sparse due to the incompleteness of the traffic cameras and possible vehicle miss from the traffic cameras. To deal with these challenges, we design a novel system to recover the vehicle trajectory with the granularity of the road intersection. In this system, we propose an iterative framework to jointly optimize the vehicle re-identification and trajectory recovery tasks. In the vehicle re-identification task, we propose an effective strategy to guide the vehicle clustering based on visual features and the spatio-temporal constraint features updated by the trajectory discovery task. In the trajectory recovery task, we model the spatial and temporal relations as well as the vehicle miss problem by a probabilistic approach to recover the trajectories. Extensive experiments demonstrate that our framework outperforms the existing state-of-art solutions. Finally, our system is deployed in practical applications of SenseTime, China, including traffic congestion analysis and traffic signal control.

Crowd simulation acts as the basic component in traffic management, urban planning, and emergency management. Most existing approaches use physics-based models due to their robustness and strong generalizability, yet they fall short in fidelity since human behaviors are too complex and heterogeneous for a universal physical model to describe. Recent research tries to solve this problem by deep learning methods. However, they are still unable to generalize well beyond training distributions. In this work, we propose to jointly leverage the strength of the physical and neural network models for crowd simulation by a Physics-Infused Machine Learning (PIML) framework. The key idea is to let the two models learn from each other by iteratively going through a physics-informed machine learning process and a machine-learning-aided physics discovery process. We present our realization of the framework with a novel neural network model, Physics-informed Crowd Simulator (PCS), and tailored interaction mechanisms enabling the two models to facilitate each other. Specifically, our designs enable the neural network model to identify generalizable signals from real-world data better and yield physically consistent simulations with the physical model's form and simulation results as a prior. Further, by performing symbolic regression on the well-trained neural network, we obtain improved physical models that better describe crowd dynamics. Extensive experiments on two publicly available large-scale real-world datasets show that, with the framework, we successfully obtain a neural network model with strong generalizability and a new physical model with valid physical meanings at the same time. Both models outperform existing state-of-the-art simulation methods in accuracy, fidelity, and generalizability, which demonstrates the effectiveness of the PIML framework for improving simulation performance and its capability for facilitating scientific discovery and deepening our understandings of crowd dynamics. We release the codes at https://github.com/tsinghua-fib-lab/PIML.

Accurate churn prediction for retaining users is keenly important for online services because it determines their survival and prosperity. Recent research has specified social influence to be one of the most important reasons for user churn, and thereby many works start to model its effects on user churn to improve the prediction performance. However, existing works only use the data's correlational information while neglecting the problem's causal nature. Specifically, the fact that a user's churn is correlated with some social factors does not mean he/she is actually influenced by his/her friends, which results in inaccurate and unexplainable predictions of the existing methods. To bridge this gap, we develop a counterfactual modeling framework for churn prediction, which can effectively capture the causal information of social influence for accurate and explainable churn predictions. Specifically, we first propose a backbone framework that uses two separate embeddings to model users' endogenous churn intentions and the exogenous social influence. Then, we propose a counterfactual data augmentation module to introduce the causal information to the model by providing partially labeled counterfactual data. Finally, we design a three-headed counterfactual prediction framework to guide the model to learn causal information to facilitate churn prediction. Extensive experiments on two large-scale datasets with different types of social relations show our model's superior prediction performance compared with the state-of-the-art baselines. We further conduct an in-depth analysis of the prediction results demonstrating our proposed method's ability to capture causal information of social influence and give explainable churn predictions, which provide insights into designing better user retention strategies.

A large-scale system for obtaining fine-grained vehicle trajectories is becoming increasingly important because it lays a solid foundation for a wide range of downstream applications, such as urban traffic optimization, road network profiling, route planning, etc. Traditional methods recover the trajectories from GPS data from apps or coarse-grained traces collected from base stations, which are costly and, more importantly, only cover limited vehicles on the road. Thus, they are not applicable to downstream tasks. To fill this gap, we explore the possibility of recovering vehicle trajectories from the video data recorded by widely deployed traffic cameras. The major challenges lie in the quality of the captured image, low sampling rate, and unbalanced temporal and spatial distribution. To address these challenges, we propose a general system to recover vehicle trajectories at the level of the road intersection, where a novel iterative framework is developed to combine both vehicle clustering and trajectory recovery tasks, which improve their performance simultaneously. The key motivation is that vehicle clustering based on visual features can provide essential discrete points for trajectory recovery, while the recovered routes can introduce spatial-temporal constraints to the initial vehicle clusters for de-noising the false results and complement the missing results. To prove the feasibility of our framework, we collect and plan to release a city-scale traffic camera dataset consisting of 24 hours of videos from 673 cameras across 1,106 intersections. To the best of our knowledge, this benchmark is the first to contain the ground truth of vehicle trajectories with a wide range of spatial and temporal coverage in an urban environment. We conduct extensive experiments and analysis on datasets of different scales to demonstrate the robustness of our framework. Last but not least, we have already deployed the whole system in the business applications of SenseTime, China, including traffic signal control and traffic flow analysis. We highly expect this dataset to further facilitate the research in this field and contribute more to traffic optimization systems in the real world.

Customer value is essential for successful customer relationship management. Although growing evidence suggests that customers’ purchase decisions can be influenced by social relationships, social influence is largely overlooked in previous research. In this work, we fill this gap with a novel framework — Motif-based Multi-view Graph Attention Networks with Gated Fusion (MAG), which jointly considers customer demographics, past behaviors, and social network structures. Specifically, (1) to make the best use of higher-order information in complex social networks, we design a motif-based multi-view graph attention module, which explicitly captures different higher-order structures, along with the attention mechanism auto-assigning high weights to informative ones. (2) To model the complex effects of customer attributes and social influence, we propose a gated fusion module with two gates: one depicts the susceptibility to social influence and the other depicts the dependency of the two factors. Extensive experiments on two large-scale datasets show superior performance of our model over the state-of-the-art baselines. Further, we discover that the increase of motifs does not guarantee better performances and identify how motifs play different roles. These findings shed light on how to understand socio-economic relationships among customers and find high-value customers.

The phenomenal success of the newly-emerging social e-commerce has demonstrated that utilizing social relations is becoming a promising approach to promote e-commerce platforms. In this new scenario, one of the most important problems is to predict the value of a community formed by closely connected users in social networks due to its tremendous business value. However, few works have addressed this problem because of 1) its novel setting and 2) its challenging nature that the structure of a community has complex effects on its value. To bridge this gap, we develop a Multi-scale Structure-aware Community value prediction network (MSC) that jointly models the structural information of different scales, including peer relations, community structure, and inter-community connections, to predict the value of given communities. Specifically, we first proposed a Masked Edge Learning Graph Convolutional Network (MEL-GCN) based on a novel masked propagation mechanism to model peer influence. Then, we design a Pair-wise Community Pooling (PCPool) module to capture critical community structures. Finally, we model the inter-community connections by distinguishing intra-community edges from inter-community edges and employing a Multi-aggregator Framework (MAF). Extensive experiments on a large-scale real-world social e-commerce dataset demonstrate our method’s superior performance over state-of-the-art baselines, with a relative performance gain of 11.40%, 10.01%, and 10.97% in MAE, RMSE, and NRMSE, respectively. Further ablation study shows the effectiveness of our designed components. Our code and dataset are available1.