Disaster has a great impact on human mobility patterns. Clustering mobility patterns by generalizing group level characteristics from diverse individual trajectories plays a vital role in informing post-disaster recovery strategies. However, predefined clustering criteria or supervised labeling alone are insufficient to adequately capture the dynamics of mobility patterns during disaster events. Besides, the existing methods are not enough to capture the sensitive changes in mobility patterns in disaster events and handle the data of high-dimensional. This research investigates an embedded deep learning-based method which can automatically extract the groups' short-term feature of mobility patterns and achieves short-term mobility pattern clustering during disaster. The proposed method employs a Transformer-based temporal encoder to capture intra-day sequence patterns and integrates a VAE component with an embedded latent variable that directly encodes group-level mobility modes. We also design a compactness–separation loss that explicitly encourages within-mode feature compactness and between-mode feature separation. Based on massive mobile data, we conduct mobility pattern clustering on the case of 2011 Fukushima Earthquake. Compared to conventional clustering approaches, the proposed model structure is more discriminative and can capture the more sensitive changes between pre- and post- events. Compared to baselines with different loss functions, proposal methods can make more accurate fitting result and obtain more discrete clusters modes. Additionally, sensitivity analysis is conducted to examine the influence of key hyper parameters within the model. Based on the clustering outcomes, five representative mobility patterns are identified. We further analyze the spatial-temporal characteristics of mobility pattern changes during the disaster events and the recovery period of mobility pattern.

Trajectory anomaly detection is critical for transportation safety, solving traffic congestion, optimizing routes, and other smart city applications. Existing methods often misclassify reasonable but non-standard trajectories, such as route changes due to traffic congestion, as anomalies. Additionally, many methods require extensive labeled datasets for training, however, in real-world scenarios, trajectory anomaly labeled data are often scarce and costly to obtain. To address the challenges, we investigate a Semi-Supervised Contrastive learning approach for Trajectory Anomaly Detection, namely SSCTAD. Specifically, SSCTAD integrates a transformer based method to process trajectories and utilizes the in-out flow encoder to capture the spatial-temporal patterns from the trajectories, which can help the model capture the traffic conditions. Then, by leveraging a small subset of labeled data along with extensive unlabeled data, SSCTAD utilizes a semi-supervised contrastive learning component enables the model to better distinguish normal and anomalous trajectories by maximizing the similarity of latent representation from the same class and diversifying those from different classes. This method improves the performance of trajectory anomaly detection and effectively reduces the dependence of the model on the labeled dataset. Finally, experiments show that SSCTAD outperforms the other baseline models.

The development of the Internet of Things (IoT) has enhanced smart city services for traffic monitoring, leading to numerous schemes for accurate flow prediction based on traffic sensors. However, existing approaches primarily capture spatio-temporal (ST) dependencies from traffic graphs and train their models using randomly ordered data. This overlooks the fact that the modeling difficulty of each sensor/node in the ST traffic graph can vary significantly due to its spatial dependencies and temporal trends, resulting in unreliable and unstable predictions in IoT scenarios. In this context, we argue that a well-designed curriculum with an easy-to-difficult order can improve the training of ST models. Therefore, this paper introduces an ST difficulty measurer to score the node-level difficulty of traffic graph from both spatial and temporal aspects, and then implements a curriculum in the ST model training process. More specifically, based on the tiered ST difficulty score, the ST model training begins with a subgraph consisting of “easy” nodes characterized by relatively consistent spatial relationships and regular temporal patterns. Gradually, more difficult nodes are incorporated into the subgraph and participate in subsequent training stages. Comprehensive experiments and analysis on two real-world traffic flow datasets confirm the effectiveness of our proposed approach.

Maritime traffic analysis plays a vital role in port operations and logistics coordination. Although Automatic Identification System (AIS) provides rich vessel movement data, analyzing port-to-port traffic remains challenging due to data heterogeneity and missing trajectory segments. We present PortVIS, a web-based interactive system that supports the comprehensive analysis and visualization of maritime traffic. PortVIS integrates multi-sourced datasets—including AIS records, port and anchorage metadata, and wind field data—to enable trajectory segmentation, regional analysis, and data imputation. Users can import raw AIS records and segment them into port-to-port trips using our system. These segmented trajectories can then be filtered and queried based on vessel information and trip attributes. Users can also define custom zones (e.g., anchorages or transit areas) and explore traffic patterns through maps and charts. Missing trajectory segments are reconstructed using our recent imputation approach to improve data quality. By integrating trajectory processing and various visual analytics, PortVIS provides a unified tool for maritime mobility analysis.

Large-scale human mobility data (e.g. GPS data) encodes valuable information interested by various fields. Extracting stay and movement behaviors from noisy positioning record sequences is a critical preliminary step to utilize human mobility data. For the past two decades, this processing has been founded on a simple intuition proposed by Hariharan and Zheng et al.[18, 48], which uses manually selected parameters to make recursive, rule-based classification as to whether data points in a positioning record sequence constitute noise, move, or stay. This de facto processing approach, despite its simplicity, is inherently sensitive to parameter choice and suffers from the low efficiency of sequential processing. These inherent limitations make it practically infeasible, when confronted with the large-scale, fine-grained human mobility datasets in industry. To address this fundamental problem in human mobility data utilization, we innotatively rethink the distinction in representation patterns of noise/stay/move within the positioning record sequence from the lens of topological persistence, culminating in a novel pipeline for robust and efficient human mobility data processing. This is grounded in our empirical observation that topological persistence features of stay/move/noise exhibit robust and generalizable discriminability across variations in parameter choice, individuals, and geographical regions. By introducing the Laplacian to simplify the computation of topological persistence features, our processing pipeline is capable to exploit GPUs' parallel capacity for efficient processing. Experiments on real-world GPS datasets totaling up to thousand billion data points demonstrate that our method produces processing results comparable to those of human annotators, while requiring only 10% of the time consumed by previous approaches. We further show that our method is scalable for cumulative data volume and remains effective in identifying stay/move behaviors that traditional techniques consistently fail to handle, even under conditions of severe positioning errors and diverse behavioral patterns.

Large-scale human mobility exhibits spatial and temporal patterns that can assist policymakers in decision making. Although traditional prediction models attempt to capture these patterns, they are often affected by nonperiodic public events, such as disasters and occasional celebrations. Since regular human mobility patterns are affected by these events, estimating their causal effects is critical to accurate mobility predictions. News articles provide unique perspectives on these events, though processing them is a challenge. In this study, we propose a causality based prediction model, CausalMob, to analyze the causal effects of public events. We first utilize large language models (LLMs) to extract human intentions from news and transform them into features that act as causal treatments. Next, the model learns representations of spatio-temporal regional covariates from multiple data sources to serve as confounders for causal inference. Finally, we present a causal effect estimation framework to ensure that event features remain independent of confounders during prediction. Based on large-scale real-world data, the experimental results show that the proposed model excels in human mobility prediction, outperforming state-of-the-art models.

Trajectory representation learning (TRL) maps trajectories to vector embeddings and facilitates tasks such as trajectory classification and similarity search. State-of-the-art (SOTA) TRL methods transform raw GPS trajectories to grid or road trajectories to capture high-level travel semantics, i.e., regions and roads. However, they lose fine-grained spatial-temporal details as multiple GPS points are grouped into a single grid cell or road segment. To tackle this problem, we propose the BLU rred Encoding method, dubbed BLUE, which gradually reduces the precision of GPS coordinates to create hierarchical patches with multiple levels. The low-level patches are small and preserve fine-grained spatial-temporal details, while the high-level patches are large and capture overall travel patterns. To complement different patch levels with each other, our BLUE is an encoder-decoder model with a pyramid structure. At each patch level, a Transformer is used to learn the trajectory embedding at the current level, while pooling prepares inputs for the higher level in the encoder, and up-resolution provides guidance for the lower level in the decoder. BLUE is trained using the trajectory reconstruction task with the MSE loss. We compare BLUE with 8 SOTA TRL methods for 3 downstream tasks, the results show that BLUE consistently achieves higher accuracy than all baselines, outperforming the best-performing baselines by an average of 30.90%. Our code is available at https://github.com/slzhou-xy/BLUE.

Real-world data distributions often shift continuously across multiple latent factors such as time, geography, and socioeconomic contexts. However, existing domain generalization approaches typically treat domains as discrete or as evolving along a single axis (e.g., time). This oversimplification fails to capture the complex, multidimensional nature of real-world variation. This paper introduces the task of Continuous Domain Generalization (CDG), which aims to generalize predictive models to unseen domains defined by arbitrary combinations of continuous variations. We present a principled framework grounded in geometric and algebraic theories, showing that optimal model parameters across domains lie on a low-dimensional manifold. To model this structure, we propose a Neural Lie Transport Operator (NeuralLio), which enables structure-preserving parameter transitions by enforcing geometric continuity and algebraic consistency. To handle noisy or incomplete domain variation descriptors, we introduce a gating mechanism to suppress irrelevant dimensions and a local chart-based strategy for robust generalization. Extensive experiments on synthetic and real-world datasets, including remote sensing, scientific documents, and traffic forecasting, demonstrate that our method significantly outperforms existing baselines in both generalization accuracy and robustness.

Analyzing the causal impact of various government-related policies on the epidemic spread is of critical importance. This article aims to investigate the problem of assessing the causal effects of different COVID-19-related policies on the USA epidemic spread in different counties at any given time period, while eliminating biased interference from unobserved confounders (e.g., the vigilance of residents). However, the infection outcome of each region is influenced not only by its own confounding factors but also by policy interventions implemented in neighboring regions. Furthermore, the government policy index may exhibit a time-delay influence on outbreak dynamics. To this end, we implement observational data about different COVID-19-related policies (treatment) and outbreak dynamics (outcome) across different U.S. counties over time and develop a causal framework that learns the representations of time-varying confounders to tackle the aforementioned issues. More specifically, we employ one recurrent structure to capture the accumulative effects stemming from the policy history and then utilize hypergraph neural network to model the interactions among spatial regions. Our experimental results demonstrate the effectiveness of the proposed framework in quantifying the causal impact of different policy types on epidemics. Compared with baseline methods, our assessment provides valuable insights for future policy-making endeavors.

Maritime transportation, vital for nearly 90% of global trade, necessitates precise vessel trajectory prediction for safety and efficiency. Although the Automatic Identification System (AIS) provides a comprehensive data source, how to model these multi-modal and heterogeneous time-varying sequences (such as vessels’ kinetic information and ocean weather factors) poses a formidable challenge. Moreover, most existing approaches are limited by the confined scope of vessel trajectory modeling, making it impossible to consider the unique characteristics of maritime transportation system. To tackle these challenges, we propose a novel framework called AISFuser to i) encode unique maritime traffic network into graphical representations, and ii) introduce the heterogeneity into multi-modal temporal embeddings through Self-Supervised Learning (SSL). Specifically, our AISFuser is constructed by combining an attention-based graph block with a transformer network to encode information across space and time, respectively. In terms of temporal dimension, one SSL auxiliary task is also designed to enhance the heterogeneity of temporal representations and supplement the main vessel prediction task. We validate the effectiveness of the proposed AISFuser on a real-world AIS dataset. Extensive experimental results demonstrate that our method can forecast multiple attributes of vessel trajectory for over 10 hours into the future, outperforming competitive baselines.

Maritime traffic management increasingly relies on vessel position information provided by terrestrial and satellite networks of the Automatic Identification System (AIS). Unfortunately, the problem of missing AIS data can lead to long-term gaps in vessel trajectory, raising corresponding security concerns regarding collision risks and illicit activities. Existing imputation approaches are often constrained by vehicle-based low-sampling trajectories, hindering their ability to address unique characteristics of maritime transportation systems and long-term missing scenarios. To tackle these challenges, we propose a novel generative framework for long-term vessel trajectory imputation. Our framework considers irregular tracks of vessels, which differ from those of cars due to the absence of a structured road network, and ensures the continuity of multi-point imputed trajectories. Specifically, we first utilize a pre-trained trajectory embedding block to capture patterns of vessel movements. Subsequently, we introduce a diffusion-based model for generating missing trajectories, where observed trajectory modeling with transformer encoding architecture and embeddings of both historical vessel trajectory and external factors serve as conditional information. In particular, we design a physics-guided discriminator in the training stage, which imposes kinematic constraints between locations and angles to improve the continuity of the imputed trajectories. Comprehensive experiments and analysis on a real-world AIS dataset confirm the effectiveness of our proposed approach.

This paper introduces a novel approach using Large Language Models (LLMs) integrated into an agent framework for flexible and effective personal mobility generation. LLMs overcome the limitations of previous models by effectively processing semantic data and offering versatility in modeling various tasks. Our approach addresses three research questions: aligning LLMs with real-world urban mobility data, developing reliable activity generation strategies, and exploring LLM applications in urban mobility. The key technical contribution is a novel LLM agent framework that accounts for individual activity patterns and motivations, including a self-consistency approach to align LLMs with real-world activity data and a retrieval-augmented strategy for interpretable activity generation. We evaluate our LLM agent framework and compare it with state-of-the-art personal mobility generation approaches, demonstrating the effectiveness of our approach and its potential applications in urban mobility. Overall, this study marks the pioneering work of designing an LLM agent framework for activity generation based on real-world human activity data, offering a promising tool for urban mobility analysis.

Perceiving and modeling urban crowd movements are of great importance to smart city-related fields. Governments and public service operators can benefit from such efforts as they can be applied to crowd management, resource scheduling, and early emergency warning. However, most prior research on urban crowd modeling has failed to describe the dynamics and continuity of human mobility, leading to inconsistent and irrelevant results when they tackle multiple homogeneous forecasting tasks as they can only be modeled independently. To overcome this drawback, we propose to model human mobility from a new perspective, which uses the citywide crowd transition process constituted by a series of transition matrices from low order to high order, to help us understand how the crowd dynamics evolve step-by-step. We further propose a Deep Transition Process Network to process and predict such new high-dimensional data, where novel grid embedding with Graph Convolutional Network, parameter-shared Convolutional LSTM, and High-Dimensional Attention mechanism are designed to learn the complicated dependencies in terms of spatial, temporal, and ordinal features. We conduct experiments on two datasets generated by a large amount of GPS data collected from a real-world smartphone application. The experiment results demonstrate the superior performance of our proposed methodology over existing approaches.

Monitoring the crowd in urban hot spot has been an important research topic in the field of urban management and has high social impact. It can allow more flexible allocation of public resources such as public transportation schedule adjustment and arrangement of police force. After 2020, because of the epidemic of COVID-19 virus, the public mobility pattern is deeply affected by the situation of epidemic as the physical close contact is the dominant way of infection. In this study, we propose a confirmed case-driven time-series prediction of crowd in urban hot spot named MobCovid. The model is a deviation of Informer, a popular time-serial prediction model proposed in 2021. The model takes both the number of nighttime staying people in downtown and confirmed cases of COVID-19 as input and predicts both the targets. In the current period of COVID, many areas and countries have relaxed the lockdown measures on public mobility. The outdoor travel of public is based on individual decision. Report of large amount of confirmed cases would restrict the public visitation of crowded downtown. But, still, government would publish some policies to try to intervene in the public mobility and control the spread of virus. For example, in Japan, there are no compulsory measures to force people to stay at home, but measures to persuade people to stay away from downtown area. Therefore, we also merge the encoding of policies on measures of mobility restriction made by government in the model to improve the precision. We use historical data of nighttime staying people in crowded downtown and confirmed cases of Tokyo and Osaka area as study case. Multiple times of comparison with other baselines including the original Informer model prove the effectiveness of our proposed method. We believe our work can make contribution to the current knowledge on forecasting the number of crowd in urban downtown during the Covid epidemic.

Urban time series data forecasting featuring significant contributions to sustainable development is widely studied as an essential task of the smart city. However, with the dramatic and rapid changes in the world environment, the assumption that data obey Independent Identically Distribution is undermined by the subsequent changes in data distribution, known as concept drift, leading to weak replicability and transferability of the model over unseen data. To address the issue, previous approaches typically retrain the model, forcing it to fit the most recent observed data. However, retraining is problematic in that it leads to model lag, consumption of resources, and model re-invalidation, causing the drift problem to be not well solved in realistic scenarios. In this study, we propose a new urban time series prediction model for the concept drift problem, which encodes the drift by considering the periodicity in the data and makes on-the-fly adjustments to the model based on the drift using a meta-dynamic network. Experiments on real-world datasets show that our design significantly outperforms state-of-the-art methods and can be well generalized to existing prediction backbones by reducing their sensitivity to distribution changes.

To understand human mobility following the typhoon, analyzing the causal impact of extreme typhoon weather on human mobility is important for disaster emergency management. However, the unobserved confounders (e.g., the characteristic of each region) correlate with the strength of typhoon weather and also affect human mobility during typhoon, which may generate biased influences on the causal analysis process. Besides, these confounders may be time-varying following the dynamic movements of typhoon. In this work, we develop a neural network-based continuous causal effect estimation framework to mitigate the interference from (unobserved) confounders and assess the continuous causal responses of typhoon-related weather (treatment) on several types of human mobility (outcome) across different counties at any given period. To this end, we integrate the big data from two huge typhoons in Japan (i.e., Typhoon Faxai and Hagibis) and leverage multiple sources of covariates (i.e., residents' vigilance and basic mobility patterns) from different counties to learn the representations of time-varying confounders. The experimental results indicate the effectiveness of our proposed framework in capturing the confounders for quantifying the causal impact of extreme weather during the typhoon process, compared with several existing causal studies.

Building outline extracted from high-resolution aerial images can be used in various application fields such as change detection and disaster assessment. However, traditional CNN model cannot recognize contours very precisely from original images. In this paper, we proposed a CNN and Transformer based model together with active contour model to deal with this problem. We also designed a triple-branch decoder structure to handle different features generated by encoder. Experiment results show that our model outperforms other baseline model on two datasets, achieving 91.1% mIoU on Vaihingen and 83.8% on Bing huts.

This paper introduces a graph encoding-based hybrid vision transformer for automatic road network extraction via high-resolution remote sensing imagery. Given that high-resolution remote sensing images covered large urban areas, traditional segmentation-based road extraction methods usually can generate good binary classification maps in simple structured road surfaces but fail in complex highway and bridge-covered areas. We introduce a graph encoding-based mechanism to address the above issues, enabling the road extraction framework extracts the road segmentation feature and build the graph structure map jointly. Compared to only segmentation-based methods, our approach learns prior geometrical structure information from the extracted ViT feature maps and has a non-local awareness of the whole road network structure. Eexperimental results demonstrated that the proposed approach outperforms the traditional segmentation-based methods.

Today, thanks to the rapid developing mobile and sensor networks in IoT (Internet of Things) systems, spatio-temporal big data are being constantly generated. They have brought us a data-driven possibility to sense and understand crowd mobility on a city scale. A fundamental task towards the next-generation mobility services, such as Intelligent Transportation Systems (ITS), Mobility-as-a-Service (MaaS), is spatio-temporal predictive modeling of the geo-sensory signals. There is a recent line of research leveraging deep learning techniques to boost the forecasting performance on such tasks. While simulating the regularity of mobility behaviors (e.g., routines, periodicity) in a more sophisticated way, the existing studies ignore an important part of urban activities, i.e., events. Including holidays, extreme weathers, pandemic, accidents, various urban events happen from time to time and cause non-stationary phenomena, which by nature make the spatio-temporal forecasting task challenging. We thereby envision an event-aware urban mobility prediction model that is capable of fast adapting and making reliable predictions in different scenarios, which is crucial to decision making towards emergency response and urban resilience.

Mobility digital twin, which is a a virtual replica of the mobility in the physical world, is the key building block of modern smart city applications at a metropolitan scale, including traffic regulation, emergency management and epidemic control. To duplicate the mobility in the physical world and show the potential outcome based on either the current state or manipulated conditions, we are facing with three main challenges: 1) how to sense real-time human mobility at a large scale and assimilate different data sources to infer a dynamic city mobility state; 2) how to make an accurate prediction for the mobility replica that adapts with dynamic city mobility state; 3) how to simulate the mobility in response to different conditions. In this talk, we will present our recent studies, practices and perspectives on mobility digital twin in addressing these above challenges, with applications in the real-world scenarios with industrial connections.

Human mobility nowcasting is a fundamental research problem for intelligent transportation planning, disaster responses and management, etc. In particular, human mobility under big disasters such as hurricanes and pandemics deviates from its daily routine to a large extent, which makes the task more challenging. Existing works mainly focus on traffic or crowd flow prediction in normal situations. To tackle this problem, in this study, disaster-related Twitter data is incorporated as a covariate to understand the public awareness and attention about the disaster events and thus perceive their impacts on the human mobility. Accordingly, we propose a Meta-knowledge-Memorizable Spatio-Temporal Network (MemeSTN), which leverages memory network and meta-learning to fuse social media and human mobility data. Extensive experiments over three real-world disasters including Japan 2019 typhoon season, Japan 2020 COVID-19 pandemic, and US 2019 hurricane season were conducted to illustrate the effectiveness of our proposed solution. Compared to the state-of-the-art spatio-temporal deep models and multivariate-time-series deep models, our model can achieve superior performance for nowcasting human mobility in disaster situations at both country level and state level.

In recent years, the use of WiFi fingerprints for indoor positioning has grown in popularity, largely due to the widespread availability of WiFi and the proliferation of mobile communication devices. However, many existing methods for constructing fingerprint data sets rely on labor-intensive and time-consuming processes of collecting large amounts of data. Additionally, these methods often focus on ideal laboratory environments, rather than considering the practical challenges of large multifloor buildings. To address these issues, we present a novel WiFi domain adversarial graph convolutional network model that can be trained using a small number of labeled site survey data and large amounts of unlabeled crowdsensed WiFi fingerprints. By constructing heterogeneous graphs based on received signal strength indicators (RSSIs) between waypoints and WiFi access points (APs), our model is able to effectively capture the topological structure of the data. We also incorporate graph convolutional networks (GCNs) to extract graph-level embeddings, a feature that has been largely overlooked in previous WiFi indoor localization studies. To deal with the challenges of large amounts of unlabeled data and multiple data domains, we employ a semisupervised domain adversarial training scheme to effectively utilize unlabeled data and align the data distributions across domains. Our system is evaluated using a public indoor localization data set that includes multiple buildings, and the results show that it performs competitively in terms of localization accuracy in large buildings.

Activity pattern prediction is a critical part of urban computing, urban planning, intelligent transportation, and so on. Based on a dataset with more than 10 million GPS trajectory records collected by mobile sensors, this research proposed a CNN-BiLSTM-VAE-ATT-based encoder-decoder model for fine-grained individual activity sequence prediction. The model combines the long-term and short-term dependencies crosswise and also considers randomness, diversity, and uncertainty of individual activity patterns. The proposed results show higher accuracy compared to the ten baselines. The model can generate high diversity results while approximating the original activity patterns distribution. Moreover, the model also has interpretability in revealing the time dependency importance of the activity pattern prediction.

Life pattern clustering is essential for abstracting the groups' characteristics of daily life patterns and activity regularity. Based on millions of GPS records, this research proposes a framework on the life pattern clustering which can efficiently identify the groups that have similar life patterns. The proposed method can retain original features of individual life pattern data without aggregation. Metagraph-based data structure is proposed for presenting the diverse life pattern. Spatial-temporal similarity includes significant places semantics, time-sequential properties and frequency are integrated into this data structure, which captures the uncertainty of an individual and the diversities between individuals. Non-negative-factorization-based method is utilized for reducing the dimension. The results show that our proposed method can effectively identify the groups that have similar life pattern in long term and takes advantage in computation efficiency and representational capacity compared with the traditional methods. We reveal the representative life pattern groups and analyze the group characteristics of human life patterns during different periods and different regions. We believe our work helps in future infrastructure planning, services improvement and policy making related to urban and transportation, thus promoting a humanized and sustainable city.

Long-term pedestrian trajectory prediction in crowds is highly valuable for safety driving and social robot navigation. The recent research of trajectory prediction usually focuses on solving the problems of modeling social interactions, physical constraints and multi-modality of futures without considering the generalization of prediction models to other scenes and objects, which is critical for real-world applications. In this paper, we propose a general framework that makes trajectory prediction models able to transfer well across unseen scenes and objects by quickly learning the prior information of trajectories. The trajectory sequences are closely related to the circumstance setting (e.g. exits, roads, buildings, entries etc.) and the objects (e.g. pedestrians, bicycles, vehicles etc.). We argue that those trajectory information varying across scenes and objects makes a trained prediction model not perform well over unseen target data. To address it, we introduce MetaTraj that contains carefully designed sub-tasks and meta-tasks to learn prior information of trajectories related to scenes and objects, which then contributes to accurate long-term future prediction. Both sub-tasks and meta-tasks are generated from trajectory sequences effortlessly and can be easily integrated into many prediction models. Extensive experiments over several trajectory prediction benchmarks demonstrate that MetaTraj can be applied to multiple prediction models and enables them generalize well to unseen scenes and objects.

Human mobility similarity comparison plays a critical role in modeling city dynamics, which exerts an enormous impact on developing intelligent transportation system. By expanding origin-destination matrix, we propose a mobility expression named mobility tableau and corresponding similarity measurement approach. Compared with traditional Origin-Destination matrix-based mobility comparison, mobility tableau comparison provides multi-dimensional similarity information, including volume similarity, spatial similarity, mass inclusiveness and structure similarity. The robustness of the measure is supported through several sensitive analysis based on real Global Positioning System dataset. The better performance of our proposed approach compared with traditional methods in two case studies including Call Detail Record based mobility tableau validation and different cities’ mobility comparison also demonstrates the practicality and superiority of our method.

The fast growing of e-commerce makes the express delivery an important component of transportation system. Absent delivery as one of the main reasons of failed delivery, brings serious waste of resources every year. Although with the development of Internet of Things technology, positioning system of recipients’ mobile devices can provide location information to help delivery routing optimization solving absent delivery, the lack of security makes it not practical. This paper proposes an IoT positioning technology based intelligent delivery system by introducing blockchain system and location information encryption, which overcomes three fatal problems of traditional IoT positioning technology based system: different party shares all information, location information interaction is too exposed and the system is unattractive to recipients. A set of analysis with real case experiment about efficiency improvement, incentive effect and security are conducted to validate the robustness of the system. Compared with previous research, the proposed system not only has high accuracy by utilizing real-time location information instead of unreliable prediction results, but also possesses high security.

How to effectively extract identifiable information from human mobility data and distinguish different agents is a significant topic for location-based services and intelligent transportation systems, which is described as the Human Mobility Signature Identification problem. A deeper understanding of the identifiable information underlain in human mobility can help us lay the foundation for applications such as irregular user behavior detection and privacy protection. However, human mobility comprises a mixture of different mobility patterns, traditional methods usually pay more attention to spatial-temporal features, while pattern dimension feature is usually ignored, which makes the result very dependent on the population agglomeration degree. To bridge the research gap, in this paper, we propose a novel Life-Travel pattern-based learning module (LTP-Net), in which spatial-temporal-pattern dimension features are embedded together to provide more comprehensive information for individual identification. A real-world mobile phone location dataset is utilized to evaluate the performance of the proposed LTP-Net and traditional methods. Several case studies are also conducted to analyze the model performance, including the abnormal behavior detection for the east Japan earthquake.

How to generate road conditions from urban GPS trajectory is an important problem in transportation systems. However, this computation process usually suffers from serious missing value problem due to the observation uncertainty or limited reports from crowdsourcing systems. Conventional tensor factorization approaches learn the spatio-temporal dependencies in a collaborative filtering way, which ignores the complex road network structure information and temporal heterogeneity. In this study, we propose a multi-view model with multiple aspects of prior knowledge to impute traffic state computed from a real-world trajectory dataset. More specifically, in the spatial view, rather than focusing on a specific type of road segment, we take the heterogeneity of road network into consideration and model the multiple relations of adjacent road segments. Meanwhile, the temporal pattern is also viewed as a heterogeneous graphical structure that discriminates the weekly/hourly adjacency in the temporal view. Finally, we fuse the above spatio-temporal features to provide a robust estimation under different sparse conditions. Intensive experiments on two types of missing scenarios (i.e., random and non-random) demonstrate that the proposed imputation method outperforms all the other state-of-the-art approaches. In addition, our model represents interpretable patterns for spatio-temporal graph analysis.

Predicting the density and flow of the crowd or traffic at a citywide level becomes possible by using the big data and cutting-edge AI technologies. It has been a very significant research topic with high social impact, which can be widely applied to emergency management, traffic regulation, and urban planning. In particular, by meshing a large urban area to a number of fine-grained mesh-grids, citywide crowd and traffic information in a continuous time period can be represented with 4D tensor (Timestep, Height, Width, Channel). Based on this idea, a series of methods have been proposed to address grid-based prediction for citywide crowd and traffic. In this study, we revisit the density and in-out flow prediction problem and publish a new aggregated human mobility dataset generated from a real-world smartphone application. Comparing with the existing ones, our dataset holds several advantages including large mesh-grid number, fine-grained mesh size, and high user sample. Towards this large-scale crowd dataset, we propose a novel deep learning model called DeepCrowd by designing pyramid architectures and high-dimensional attention mechanism based on Convolutional LSTM. Lastly, thorough and comprehensive performance evaluations are conducted to demonstrate the superiority of the proposed DeepCrowd comparing to multiple state-of-the-art methods.

Travel time estimation from GPS trips is of great importance to order duration, ridesharing, taxi dispatching, etc. However, the dense trajectory is not always available due to the limitation of data privacy and acquisition, while the origin-destination (OD) type of data, such as NYC taxi data, NYC bike data, and Capital Bikeshare data, is more accessible. To address this issue, this paper starts to estimate the OD trips travel time combined with the road network. Subsequently, aMulti-taskWeaklySupervisedLearning Framework forTravelTimeEstimation (MWSL-TTE) has been proposed to infer transition probability between roads segments, and the travel time on road segments and intersection simultaneously. Technically, given an OD pair, the transition probability intends to recover the most possible route. And then, the output of travel time is equal to the summation of all segments’ and intersections’ travel time in this route. A novel route recovery function has been proposed to iteratively maximize the current routes’ co-occurrence probability, and minimize the discrepancy between routes’ probability distribution and the inverse distribution of routes’ estimation loss. Moreover, the expected log-likelihood function based on a weakly-supervised framework has been deployed in optimizing the travel time from road segments and intersections concurrently. We conduct experiments on a wide range of real-world taxi datasets in Xi’an and Chengdu and demonstrate our method's effectiveness on route recovery and travel time estimation.

Aggregating crowd density in grids from big mobile datasets is a basic but critical work in urban computing and mobile computing. The error of position estimation in raw mobile data, including spatial deviation and temporal deviation, is inevitable and directly impacts the accuracy of aggregated crowd density results. In this case, a key modifiable areal unit problem is raised to understand the relationship among the crowd density accuracy, raw mobile data error, grid shape, and size, but few studies focused on it. This paper analyzes this modifiable areal unit problem of the error in crowd density estimation from big mobility data. By regarding the error as the result of a convolution operation, an optimization model based restoration method was proposed to fix the error of the estimated result, and we analyzed the restoration effect under different circumstances by several simulation experiments. A real application for grided population distribution map construction and restoration from Call Detail Record was conducted to prove the reliability of the whole analysis, which demonstrates the restoration method can reduce the error by nearly 40% under certain conditions.

The outbreak of coronavirus disease (COVID-19) has swept across more than 180 countries and territories since late January 2020. As a worldwide emergency response, governments have implemented various measures and policies, such as self-quarantine, travel restrictions, work from home, and regional lockdown, to control the spread of the epidemic. These countermeasures seek to restrict human mobility because COVID-19 is a highly contagious disease that is spread by human-to-human transmission. Medical experts and policymakers have expressed the urgency to effectively evaluate the outcome of human restriction policies with the aid of big data and information technology. Thus, based on big human mobility data and city POI data, an interactive visual analytics system called Epidemic Mobility (EpiMob) was designed in this study. The system interactively simulates the changes in human mobility and infection status in response to the implementation of a certain restriction policy or a combination of policies (e.g., regional lockdown, telecommuting, screening). Users can conveniently designate the spatial and temporal ranges for different mobility restriction policies. Then, the results reflecting the infection situation under different policies are dynamically displayed and can be flexibly compared and analyzed in depth. Multiple case studies consisting of interviews with domain experts were conducted in the largest metropolitan area of Japan (i.e., Greater Tokyo Area) to demonstrate that the system can provide insight into the effects of different human mobility restriction policies for epidemic control, through measurements and comparisons.

As a decisive part in the success of Mobility-as-a-Service (MaaS), spatio-temporal predictive modeling for crowd movements is a challenging task particularly considering scenarios where societal events drive mobility behavior deviated from the normality. While tremendous progress has been made to model high-level spatio-temporal regularities with deep learning, most, if not all of the existing methods are neither aware of the dynamic interactions among multiple transport modes nor adaptive to unprecedented volatility brought by potential societal events. In this paper, we are therefore motivated to improve the canonical spatio-temporal network (ST-Net) from two perspectives: (1) design a heterogeneous mobility information network (HMIN) to explicitly represent intermodality in multimodal mobility; (2) propose a memory-augmented dynamic filter generator (MDFG) to generate sequence-specific parameters in an on-the-fly fashion for various scenarios. The enhanced event-aware spatio-temporal network, namely EAST-Net, is evaluated on several real-world datasets with a wide variety and coverage of societal events. Both quantitative and qualitative experimental results verify the superiority of our approach compared with the state-of-the-art baselines. Code and data are published on https://github.com/underdoc-wang/EAST-Net.

Citywide crowd prediction can be of great importance for emergency management, traffic regulation, and urban planning. By meshing a large urban area to a number of fine-grained mesh-grids as illustrated in Fig. 1 , citywide crowd in a continuous time period can be represented with a four-dimensional tensor ${\mathbb{R}^{Timestep{\text{ }}p \times {\text{ }}Height{\text{ }} \times {\text{ }}Width{\text{ }} \times {\text{ }}Channel{\text{ }}}}$ in an analogous manner to video data, where each Timestep can be seen as one video frame, Height , Width is two-dimensional index for mesh-grids, and each Channel stores an aggregated scalar value for each mesh-grid. Specifically, given historical observations of crowd density and in-out flow x d = d 1 ,…,d t , xf = f 1 ,…, f t at timestamp t , we aim to build prediction models for the next-step density and in-out flow y d = d t +1, yf = f t +1, where y d means how many people will be in each mesh-grid at the next timestamp, and yf means how many people will flow into or out from each mesh-grid in next time interval. Al-though many deep models [1] – [6] have been proposed to address such tasks, their actual effects are still not well validated on large-scale and high-quality datasets. The datasets used in most of the works so far are originally generated based on taxi or bicycle trip data, which don’t cover and reflect the citywide crowd density and flow. Thus, we first publish new crowd flow data called BousaiTYO and BousaiOSA [7] . These new datasets are created using the GPS log data collected from a popular smartphone app of Yahoo! Japan Corporation, which can well reflect the real-world crowd flow in Tokyo and Osaka. As shown by Table 1 , our dataset has: (1) larger spatial area; (2) finer mesh size; (3) higher user sample.

Finding out similar regions between cities is important to a variety of real-world applications, such as point-of-interest recommendations, site selection, and travel guidance. With the help of the increasing number of location-based social network users, we can measure the intercity similarity from a new perspective of spatiotemporal characteristics of human mobility. In this paper, we developed an interactive intercity regional similarity explorer (IRSE) that 1) visualizes regional spatiotemporal human mobility features, 2) searches similar region candidates in the target city, and 3) explores the regional similarity from different views in a quantitative and illustrative way. In this paper, we show how our system can be useful in exploring regional similarity across cities in the world by use cases, which will interest various users from different countries in the demonstration session. Demo available at: https://bit.ly/3OjwnGu

Knowing "what is happening" and "what will happen" of the mobility in a city is the building block of a data-driven smart city system. In recent years, mobility digital twin that makes a virtual replication of human mobility and predicting or simulating the fine-grained movements of the subjects in a virtual space at a metropolitan scale in near real-time has shown its great potential in modern urban intelligent systems. However, few studies have provided practical solutions. The main difficulties are four-folds: 1) the daily variation of human mobility is hard to model and predict; 2) the transportation network enforces a complex constraints on human mobility; 3) generating a rational fine-grained human trajectory is challenging for existing machine learning models; and 4) making a fine-grained prediction incurs high computational costs, which is challenging for an online system. Bearing these difficulties in mind, in this paper we propose a two-stage human mobility predictor that stratifies the coarse and fine-grained level predictions. In the first stage, to encode the daily variation of human mobility at a metropolitan level, we automatically extract citywide mobility trends as crowd contexts and predict long-term and long-distance movements at a coarse level. In the second stage, the coarse predictions are resolved to a fine-grained level via a probabilistic trajectory retrieval method, which offloads most of the heavy computations to the offline phase. We tested our method using a real-world mobile phone GPS dataset in the Kanto area in Japan, and achieved good prediction accuracy and a time efficiency of about 2 min in predicting future 1h movements of about 220K mobile phone users on a single machine to support more higher-level analysis of mobility prediction.

Predicting the density and flow of the crowd at a citywide level is significant for city management. By meshing a large urban area to a number of fine-grained mesh-grids, citywide crowd and traffic information in a continuous time period can be represented with 4D tensor (Timestep, Height, Width, Channel). Based on this, we revisit the density and in-out flow prediction problem and publish a new aggregated human mobility dataset generated from a real-world smartphone application. Compared with the existing ones, our dataset has larger mesh-grid number, finer-grained mesh size, and higher user sample. Towards such kind of large-scale crowd dataset, we propose a novel deep learning model called DeepCrowd by designing pyramid architectures and high-dimensional attention mechanism based on Convolutional LSTM. Both the datasets and codes are made available at https://github.com/deepkashiwa20/DeepCrowd.

This paper introduces a cross-scale attention based end-to-end learning framework for tree crown detection via UAV imagery. Given that UAV images covered a large forests, the illumination variations, shadow obstacles and texture repetition always lead to inaccurate tree crown detection results. We introduce a cross-scale attention based mechanism to address the above issues, enabling the tree crown detection framework to reason about the RGB texture information and depth information introduced by the automatically generated depth map jointly. Compared to traditional image based tree crown detection methods, our approach learns prior over geometrical structure information from the real 3D world, which is robust to the texture repetition and small tree crowns. The experimental results demonstrated that the proposed approach outperforms the traditional CNN based method.

Estimating the travel time of a path is an essential topic for the intelligent transportation system. It serves as the foundation for real-world applications, such as traffic monitoring, route planning, and taxi dispatching. However, building a model for such a data-driven task requires a large amount of users’ travel information, which closely relates to their privacy and, thus, is less likely to be shared. The not independent and identically distributed (Non-IID) trajectory data across data owners also make a predictive model extremely challenging to be personalized if we directly apply federated learning. Finally, previous work on travel time estimation (TTE) does not consider the real-time traffic state of roads, which we argue, can significantly influence the prediction. To address the above challenges, we introduce GOF-TTE for the mobile user group, generative online federated learning framework for TTE, which 1) utilizes the federated learning approach, allowing private data to be kept on client devices while training, and designs the global model as an online generative model shared by all clients to infer the real-time road traffic state and 2) apart from sharing a base model at the server, adapts a fine-tuned personalized model for every client to study their personal driving habits, making up for the residual error made by localized global model prediction. We also employ a simple privacy attack to our framework and implement the differential privacy mechanism to guarantee privacy safety further. Finally, we conduct experiments on two real-world public taxi data sets of DiDi Chengdu and Xi’an. The experimental results demonstrate the effectiveness of our proposed framework.

Benefiting from the high coverage and re-visiting frequency, the satellite imagery is an ideal data for large-scale, real-time earth observation. However, due to the limited resolution and chromatic information, the satellite images, especially the panchromatic images, are not capable of being used for accurate earth observations, such as road extraction, vehicle detection, and building segmentation. In this research, we propose a cascaded fully convolutional network (CFCN) consists of a residual dense super-resolution network (RDSRN) for grayscale image super-resolution (SR) and a residual deconvolution colorization network (RDCN) for grayscale image colorization. The unique architecture can simultaneously learn texture detail and color information from aerial images and then transfer to enhance panchromatic images. Furthermore, we also introduce an indirect evaluation metric, learned extraction similarity (LES), to estimate the image quality of the generated image in the absence of the ground truth. Experiments on a multispectral image dataset demonstrate that panchromatic images enhanced by the proposed CFCN are with both texture and color fidelity as compared to aerial image. For pre-trained U-Net, compared to the performance on raw panchromatic images, the CFCN enhanced images increase 12.8% of LES of overall accuracy (96.4% versus 83.6%).

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g., police) and public service operators (e.g., subway/bus operator) to protect people’s safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the “deep” trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent, which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

We consider long-term path forecasting problems in crowds, where future sequence trajectories are generated given a short observation. Recent methods for this problem have focused on modeling social interactions and predicting multi-modal futures. However, it is not easy for machines to successfully consider social interactions, such as avoiding collisions while considering the uncertainty of futures under a highly interactive and dynamic scenario. In this paper, we propose a model that incorporates multiple interacting motion sequences jointly and predicts multi-modal socially acceptable distributions of futures. Specifically, we introduce a new aggregation mechanism for social interactions, which selectively models long-term inter-related dynamics between movements in a shared environment through a message passing mechanism. Moreover, we propose a loss function that not only accesses how accurate the estimated distributions of the futures are but also considers collision avoidance. We further utilize mixture density functions to describe the trajectories and learn the multi-modality of future paths. Extensive experiments over several trajectory prediction benchmarks demonstrate that our method is able to forecast socially acceptable distributions in complex scenarios.

Nowadays, with the rapid development of IoT (Internet of Things) and CPS (Cyber-Physical Systems) technologies, big spatiotemporal data are being generated from mobile phones, car navigation systems, and traffic sensors. By leveraging state-of-the-art deep learning technologies on such data, urban traffic prediction has drawn a lot of attention in AI and Intelligent Transportation System community. The problem can be uniformly modeled with a 3D tensor (T, N, C), where T denotes the total time steps, N denotes the size of the spatial domain (i.e., mesh-grids or graph-nodes), and C denotes the channels of information. According to the specific modeling strategy, the state-of-the-art deep learning models can be divided into three categories: grid-based, graph-based, and multivariate time-series models. In this study, we first synthetically review the deep traffic models as well as the widely used datasets, then build a standard benchmark to comprehensively evaluate their performances with the same settings and metrics. Our study named DL-Traff is implemented with two most popular deep learning frameworks, i.e., TensorFlow and PyTorch, which is already publicly available as two GitHub repositories https://github.com/deepkashiwa20/DL-Traff-Grid and https://github.com/deepkashiwa20/DL-Traff-Graph. With DL-Traff, we hope to deliver a useful resource to researchers who are interested in spatiotemporal data analysis.

Forecasting incident occurrences (e.g. crime, EMS, traffic accident) is a crucial task for emergency service providers and transportation agencies in performing response time optimization and dynamic fleet management. However, such events are by nature rare and sparse, which causes the label imbalance problem and inferior performance of models relying on data sufficiency. The existing studies circumvent, instead of truly solving, this issue by defining the incident prediction problem in a coarse-grained temporal (e.g. daily) setting, which leaves the proposed models unrobust to fine-grained dynamics and trivial for the real-world decision making. In this paper, we tackle the temporally fine-grained incident prediction problem in a sparse setting by explicitly exploiting the behind-the-scene chainlike triggering mechanism. Moreover, this chain effect roots in multiple domains (i.e. spatial, categorical), which further entangles with the temporal dimension and happens to be time-variant. To be specific, we propose a novel deep learning framework, namely Spatio-Temporal-Categorical Graph Neural Networks (STC-GNN), to handle the multidimensional and dynamic chain effect for performing fine-grained multi-incident co-prediction. Extensive experiments on three real-world city-level incident datasets verify the insightfulness of our perspective and effectiveness of the proposed model.

Forecasting regional ambulance demand plays a fundamental part in dynamic fleet allocation and redeployment. This topic has been gaining increasing significance, as virtually every country is experiencing an aging population, with generally higher level of vulnerability and demand for the emergency medical service (EMS). Although exploring the spatial and temporal correlations in EMS historical records, the existing methods principally consider the former time-invariant, which does not necessarily hold in reality. Moreover, this assumption ignores the fact that the behind-the-scenes dynamics are people, whose demographic profiles and activity patterns could be determinants of regional EMS demands. In this paper, we are therefore motivated to mine the collective daily routines in human mobility, to further represent the evolving spatial correlations. Particularly, we model profiled mobility groups as multiple random walkers and propose a novel bicomponent neural network, including a heterogeneous multi-graph convolution layer and spatio-temporal interlacing attention module, to perform the prediction task. Experimental results on the real-world data verify the effectiveness of introducing dynamic human mobility and the advantage of our approach over the state-of-the-art models.

While recent advancement of domain adaptation techniques is significant, most of methods only align a feature extractor and do not adapt a classifier to target domain, which would be a cause of performance degradation. We propose a novel domain adaptation technique for object detection that aligns prediction output space. In addition to feature alignment, we aligned predictions of locations and class confidences of our vehicle detector for satellite images by adversarial training. The proposed method significantly improved AP score by over 5%, which shows effectivity of our method for object detection tasks in satellite images. Our code is available at https://github.com/monotaro3/vd_pred_align.

Due to the limited resolution and chrominance information, panchromatic images can not be widely used in accurate earth observation applications, such as road extraction, vehicle detection, and building segmentation. In this research, we propose a cascaded fully convolutional network (CFCN) to achieve panchromatic image super-resolution and image colorization in an end-to-end manner. Experiments on a multispectral image dataset demonstrate that panchromatic images enhanced by the proposed CFCN can achieve high learned extraction similarity as compared to aerial images.

Accurate human path forecasting in complex and crowded scenarios is critical for collision avoidance of autonomous driving and social robots navigation. It still remains as a challenging problem because of dynamic human interaction and intrinsic multimodality of human motion. Given the observation, there is a rich set of plausible ways for an agent to walk through the circumstance. To address those issues, we propose a spatio-temporal model that can aggregate the information from socially interacting agents and capture the multimodality of the motion patterns. We use mixture density functions to describe the human path and predict the distribution of future paths with explicit density. To integrate more factors to model interacting people, we further introduce a coordinate transformation to represent the relative motion between people. Extensive experiments over several trajectory prediction benchmarks demonstrate that our method is able to forecast various plausible futures in complex scenarios and achieves state-of-the-art performance.

Nowadays, GPS devices have increased explosively and produced huge amounts of trajectory data related to people's outgoing. Through those big location data, many researches aim to analyze human mobility for urban development, such as human movement prediction/modeling, POI (Point-Of-Interest) recommendation. However, trajectory data only contains timestamp and location information. The intention of human movement is not explicit so that it is hard to understand why people go to somewhere. The intention prior to the activity could be of great significance for analyzing and predicting human mobility, which has not been taken into consideration by the existing researches until the present. Thus, in this study, we propose a brand-new concept called human intentional mobility, aiming to employ intention information to predict people's outgoing. We carefully utilize user's search query to sense his intention as well as the intensity. For instance, if a user searches a certain POI for many times in a short period, it will represent a relatively high intention to go there. Then, to fully utilize this intention representation for predicting whether user will visit searched POI or not, we specially design Dual Sequential Interaction Network (DualSIN) as a novel and unique deep-learning model, which can effectively capture the sophisticated interactions among two kinds of sequential information (i.e., search sequence and mobility sequence) and typical categorical information (i.e., user attributes). Last, we evaluate our model on real-world dataset collected from Yahoo! Japan portal application, and demonstrate that it can achieve superior satisfactory performances to the-state-of-the-art models on multiple POI search queries.

When conducting agent economic simulation for supply chains, inter-company transaction data are essential. However, the current inter-firm transaction data are network data in which branch office information is aggregated into headquarters transaction data. This study proposes a method to estimate branch office transactions from inter-company transaction data aggregated among headquarters by using a gravity model. We also confirm the method's reliability by comparing the estimated transaction data with the inter-regional input-output tables. We analytically considered the transition for all network configurations, demonstrating that the transaction quantity depends on the amount of labor and distance. We also demonstrated that our model fits well with data from business transactions, implying that the whole network structure can be used to model money flow in the real world.

Many previous studies showed that house rent is affected by residential property characteristics, house surrounding environment, facilities, and so on. However, there are few researches on finding the relationship between house rent and people's activities. Thus, we used hourly location-based big data collected by mobile phone GPS data to monitor people's activities all over the city. Multiple residential property characteristics and environments helped to verify if there is a relationship between house rent and people congestion in Tokyo. We find that people congestion has relationship with house rent and make more accurate prediction. We also employed linear and regularization regression and artificial neural network as algorithm and find artificial neural network might be the best calculation method.

Extracting identifiable information from human trajectories is a fundamental task in many location-based services (LBS). However, various mobility patterns underlain in human trajectories are difficult to model by existing models. Moreover, we could hardly define a clear user set for user identification because the set of users are dynamic and changing everyday. Bearing these in mind, we apply a deep multiple instance learning method to handle the multimodal mobility patterns in a weak-supervised learning way, and address the dynamic user set problems via a pairwise loss with negative sampling. We utilize a multi-head attention mechanism to automatically extract multiple aspects and match the corresponding information between query trajectories and historical trajectories. Our method shows a good identification accuracy on three human GPS trajectory data sets comparing with baseline methods.

Recently, owing to the outstanding capability of deep learning in solving ill-posed problems, the single-frame super-resolution (SR) researches tend to focus on deep learning methods largely. However, related researches are implemented and evaluated through various datasets and different deep learning frameworks, which hinders the comparison of performance among different methods and heavily hampers the progress of SR techniques. In this study, we present GeoSR, an open source computer vision package for deep learning based single-frame remote sensing imagery super-resolution to facilitate the development of the SR community. As a unified, simple, and flexible package, GeoSR contains pipeline-like integrated tools from data retrieval to final result evaluation, which enables users to develop self-defined models conveniently; several state-of-the-art models trained through the same high-quality dataset are provided as the baseline in the package as well. Moreover, the proposed package could potentially serve as a viable backend for other related packages such as image segmentation with high efficiency.

The recent diversification and accumulation of data from GPS equipped mobile phones, building sensors, and other resources in Japan has caused a large increase in the number of earthquake disaster scenarios that can be identified. Disaster prevention planning requires us to contemplate which scenario should be focused on and the required response to various scenarios. As a means to solve this problem, the damage distribution of building collapse and fire from GPS data can be used to estimate future damage based on people flow and various hypocenter models of earthquakes. We propose a method that uses sparse modeling to extract scenarios that are important for disaster estimation and prevention. As a result, this paper makes it possible to quickly grasp the scenario distribution, which was previously impossible to do, and to extract the significant scenarios.

Recently, deep learning algorithms, especially fully convolutional network based methods, have become very popular in the field of remote sensing. However, these methods are implemented and evaluated through various datasets and deep learning frameworks. There has not been a package that covers these methods in a unified manner. In this study, we introduce a computer vision package termed Geoseg that focuses on building segmentation and outline extraction. Geoseg implements nine state-of-the-art models as well as utility scripts needed to conduct model training, logging, evaluation, and visualization. The implementation of Geoseg emphasizes unification, simplicity, and flexibility. The performance and computational efficiency of all implemented methods are evaluated by a comparison experiment using a unified, high-quality aerial image dataset.

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g. police) and public service operators (e.g. subway/bus operator) to protect people's safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the deep trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly-complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

Big human mobility data are being continuously generated through a variety of sources, some of which can be treated and used as streaming data for understanding and predicting urban dynamics. With such streaming mobility data, the online prediction of short-term human mobility at the city level can be of great significance for transportation scheduling, urban regulation, and emergency management. In particular, when big rare events or disasters happen, such as large earthquakes or severe traffic accidents, people change their behaviors from their routine activities. This means people's movements will almost be uncorrelated with their past movements. Therefore, in this study, we build an online system called DeepUrbanMomentum to conduct the next short-term mobility predictions by using (the limited steps of) currently observed human mobility data. A deep-learning architecture built with recurrent neural networks is designed to effectively model these highly complex sequential data for a huge urban area. Experimental results demonstrate the superior performance of our proposed model as compared to the existing approaches. Lastly, we apply our system to a real emergency scenario and demonstrate that our system is applicable in the real world.

This study examines the economic impact on the supply chain of a large-scale flood in the Arakawa river area in Tokyo by using the geographic information system (GIS) data on inter-firm transactions. First, we identify the firms that were unable to continue doing business due to damage by using flood simulation analysis data on the Arakawa area. Second, we identify the firms that have business relationships with affected firms by using inter-firm transaction network data. Then, we discuss and analyze the industrial structure of the firms that have relationships with affected firms. Finally, we estimate the amount of transactions affected. Our results are as follows. First, 15-28% of the firms in Japan have business relationships with the affected firms. The figure increases to 44-48% when second-order firms whose transaction partners have business relationships with affected firms are included, and to nearly 52% when fifth-order firms are included. The impact on major cities, such as Tokyo and Osaka, is significant. By industry, the economic impact on the financial and transportation industries is large, while that on the agriculture, forestry, and fisheries industries is small. These results indicate the importance of business continuity planning for large-scale floods.

In this study, we present population distribution map and automated human settlement map for Maputo. To identify buildings from high-resolution remote sensing (HRRS) images of highly dense area, we used pixel-based classification frame with Ensemble Convolution Neural Network (ECNN) algorithm for building detection and used these identified buildings with zone based population data provided by the Japan International Cooperation Agency (JICA) to create population distribution map. Because of enormous HRRS image data, the training was performed on TSUBAME, a supercomputer operated by Tokyo Institute of Technology. The system used up to 28 Physical CPU cores, 128 GB of system memory and 16 GB of GPU memory for training. The result from building detection algorithm achieved overall accuracy of 96.8% and the average kappa of 70% and the population map created from it has a resolution of 10m × 10m.

The automatic digitizing of paper maps is a significant and challenging task for both academia and industry. As an important procedure of map digitizing, the semantic segmentation section is mainly relied on manual visual interpretation with low efficiency. In this study, we select urban planning maps as a representative sample and investigate the feasibility of utilizing U-shape fully convolutional based architecture to perform end-to-end map semantic segmentation. The experimental results obtained from the test area in Shibuya district, Tokyo, demonstrate that our proposed method could achieve a very high Jaccard similarity coefficient of 93.63% and an overall accuracy of 99.36%. For implementation on GPGPU and cuDNN, the required processing time for the whole Shibuya district can be less than three minutes. The results indicate the proposed method can serve as a viable tool for urban planning map semantic segmentation task with high accuracy and efficiency.

In this paper, we designed unsupervised domain adaptation (DA) methods to vehicle detection in high-resolution satellite images. We applied two Single Shot MultiBox Detectors, which have advantages in handling image feature differences among various kinds of image data: Correlation Alignment DA (CORAL DA) and adversarial DA. These novel methods can much improve accuracy without annotated data by finding the common feature space of source and target domains and aligning the features. While a mean of average precision (AP) and F1 dropped from 84.1 % in the source domain to 66.3% in the target domain, the CORAL DA and adversarial DA improved it to 76.8% and 75.9% respectively. These improvements were over a half of the performance degradation, indicating the usability of our methods.

Mobile Phones are near ubiquitous. We introduce a sensing technique to count how many people flow into and out of an urban area in an average weekday and weekend by combining subscribers billing records, road infrastructure network, and population data. Furthermore, trips are disaggregated by entry region and validated to the most recent available traffic survey. We found that 69,439 trip, 69,164 trip flow into and out of Greater Maputo in Mozambique, our study area, respectively in an average weekday, while those numbers are 46,976 trip, 45,237 trip on an average weekend.

The four papers in this special section examine the use of multimedia Big Data for leveraging big events such as natural disasters. Leveraging technological advances and using Big Data approaches for information management and processing in extreme events have attracted much research attention. Multimedia is increasingly becoming the most important and valuable source for insight and information on extreme events. It covers from everyone’s experiences to everything happening in the world. There will be lots of multimedia big data in extreme events—surveillance video, disaster images, social media, voice and video, to name a few, only if their volumes grow to the extent that the traditional multimedia processing and analysis systems may not handle effectively. As such, multimedia big data in extreme events are spurring on tremendous amounts of research and development of related technologies and applications. The papers in this issue is intended to provide a premier forum for researchers working on multimedia big data for extreme events to present their recent research results. It also provides an important opportunity for multidisciplinary studies connecting multimedia computing to disaster information management.

Globalization leads to more and more outbound behavior. Understanding these behavior is of great significance to both outbound and inbound countries in a lot of fields. However, the previous research on outbound behavior mainly unitizes questionnaires or purely analyzes the statistical data, these data are either too difficult to acquire or unable to directly analyze public opinion. This paper aim to analyze outbound behavior through large volume of text and location information published on social network services (SNS) and look into a case study of Chinese people in Japan with the data acquired from Chinese microblog website, Sina Weibo. For text analysis, we utilize an adapted latent Dirichlet allocation model to extract the topics in text and discuss the spatial and temporal distribution of the topics. The experiment proves that the topics represent outbound behavior of tourists and foreign visitors can be found and explained in the experiment dataset, which indicates that SNS data is valid for analyzing outbound behavior.

In recent decades, the frequency and intensity of natural disasters has increased significantly, and this trend is expected to continue. Therefore, understanding and predicting human behavior and mobility during a disaster will play a vital role in planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, such research is very difficult to perform owing to the uniqueness of various disasters and the unavailability of reliable and large-scale human mobility data. In this study, we collect big and heterogeneous data (e.g., GPS records of 1.6 million users 1 over 3 years, data on earthquakes that have occurred in Japan over 4 years, news report data, and transportation network data) to study human mobility following natural disasters. An empirical analysis is conducted to explore the basic laws governing human mobility following disasters, and an effective human mobility model is developed to predict and simulate population movements. The experimental results demonstrate the efficiency of our model, and they suggest that human mobility following disasters can be significantly more predictable and be more easily simulated than previously thought.

The frequency and intensity of natural disasters has increased significantly in recent decades, and this trend is expected to continue. Hence, understanding and predicting human evacuation behavior and mobility will play a vital role in planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, existing models are shallow models, and it is difficult to apply them for understanding the “deep knowledge” of human mobility. Therefore, in this study, we collect big and heterogeneous data (e.g., GPS records of 1.6 million users over 3 years, data on earthquakes that have occurred in Japan over 4 years, news report data, and transportation network data), and we build an intelligent system, namely, DeepMob, for understanding and predicting human evacuation behavior and mobility following different types of natural disasters. The key component of DeepMob is based on a deep learning architecture that aims to understand the basic laws that govern human behavior and mobility following natural disasters, from big and heterogeneous data. Furthermore, based on the deep learning model, DeepMob can accurately predict or simulate a person’s future evacuation behaviors or evacuation routes under different disaster conditions. Experimental results and validations demonstrate the efficiency and superior performance of our system, and suggest that human mobility following disasters may be predicted and simulated more easily than previously thought.

With the rapid development of urbanization and public transportation system, the number of traffic accidents have significantly increased globally over the past decades and become a big problem for human society. Facing these possible and unexpected traffic accidents, understanding what causes traffic accident and early alarms for some possible ones will play a critical role on planning effective traffic management. However, due to the lack of supported sensing data, research is very limited on the field of updating traffic accident risk in real-time. Therefore, in this paper, we collect big and heterogeneous data (7 months traffic accident data and 1.6 million users' GPS records) to understand how human mobility will affect traffic accident risk. By mining these data, we develop a deep model of Stack denoise Autoencoder to learn hierarchical feature representation of human mobility. And these features are used for efficient prediction of traffic accident risk level. Once the model has been trained, our model can simulate corresponding traffic accident risk map with given real-time input of human mobility. The experimental results demonstrate the efficiency of our model and suggest that traffic accident risk can be significantly more predictable through human mobility.

There are two broad categories of citywide human mobility, routine, composed of daily or periodic travel, and rare, which occurs during events such as the Olympic Games or natural disasters. State-of-the-art studies have shown that routine mobility patterns can be modeled stochastically, while rare human mobility modeling, essential to a variety of urban computing scenarios, such as emergency management and traffic regulation, is a much more challenging and understudied problem. Instead of training a rare-event-specific human mobility model, which suffers from the particularity of the rare events, in this paper we provide a new insight into rare events and propose a novel algorithm, CityCoupling, which establishes an intercity spatial mapping that uses human mobility in one city as input and reproduces human mobility in another city. More intuitively, we attempt to answer the question "What if this rare event happened in another city?". To find the optimal intercity spatial mapping, we utilize an expectation-maximization algorithm to estimate a probabilistic geographical correspondence matrix by regarding intercity trajectory matching as latent variables. Thereafter, a Gibbs sampling-based multiple hidden Markov model generates simulated trajectories. We apply our approach to a large mobile phone GPS dataset in Japan and determine the spatial mapping between Tokyo and Osaka to transfer the human mobility at the Great Eastern Japan Earthquake in Tokyo, which was heavily affected, to simulate what might have occurred if Osaka had been struck by the earthquake. We conduct the evaluation by assuming that New Year's Countdown is a rare event that occurs simultaneously in both Tokyo and Osaka, and thus we quantitatively compare our simulation with the ground truth in Osaka.

Automation of satellite-based human settlement mapping is highly needed to utilize historical archives of satellite data for urgent issues of urban development in global scale. We developed an automated algorithm to detect human settlement from Landsat satellite data. A machine learning algorithm, Local and Global Consistency (LLGC), was applied with improvements for remote sensing data. The algorithm enables to use existing coarse-resolution land cover maps as a training dataset so that any manual process is not required for preparation of training data. In addition, for better robustness against uncertainty in satellite data, we proposed a method to combine several LLGC results for several dates in a certain period from a target date into a single human settlement map with a pixel-based median composition among the input LLGC results. Combination of the methods enabled to develop timeseries human settlement maps using Landsat data, single ones of which could be affected by cloud contaminations. We applied the algorithm to Landsat data of 838 WRS2 tiles for cities with more than one million people worldwide for 1990, 2000, 2005, and 2010. MCD12Q1, a MODIS-based global land cover map with 500-m resolution, was used as training data. Visual assessment of the results suggested next steps for improvement of the method.

We categorize this research in terms of its contribution to both graph theory and computer vision. From the theoretical perspective, this study can be considered as the first attempt to formulate the idea of mining maximal frequent subgraphs in the challenging domain of messy visual data, and as a conceptual extension to the unsupervised learning of graph matching. We define a soft attributed pattern (SAP) to represent the common subgraph pattern among a set of attributed relational graphs (ARGs), considering both their structure and attributes. Regarding the differences between ARGs with fuzzy attributes and conventional labeled graphs, we propose a new mining strategy that directly extracts the SAP with the maximal graph size without applying node enumeration. Given an initial graph template and a number of ARGs, we develop an unsupervised method to modify the graph template into the maximal-size SAP. From a practical perspective, this research develops a general platform for learning the category model (i.e., the SAP) from cluttered visual data (i.e., the ARGs) without labeling "what is where," thereby opening the possibility for a series of applications in the era of big visual data. Experiments demonstrate the superior performance of the proposed method on RGB/RGB-D images and videos.

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, understanding and simulating of human emergency mobility following disasters will becomethe critical issue for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. However, due to the uniquenessof various disasters and the unavailability of reliable and large scale human mobility data, such kind of research is very difficult to be performed. Hence, in this paper,we collect big and heterogeneous data (e.g. 1.6 million users' GPS records in three years, 17520 times of Japan earthquake data in four years, news reporting data, transportation network data and etc.) to capture and analyze human emergency mobility following different disasters. By mining these big data, we aim to understand what basic laws govern human mobility following disasters, and develop a general model of human emergency mobility for generating and simulating large amount of human emergency movements. The experimental results and validations demonstrate the efficiency of our simulation model, and suggest that human mobility following disasters may be significantly morepredictable and can be easier simulated than previously thought.

Applications utilizing many types of location data, such as traffic congestion estimation and facility management using indoor pedestrian tracks, have been rapidly increasing. Such applications require the integration of various locational data from different data sources to produce more values. Efforts to ensure smoother data exchange are required for promoting the use of such applications because handling and integrating location data will enlarge the market for geo-spatial information. In response to this need, we had proposed a data encoding standard called `OGC®Moving Features' to contribute to smoother data exchange, and it was adopted as an international standard on Feb. 2015. We demonstrate in this work that OGC®Moving Features is an effective tool to advance technologies for applications using many types of location data, with referring "4Vs" to represent the most pressing bigdata issues.

Human movements are difficult to predict, especially, when we consider rare behaviors that deviate from normal daily routines. By tracing the behavior of a person over a long period, we can model their daily routines and predict periodical behaviors, whereas rare behaviors, such as participating in the New Year's Eve countdown, can hardly be predicted readily and thus they have usually been treated as outliers of the daily routines in most existing studies. However, for scenarios such as emergency management or intelligent traffic regulation, we are more interested in rare behaviors than daily routines. Using human mobility Big Data, the rare behavior of each individual in a social crowd is no longer rare and thus it may be predicted when we analyze the crowd behavior at a citywide level. Therefore in this study, instead of predicting movement based on daily routines, we make short-term predictions based on the recent movement observations. We propose a novel model called CityMomentum as a predicting-by-clustering framework for sampling future movement using a mixture of multiple random Markov chains, each of which is a Naive Movement Predictive model trained with the movements of the subjects that belong to each cluster. We apply our approach to a big mobile phone GPS log dataset and predict the short-term future movements, especially during the Comiket 80 and New Year's Eve celebration. We evaluate our prediction by a Earth Mover Distance (EMD) based metric, and show our approach accurately predicts the crowd behavior during the rare crowd events, which makes an early crowd event warning and regulation possible in the emergent situations.

Weather index insurance against severe weather and flooding is an important research issue to protect both people and agriculture in developed and developing countries. This study utilized remote sensing data and climate data to find the relationship between crop yield and different indicators, like EVI, crop stage at flooding, inundation period, and other climate factors to reduce the basis risk. In this study, MODIS and PALSAR imageries are analyzed for flood detection using SVM in Illinois state. Results indicate that flood mostly destroyed those cropped areas which were near the river. Remote sensing clears the affect of weather, crop phenology and flooding on crop harvest loss, which is key to lower basis risk for weather index insurance.

This paper describes Illinois corn yield estimation using deep learning and another machine learning, SVR. Deep learning is a technique that has been attracting attention in recent years of machine learning, it is possible to implement using the Caffe. High accuracy estimation of crop yield is very important from the viewpoint of food security. However, since every country prepare data inhomogeneously, the implementation of the crop model in all regions is difficult. Deep learning is possible to extract important features for estimating the object from the input data, so it can be expected to reduce dependency of input data. The network model of two InnerProductLayer was the best algorithm in this study, achieving RMSE of 6.298 (standard value). This study highlights the advantages of deep learning for agricultural yield estimating.

The understanding of mass population movements has greatly advanced with the rapid spread of ubiquitous devices. Anonymized call detail records (CDRs) for mobile phones have enabled us to not only trace individual trajectories but also approximate activity patterns, including significant locations such as homes and workplaces. The majority of studies analyzing CDRs attempt to utilize the mobility patterns of anonymized crowds to improve transportation and public health. This is quite reasonable because CDRs can capture the movements of people at given times and places, whereas general statistics usually account for a population based on their locations of residence. However, it has also been pointed out that there are discrepancies between the movements of people as observed through CDRs and those of an entire population in a given area. This is because CDRs only represent device users. In fact, we can never learn about the population that is unobservable through CDRs only by analyzing CDRs. Therefore, this study attempts to provide clues to help us understand the whereabouts of the unobservable population by analyzing two months of the CDRs for 58 volunteers with mobile device service from a major telecommunications company in combination with field survey data from Dhaka. We surveyed the personal and household attributes of mobile users in relation to their calling behavior. The analysis results show that per mobile user observed in CDRs, there is an average of roughly 2.4 to 2.8 unobservable people. Their age groups and gender composition are also provided. We find that male and female users exhibit opposite trends in call locations according to the presence of children within the household. In addition, based on field observations, we find that the location and time distributions of small children follow some specific routines. Our findings contribute to the understanding of the whereabouts of the unobservable population, the majority of whom are children and are considered to be vulnerable to disasters or infectious diseases but are difficult to locate through CDRs alone.

Mining object-level knowledge, that is, building a comprehensive category model base, from a large set of cluttered scenes presents a considerable challenge to the field of artificial intelligence. How to initiate model learning with the least human supervision (i.e., manual labeling) and how to encode the structural knowledge are two elements of this challenge, as they largely determine the scalability and applicability of any solution. In this article, we propose a model-learning method that starts from a single-labeled object for each category, and mines further model knowledge from a number of informally captured, cluttered scenes. However, in these scenes, target objects are relatively small and have large variations in texture, scale, and rotation. Thus, to reduce the model bias normally associated with less supervised learning methods, we use the robust 3D shape in RGB-D images to guide our model learning, then apply the properly trained category models to both object detection and recognition in more conventional RGB images. In addition to model training for their own categories, the knowledge extracted from the RGB-D images can also be transferred to guide model learning for a new category, in which only RGB images without depth information in the new category are provided for training. Preliminary testing shows that the proposed method performs as well as fully supervised learning methods.

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, urban emergency management has become the especially important issue for the whole governments around the world.In this paper, we present a novel intelligent system for urban emergency management during the large-scale disasters. The proposed systemstores and manages the global positioning system (GPS) records from mobile devices used by approximately 1.6 million people throughout Japan over one year. By mining and analyzing population movements after the Great East Japan Earthquake, our system can automatically learn a probabilistic model to better understand and simulate human mobility during the emergency situations. Based on the learning model, population mobility in various urban areas impacted by the earthquake throughout Japan can be automatically simulated or predicted. On the basis of such kind of system, it is easy for us to find some new features or population mobility patterns after the recent and unprecedented composite disasters, which are likely to provide valuable experience and play a vital role for future disaster management worldwide.

3D reconstruction from a single image is a classical problem in computer vision. However, it still poses great challenges for the reconstruction of daily-use objects with irregular shapes. In this paper, we propose to learn 3D reconstruction knowledge from informally captured RGB-D images, which will probably be ubiquitously used in daily life. The learning of 3D reconstruction is defined as a category modeling problem, in which a model for each category is trained to encode category-specific knowledge for 3D reconstruction. The category model estimates the pixel-level 3D structure of an object from its 2D appearance, by taking into account considerable variations in rotation, 3D structure, and texture. Learning 3D reconstruction from ubiquitous RGB-D images creates a new set of challenges. Experimental results have demonstrated the effectiveness of the proposed approach.

Graph matching and graph mining are two typical areas in artificial intelligence. In this paper, we define the soft attributed pattern (SAP) to describe the common subgraph pattern among a set of attributed relational graphs (ARGs), considering both the graphical structure and graph attributes. We propose a direct solution to extract the SAP with the maximal graph size without node enumeration. Given an initial graph template and a number of ARGs, we modify the graph template into the maximal SAP among the ARGs in an unsupervised fashion. The maximal SAP extraction is equivalent to learning a graphical model (i.e. an object model) from large ARGs (i.e. cluttered RGB/RGB-D images) for graph matching, which extends the concept of "unsupervised learning for graph matching." Furthermore, this study can be also regarded as the first known approach to formulating "maximal graph mining" in the graph domain of ARGs. Our method exhibits superior performance on RGB and RGB-D images.

People flow at a citywide level is in a mixed state with several basic patterns (e.g. commuting, working, commercial), and it is therefore difficult to extract useful information from such a mixture of patterns directly. In this paper, we proposed a novel tensor factorization approach to modeling city dynamics in a basic life pattern space (CitySpectral Space). To obtain the CitySpectrum, we utilized Non-negative Tensor Factorization (NTF) to decompose a people flow tensor into basic life pattern tensors, described by three bases i.e. the intensity variation among different regions, the time-of-day and the sample days. We apply our approach to a big mobile phone GPS log dataset (containing 1.6 million users) to model the fluctuation in people flow before and after the Great East Japan Earthquake from a CitySpectral perspective. In addition, our framework is extensible to a variety of auxiliary spatial-temporal data. We parametrize a people flow with a spatial distribution of the Points of Interest (POIs) to quantitatively analyze the relationship between human mobility and POI distribution. Based on the parametric people flow, we propose a spectral approach for a site-selection recommendation and people flow simulation in another similar area using POI distribution.

A large category model base can provide object-level knowledge for various perception tasks of the intelligent vehicle system. The automatic and efficient construction of such a model base is highly desirable but challenging. This paper presents a novel semi-supervised approach to discover possible prototype models of 3D object structures from the point cloud of a large and complex environment, given a limited number of seeds in an object category. Our method incrementally trains the models while simultaneously collecting object samples. Considering the bias problem of model learning caused by bias accumulation in a sample collection, we propose to gradually differentiate the standard category model into several sub-category models to represent different intra-category structural styles. Thus, new sub-categories are discovered and modeled, old models are improved, and redundant models for similar structures are deleted iteratively during the learning process. This multiple-model strategy provides several interactive options for the category boundary to deal with the bias problem. Experimental results demonstrate the effectiveness and high efficiency of our approach to model mining from “big point cloud data”.

The frequency and intensity of natural disasters has significantly increased over the past decades and this trend is predicted to continue. Facing these possible and unexpected disasters, accurately predicting human emergency behavior and their mobility will become the critical issue for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. In this paper, we build up a large human mobility database (GPS records of 1.6 million users over one year) and several different datasets to capture and analyze human emergency behavior and their mobility following the Great East Japan Earthquake and Fukushima nuclear accident. Based on our empirical analysis through these data, we find that human behavior and their mobility following large-scale disaster sometimes correlate with their mobility patterns during normal times, and are also highly impacted by their social relationship, intensity of disaster, damage level, government appointed shelters, news reporting, large population flow and etc. On the basis of these findings, we develop a model of human behavior that takes into account these factors for accurately predicting human emergency behavior and their mobility following large-scale disaster. The experimental results and validations demonstrate the efficiency of our behavior model, and suggest that human behavior and their movements during disasters may be significantly more predictable than previously thought.

Mobile phones are arguably one of the most prolific sources of large-scale human mobility data. The availability of this data has generated a massive body of research focused on understanding the dynamics and patterns of human mobility. However, it is increasingly evident that additional value can be derived from such data. This paper proposes a novel approach for understanding the attributes of mobile users by analyzing calling behavior derived from field survey data, in combination with call detail records (CDRs). Our survey reveals distinctive traits in calling behavior that correspond to user attributes. Analysis results demonstrate that frequent call locations, the variability in call time distributions, and the locations from which calls are made around midday are all keys to distinguishing gender. In addition, the location of calls initiated during the morning hours is a key to analyzing income levels for males.

Registering multiple views is an essential and challenging problem for many intelligent transportation applications that employ a mobile sensing platform or consist of multiple stationary sensors. In this paper a novel algorithm is presented for multiple-view registration under a loop closure constraint. Different from most existing methods, which use general optimization techniques, our method studies the mechanism of adjusting the poses of views in a loop and provides a highly efficient and accurate solution. We prove that translation vectors can be decoupled if the same point set is used in each view to associate the previous and subsequent views, leading to our solution for such decouplable cases. If this condition does not hold, an exact solution of translation vectors is provided when rotation parameters are given, which results in our iterative solution for general cases by updating rotation and translation alternately. In our method, the effect of the accumulated pose error in a loop can be distributed to all views efficiently through loop factors, and only a few iterations are needed. Most important of all, in each iteration our method has linear computational complexity with respect to the number of views, which is much superior to that of state-of-the-art methods. A series of experiments was conducted, involving simulation of thousands of views and real vehicle-borne sensing data that include 65 371 point pairs in 352 views. Experimental results show that our proposed method is not only stable and highly efficient but also provides competitive accuracy relative to existing methods.

An object model base that covers a large number of object categories is of great value for many computer vision tasks. As artifacts are usually designed to have various textures, their structure is the primary distinguishing feature between different categories. Thus, how to encode this structural information and how to start the model learning with a minimum of human labeling become two key challenges for the construction of the model base. We design a graphical model that uses object edges to represent object structures, and this paper aims to incrementally learn this category model from one labeled object and a number of casually captured scenes. However, the incremental model learning may be biased due to the limited human labeling. Therefore, we propose a new strategy that uses the depth information in RGBD images to guide the model learning for object detection in ordinary RGB images. In experiments, the proposed method achieves superior performance as good as the supervised methods that require the labeling of all target objects.

Although graph matching is a fundamental problem in pattern recognition, and has drawn broad interest from many fields, the problem of learning graph matching has not received much attention. In this paper, we redefine the learning of graph matching as a model learning problem. In addition to conventional training of matching parameters, our approach modifies the graph structure and attributes to generate a graphical model. In this way, the model learning is oriented toward both matching and recognition performance, and can proceed in an unsupervised fashion. Experiments demonstrate that our approach outperforms conventional methods for learning graph matching.

The building of an object-level knowledge base is the foundation of a new methodology for many perception tasks in artificial intelligence, and is an area that has received increasing attention in recent years. In this paper, we propose, for the first time, to mine category shape patterns directly from a large urban environment, thus constructing a category structure base. Conventionally, category patterns are learned from a large collection of object samples, but automatic object collection requires prior knowledge of category structures. To solve this chicken-and-egg problem, we learn shape patterns from raw segmentations, and then refine these segmentations based on the pattern knowledge. In the process, we solve two challenging problems of knowledge mining. First, as some categories have large intra-category structure variations, we design an entropy-based method to determine the structure variation for each category, in order to establish the correct range of sample collection. Second, because incorrect segmentation is unavoidable without prior knowledge, we propose a novel unsupervised method that uses a pattern competition strategy to identify and subtract shape patterns formed by incorrectly segmented objects. This ensures that shape patterns are meaningful at the object level. Experimental results demonstrated the effectiveness of the proposed method for category structure mining in a large urban environment.

Though a variety of indoor positioning systems have been proposed for a GPS-denied environment, most of them depend on dedicated devices which must be installed in a building. The preparation of this infrastructure usually entails considerable cost, which prevents the spread of indoor positioning technology. On the other hand, smartphones are already widespread and their low cost sensors are promising as rich data sources that can be used for indoor navigation. Several studies have demonstrated the possibility of smartphone-based infrastructure-free positioning. However, this navigation is not yet practical because of the accumulative error of inertial sensors and the communication range measurement error. We propose a solution to reduce the accumulative error of pedestrian dead reckoning carried out with only the low cost sensors and WiFi in smartphones by realizing cooperative positioning among multiple pedestrians. Our proposed method introduces linkage structures to simplify trajectories of pedestrians; these structures work as a constraint to reduce the number of variables to be estimated. Since this constraint makes positioning problems solvable with a small number of observations, our proposed method screened WiFi data by the signal strength and selected only strong signals for use in positioning. Our testing results showed that the positioning accuracy improved as the number of participants in the cooperative positioning operation increased. The positioning uncertainty was a few meters, comparable to GPS, when the density of participants is high enough.

The Great East Japan Earthquake and the Fukushima nuclear accident cause large human population movements and evacuations. Understanding and predicting these movements is critical for planning effective humanitarian relief, disaster management, and long-term societal reconstruction. In this paper, we construct a large human mobility database that stores and manages GPS records from mobile devices used by approximately 1.6 million people throughout Japan from 1 August 2010 to 31 July 2011. By mining this enormous set of Auto-GPS mobile sensor data, the short-term and long-term evacuation behaviors for individuals throughout Japan during this disaster are able to be automatically discovered. To better understand and simulate human mobility during the disasters, we develop a probabilistic model that is able to be effectively trained by the discovered evacuations via machine learning technique. Based on our training model, population mobility in various cities impacted by the disasters throughout the country is able to be automatically simulated or predicted. On the basis of the whole database, developed model, and experimental results, it is easy for us to find some new features or population mobility patterns after the recent severe earthquake, tsunami and release of radioactivity in Japan, which are likely to play a vital role in future disaster relief and management worldwide.

For reasons of public security, an intelligent surveillance system that can cover a large, crowded public area has become an urgent need. In this article, we propose a novel laser-based system that can simultaneously perform tracking, semantic scene learning, and abnormality detection in a fully online and unsupervised way. Furthermore, these three tasks cooperate with each other in one framework to improve their respective performances. The proposed system has the following key advantages over previous ones: (1) It can cover quite a large area (more than 60×35m), and simultaneously perform robust tracking, semantic scene learning, and abnormality detection in a high-density situation. (2) The overall system can vary with time, incrementally learn the structure of the scene, and perform fully online abnormal activity detection and tracking. This feature makes our system suitable for real-time applications. (3) The surveillance tasks are carried out in a fully unsupervised manner, so that there is no need for manual labeling and the construction of huge training datasets. We successfully apply the proposed system to the JR subway station in Tokyo, and demonstrate that it can cover an area of 60×35m, robustly track more than 150 targets at the same time, and simultaneously perform online semantic scene learning and abnormality detection with no human intervention.

Multitarget tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This article presents a promising online system to deal with these problems. The novelty of this system is that laser and vision are integrated with tracking and online learning to complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers online for “possible interacting targets”. When the targets are in close proximity, the classifiers learned online are used alongside visual information to assist in tracking. Therefore, this mode of cooperation not only deals with various tough problems encountered in tracking, but also ensures that the entire process can be completely online and automatic. Experimental results demonstrate that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a good trade-off between tracking accuracy and the time-cost factor.

A "mixed autoregressive hidden Markov model" (MAR-HMM) is proposed for modeling people's movements. MAR-HMM is equivalent to a special case of an autoregressive hidden Markov model (AR-HMM), which takes into account changes of people's internal properties. The number of parameters is thus reduced in the case of MAR-HMM. A dataset is applied to evaluate MAR-HMM in this study. The prediction rate of MAR-HMM is 56.8% and that of AR-HMM is 51.5%. It is therefore concluded that MAR-HMM is applicable to trajectory analysis of pedestrians.

Abnormal activity detection plays a crucial role in surveillance applications, and a surveillance system that can perform robustly in the extremely crowded area has become an urgent need for public security. In this paper, we propose a novel laser-based system which can simultaneously perform the tracking, semantic scene learning and abnormality detection in the large and crowded environment. In our system, a novel abnormality detection model is proposed, and it considers and combines various factors that will influence human activity. Moreover, this model intensively investigate the relationship between pedestrians' social behaviors and their walking scenarios. We successfully applied the proposed system to the JR subway station of Tokyo, which can cover a 60×35m area, robustly track more than 180 targets at the same time and simultaneously perform the online semantic scene learning and abnormality detection with no human intervention.

In our previous work, we reported a system that monitors an intersection using a network of horizontal laser scanners. This paper focuses on an algorithm for moving-object detection and tracking, given a sequence of distributed laser scan data of an intersection. The goal is to detect each moving object that enters the intersection; estimate state parameters such as size; and track its location, speed, and direction while it passes through the intersection. This work is unique, to the best of the authors' knowledge, in that the data is novel, which provides new possibilities but with great challenges; the algorithm is the first proposal that uses such data in detecting and tracking all moving objects that pass through a large crowded intersection with focus on achieving robustness to partial observations, some of which result from occlusions, and on performing correct data associations in crowded situations. Promising results are demonstrated using experimental data from real intersections, whereby, for 1063 objects moving through an intersection over 20 min, 988 are perfectly tracked from entrance to exit with an excellent tracking ratio of 92.9%. System advantages, limitations, and future work are discussed.

Abnormal activity detection plays a crucial role in surveillance applications, and such system has become an urgent need for public security. In this paper, we propose a novel laser-based system, which can perform the online detection of abnormal activity with an unsupervised way. The proposed system has the following key features that make it advantageous over previous ones: (1) It can cover quite a large and crowded area, such as subway station, public square, intersection and etc. (2) The overall system can vary with time period, incrementally learn the behavior pattern of pedestrians and perform the fully online detection of abnormal activity. This feature makes our system be quite suitable for the real-time applications. (3) The abnormal activity detection is carried out with a fully unsupervised way, there is no need for manual labelling and constructing the huge training datasets. We successfully applied the proposed system into the JR subway station of Tokyo, which can cover a 60×35m area, track more 150 targets at the same time and simultaneously perform the robust detection of abnormal activity with no human intervention.

In this study, we present a novel system for crowd surveillance and quantified analysis based on laser range scanners. By mounting a laser scanner at a swinging platform, the spatial information of passengers inside the area of interest can be reconstructed in a form of 3D points. Multiple laser scanners are integrated together by semi-auto calibration procedures. Background map is generated through histogram analysis of scan maps, and is further applied for 3D moving object detection. An improved version of mean-shift clustering algorithm is proposed to extract individual passengers efficiently. In addition, quantified crowdness analysis is conducted from different aspects to indicate the situation inside the surveillance area according to the extraction results of passengers. The proposed system was tested in a central subway station in Tokyo and experimental results demonstrate the effectiveness of our proposed system.

This study analyzes the walking improvement of young students, who have not suffered any orthopedic problems, while walking listening to sounds, based on 1/f (f=frequency) fluctuation of center body gravity (COG) using Auto Regressive (AR) modeling, provides clear results for the evaluation of improvement to walking stability. While walking listening to Mozart's music, the power spectra of COG fluctuation was close to 1/f fluctuation, and their curves in high frequency ranges were smooth in comparison to that of non-while listening to music. In the case of listening to rock music, the power spectra of COG fluctuation while walking was out of proportion of 1/f fluctuation in high frequency ranges in comparison to that of students who listened to Mozart's music and those which didn't. Mozart music indicates walking improvement whether students like it or not. This study provides a useful method to improve physical fitness and a means for subjects to maintain a state of well-being.

We assessed walking stability, using a portable acceleration measurement system. We measured the acceleration displacement at the center of gravity (COG) of a normal walking subject and evaluated walking stability. We noted the dispersion of high and low acceleration at the COG that is a factor of walking stability. We analyzed the two coefficients of variation (CV) of high and low acceleration in every walking cycle. Comparing same subjects with normal, flat shoes and slippers, we found that the difference of the two CV. In the case of the subjects wearing normal shoes, two kind of CV were within 20%. However, wearing slippers that are hard to walk, the CV for high or low accelerations were over 25%. Moreover by using the method of support vector machine (SVM), we classify walking stability and instability. Using these analyses of CV and SVM provide useful information for understanding walking stability in preventive medicine and rehabilitative medicine.

Learning the knowledge of scene structure and tracking a large number of targets are both active topics of computer vision in recent years, which plays a crucial role in surveillance, activity analysis, object classification and etc. In this paper, we propose a novel system which simultaneously performs the Learning-Semantic-Scene and Tracking, and makes them supplement each other in one framework. The trajectories obtained by the tracking are utilized to continually learn and update the scene knowledge via an online un-supervised learning. On the other hand, the learned knowledge of scene in turn is utilized to supervise and improve the tracking results. Therefore, this “adaptive learning-tracking loop” can not only perform the robust tracking in high density crowd scene, dynamically update the knowledge of scene structure and output semantic words, but also ensures that the entire process is completely automatic and online. We successfully applied the proposed system into the JR subway station of Tokyo, which can dynamically obtain the semantic scene structure and robustly track more than 150 targets at the same time.

Multi-target tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This paper presents a promising tracking system to deal with these problems. The novelty of this system is that laser and vision, tracking and learning are integrated and can complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers for the “possible interacting targets”. When the targets are in close proximity, the learned classifiers and visual information are used to assist in tracking. Therefore, this mode of co-operation between them not only deals with various tough problems encountered in the tracking, but also ensures that the entire process can be completely on-line and automatic. Experimental results demonstrated that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a perfect trade-off between tracking accuracy and time-cost.

To seek and collect useful sensor data in a participatory sensing environment, participants should be able to coordinate their activities in a timely manner. However, existing systems deal separately with the "preparation time" to define the goal and method of sensing, and the "sensing time" in the field. Therefore, participants cannot easily modify the sensing method, including the collaboration setting, once they are out in the field. In this demonstration, we will present a system called Kitokito, which allows participants to easily create small sensing tasks, and iteratively modify the collaboration and sensing method during the "sensing time."

We propose a novel system for extracting multiple people in crowded scenes by employing LED 3D range sensor. This new kind of device can capture depth image at a relatively high frame speed, which makes the extraction of dynamic objects possible. However, it suffers from the strong noise caused by the environment illumination. Here we propose a novel method for multiple people extraction by integrating an improved version of mean-shift clustering algorithm and total variation based denoising technique. When handling the depth image, it is considered both as a point cloud as well as a 2D image. In this way, different properties of the depth image are sufficiently exploited. The proposed method can fully automatically extract multiple people in a cluttered environment, even from a mobile platform. The experiment conducted at the platform of a railway station demonstrates the effectiveness of our proposed algorithm.

Motivated by two potential applications, i.e. enhancing driving safety and traffic data collection, a system has been developed using a single-layer horizontal laser scanner as the major sensor for both localization and perception of the surroundings in a large dynamic urban environment. This research focuses on a classification method, that given a stream of laser measurements, classify the moving object into either a person, a group of people, a bicycle or a car. In this research, a number of features are defined after examining the property of data appearance. A classification method is proposed after examining the likelihood measures between each pair of feature and class. Experimental results are presented, demonstrating that the algorithm has efficiency with respect to both driving safety and traffic data collection in highly dynamic environment.

To represent 3-D space in detail, it is necessary to acquire 3-D shapes and textures simultaneously and efficiently through the use of precise trajectories of sensors. However, there is no reliable, quick, cheap, and handy method for acquiring accurate high-resolution 3-D data on objects in outdoor and moving environments. In this paper, we propose a combination of charge-coupled device cameras, a small and inexpensive laser scanner, an inexpensive inertial measurement unit, and Global Positioning System for a UAV-borne 3-D mapping system. Direct georeferencing is achieved automatically using all of the sensors without any ground control points. A new method of direct georeferencing by the combination of bundle block adjustment and Kalman filtering is proposed. This allows objects to be rendered richly in shape and detailed texture automatically via a UAV from low altitude. This mapping system has been experimentally used in recovery efforts after natural disasters such as landslides, as well as in applications such as river monitoring.

This work is motivated by the following two potential applications: 1) enhancing driving safety and 2) collecting traffic data in a large dynamic urban environment. A laser-scanner-based approach is proposed. The problem is formulated as a simultaneous localization and mapping (SLAM) with object tracking and classification, where the focus is on managing a mixture of data from both dynamic and static objects in a highly dynamic environment. A trajectory-oriented closure is also proposed using the sporadically available global positioning system (GPS) measurements in urban areas to assist for global accuracy, particularly when the vehicle makes a noncyclical measurement in a large outdoor environment. Experiments are conducted using the data that were collected along a course near 4.5 km in a highly dynamic environment. Possibilities of the approaches toward the two potential applications are demonstrated, and avenues for future works are discussed.

In this research, we propose a method of SLAM in a dynamic large outdoor environment using a laser scanner. Focus are cast on solving two major problems: 1) achieving global accuracy especially in non-cyclical environment, 2) tackling a mixture of data from both dynamic and static objects. Algorithms are developed, where GPS data and control inputs are used to diagnose pose error and guide to achieve a global accuracy; Classification of laser points and objects are conducted not in an independent module but across the processing in a framework of SLAM with moving object detection and tracking. Experiments are conducted using the data from two test-bed vehicles, and performance of the algorithms are demonstrated.

We propose a novel system for tracking a variable number of pedestrians in crowded scenes by exploiting laser range scanners. Based on the specific pattern generated by walking feet in the spatio-temporal domain, a walking model is constructed and applied to track pedestrians. To track interactive targets, an algorithm based on Interactive Multiple Particle Filters (IMPF) is proposed, whose computation load increases linearly with the number of targets. To handle a variable number of pedestrians, spatio-temporal correlation analysis in combination with a mean shift based clustering technique is proposed. Compared with camera-based surveillance methods, our system provides a novel technique for automatically tracking a large number of pedestrians in a relatively large area. The experiments, in which over 2600 pedestrians were tracked in 10 minutes at a 60 m × 20 m subway station, show the effectiveness of our proposed algorithm.

In this research, we proposed a system of detecting and monitoring pedestrians' motion trajectories at a populated and wide environment, such as exhibition hall, supermarket etc., using the horizontally profiling single-row laser range scanners on a mobile platform. A simplified walking model is defined to track the rhythmic swing feet at the ground level. Pedestrians are recognized by detecting the braided styles, which is a typical appearance that could discriminate the data of moving feet with other mobile and motionless objects. Two experiments are conducted. One is at the laboratory environment, the purpose of which is to examine the algorithm in details. Another is at an exhibition hall, a populated and wide environment, the purpose is to examine whether the system could be applied for practical needs. It is a big challenge, while the system did well. Pedestrians in the exhibition hall at the moment of measurement are detected. Their motion trajectories are extracted, and associated to the background map, which is made of the motionless objects, and covers the whole exhibition hall.

We describe approaches that the Global Earth Observation System of Systems (GEOSS) is employing to achieve interoperability among its component systems and give an overview of the GEOSS Interoperability Process Pilot Project (IP3). The IP3 is contributing to the development of the GEOSS architecture by exercising core GEOSS components and processes.

We propose a novel system for tracking multiple pedestrians in a crowded scene by exploiting single-row laser range scanners that measure distances of surrounding objects. A walking model is built to describe the periodicity of the movement of the feet in the spatial-temporal domain, and a mean-shift clustering technique in combination with spatial- temporal correlation analysis is applied to detect pedestrians. Based on the walking model, particle filter is employed to track multiple pedestrians. Compared with camera-based methods, our system provides a novel technique to track multiple pedestrians in a relatively large area. The experiments, in which over 300 pedestrians were tracked in 5 minutes, show the validity of the proposed system.

Global maps of land cover and leaf area index (LAI) from the moderate resolution imaging spectroradiometer (MODIS) are available from the earth resources observation system data center. Validation of these MODIS terrestrial products is crucial for scientific research. Numerous pre- and postlaunch validation activities have been completed during the last several years. However, such work was mainly located in North America, Europe, and Africa. The validated vegetation types were principally focused on woodland and savanna. In this paper, we evaluated the Collection 3 MODIS land cover (MOD12Q1) and the Collection 4 MODIS LAI (MOD15A2) products in the North China Plain (NCP). We placed the emphasis on croplands by: (1) developing an accurate land cover map, based on the MODIS LAI/fraction of photosynthetically active radiation land cover classification scheme; (2) deriving empirical LAI validation maps, based onin situmeasurement and landsat thematic mapper imagery; and (3) conducting an initial MODIS validation exercise to assess the quality of MODIS products. The results indicate that an apparent misclassification exists between grasses/cereal crop and broadleaf crop biomes in the MODIS land cover product. Although MODIS LAI could resemble the general pattern of the LAI seasonal vegetation variation in the NCP, the MODIS LAI algorithm markedly underestimated LAI values (down to 2-3 m2/m2) for the April 2004 winter wheat cropland. Biome misclassification and the atmospheric effect of cloud or aerosol were proven to decrease the value of MODIS LAI, but these impacts cannot fully explain the significant underestimation. Additional studies are needed to understand the main reasons for this underestimation in the study area.

A novel image-mosaicking technique suitable for 3-D visualization of roadside buildings on websites or mobile systems is proposed. Our method was tested on a roadside building scene taken using a side-looking video camera employing a continuous set of vertical-textured planar faces. A vertical plane approximation of the scene geometry for each frame was calculated using sparsely distributed feature points that were assigned 3-D data through bundle adjustments. These vertical planes were concatenated to create an approximate model on which the images could be backprojected as textures and blended together. Additionally, our proposed method includes an expanded crossed-slits projection around far-range areas to reduce the "ghost effect," a phenomenon in which a particular object appears repeatedly in a created image mosaic. The final step was to produce seamless image mosaics using Dijkstra's algorithm to find the optimum seam line to blend overlapping images. We used our algorithm to create efficient image mosaics in 3-D space from a sequence of real images.

Most of visual tracking algorithms have been achieved by matching-based searching strategies or detection-based data association algorithms. In this paper, our objective is to analysis laser scan image sequences to track multiple people in a crowded environment. Due to the poor features provided by laser scan images, neither of the above two approaches can achieves good tracking. To address the problem, we propose a novel multiple-target tracking algorithm fusing both detection and matching based strategies. First, target to detected measurement data association is incorporated to the joint state proposal, to form a mixture proposal that combines information from the dynamic model and the detected measurements. And then, we utilize a MCMC sampling step to obtain a more efficient multi-target filter. Our approach has been applied to the real laser scan image data. Evaluations show that the proposed method is a robust and effective multi-target tracking algorithm.

Provides a listing of current committee members.

This study presents the research result on estimating winter wheat yield in North China Plain, by assimilating multi- temporal Landsat TM images into the GIS-based EPIC model (Erosion Productivity Impact Calculator, recently renamed Environmental Policy Integrated Climate). The results indicate that the assimilation of LAI maps retrieved from remotely sensed data has improved the yield simulation accuracy of GIS-based crop growth model at the regional scale significantly.

Laser based people tracking systems have been developed for mobile robotics and intelligent surveillance areas. Existing systems rely on simple laser point clustering methods to extract object locations. However, when dealing with multiple interacting people, laser points of different persons are often interlaced and undistinguishable due to measurement noise and they can not provide reliable features. It causes current systems quite fragile and unreliable. In this paper, we try to explore potentials from multi-level observations including weakly detected features, stably extracted features and foreground points. For inference, detection incorporated joint particle filter is used. And stably extracted features are utilized to properly estimate parameters of dynamic model for each target. In real experiments, we obtain raw data from multiple registered laser scanners, which measure two legs for each people. Evaluations with real data show that the proposed method is more robust and effective than existing approaches

We propose a novel system for analyzing pedestrians' walking patterns by exploiting single-row laser range scanners that measure distances of surrounding objects by reflecting eye-safe laser beams. A walking model is built in the spatial-temporal domain to describe the periodicity of the movement of the feet, and a mean-shift technique is applied to recover model parameters. Compared with camera-based methods, our system provides a novel technique to analyze the behavior of pedestrians. The experiments show the validity of the algorithm.

The Advanced Land Observing Satellite (ALOS) has been developed to contribute to the fields of mapping, precise land coverage observation, disaster monitoring, and resource surveying and will be launched in 2005. The Panchromatic Remote-sensing Instrument for Stereo Mapping (PRISM) carried at ALOS satellite is expected to generate worldwide Digital Maps in respects of its high resolution and stereoscopic observation. The development and evaluation of the software for generating digital surface model (DSM), based on this aim of ALOS operation, is now undertaking for the purpose of equipping the system which producing DSM semi-routinely in the Earth Observation Research and Application Center (EORC). In this paper, the statuses of pre-launch simulations and DSM assessment plans are described. PRISM consists of three panchromatic radiometers for forward, nadir, and backward views and acquires each image in the same orbit at almost the same time. It enables to generate more precision DSM frequently than usual stereoscopic observations. For this specific stereo image configuration, the triplet images matching algorithm is applied for the DSM generation. To evaluate the algorithm, three line scanner (TLS) data is used as the PRISM simulation data. The DSM generation using the JERS-1 OPS stereo images is being performed as the operational simulation. The basic functions of the software such as bundle adjustment, DSM mosaic, mask operation, etc. are evaluated in those simulations. The accuracy assessment of generated DSM is planned to be performed using the LIDAR DSM data, SRTM data, GCP etc.

We present a novel system that aims at reliably detecting and tracking multiple people in an open area. Multiple single-row laser scanners and one video camera are utilized. Feet trajectory tracking based on registration of distance information from multiple laser scanners and visual body region tracking based on color histogram are combined in a Bayesian formulation. Results from tests in a real environment are reported to demonstrate that the system can detect and track multiple people simultaneously with reliable and real-time performance.

We propose a novel system for tracking pedestrians in a wide and open area, such as a shopping mall and exhibition hall, using a number of single-row laser-range scanners (LD-A), which have a profiling rate of 10 Hz and a scanning angle of 270/spl deg/. LD-As are set directly on the floor doing horizontal scanning at an elevation of about 20 cm above the ground, so that horizontal cross sections of the surroundings, containing moving feet of pedestrians as well as still objects, are obtained in a rectangular coordinate system of real dimension. The data of moving feet are extracted through background subtraction by the client computers that control each LD-A, and sent to a server computer, where they are spatially and temporally integrated into a global coordinate system. A simplified pedestrian's walking model based on the typical appearance of moving feet is defined and a tracking method utilizing Kalman filter is developed to track pedestrian's trajectories. The system is evaluated through both real experiment and computer simulation. A real experiment is conducted in an exhibition hall, where three LD-As are used covering an area of about 60/spl times/60 m/sup 2/. Changes in visitors' flow during the whole exhibition day are analyzed, where in the peak hour, about 100 trajectories are extracted simultaneously. On the other hand, a computer simulation is conducted to quantitatively examine system performance with respect to different crowd density.

The Advanced Land Observing Satellite (ALOS) has been developed to contribute to the fields of mapping, precise land coverage observation, disaster monitoring, and resource surveying. The Panchromatic Remote-sensing Instrument for Stereo Mapping (PRISM), carried on ALOS satellite is expected to generate worldwide Digital Maps in respects of its high resolution and stereoscopic observation. The development and evaluation of the software for generating Digital Elevation Model (DEM), based on this aim of ALOS operation, is now undertaking for the purpose of equipping the system which producing DEM semi-routinely in the Earth Observation Research and application Center (EORC). In this paper, the software's outline of functions, the algorithm and the current status of its development and evaluation are described. PRISM consists of three panchromatic radiometers for forward, nadir, and backward views and acquires each image in the same orbit at almost the same time. It enables to generate more precision DEM frequently than usual stereoscopic observations. For this specific stereo image configuration, the triplet images matching algorithm is applied for the DEM generation. To evaluate the algorithm, Three Line Scanner (TLS) data is used as the PRISM simulation data and the Lider DEM is used as the reference data. The absolute accuracy of generated DEM is estimated in this simulation

Traffic congestion on roads in cities such as Tokyo is becoming worse. Therefore, collecting on-street vehicle statistics is important for traffic management in the short or long term. Therefore, a new approach to vehicle detection using the Three Line Scanner or TLS, and new imagery has been proposed. The framework of the approach consists of four steps: preparation, vehicle detection, moving/stopping vehicle discrimination and parking/signals waiting vehicle classification. This study focuses on the first two steps of a study framework. Pre-processing is the approach to prepare fundamental information for further processing such as image segmentation, non-vehicle removal, building shadow correction and so on. The second stage is the algorithm of vehicle detection using a vehicle explicit model on ultra-resolution images of TLS. The third stage involves the newly-proposed concepts of moving/parking discrimination by applying stereoscopic measurement. The final stage is the new approach of parking/signal waiting classification by integrating a digital road map. The first two stages of a framework have been fulfilled with promising results. The third stage has been proposed conceptually and the last stage is being developed with conceptual design.

In this research, a novel vehicle-borne system of measuring three-dimensional (3-D) urban data using single-row laser range scanners is proposed. Two single-row laser range scanners are mounted on the roof of a vehicle, doing horizontal and vertical profiling respectively. As the vehicle moves ahead, a horizontal and a vertical range profile of the surroundings are captured at each odometer trigger. The freedom of vehicle motion is reduced from six to three by assuming that the ground surface is flat and smooth so resulting in the vehicle moving on almost the same horizontal plane. Horizontal range profiles, which have an overwhelming overlay between successive ones, are registered to trace vehicle location and attitude. Vertical range profiles are aligned to the coordinate system of the horizontal one according to the physical geometry between the pair of laser range scanners, and subsequently to a global coordinate system to make up 3-D data. An experiment is conducted where 3-D data of a real urban scene is obtained by registering and integrating 2412 horizontal and vertical range profiles. Two ground truths are used in examination. They are the outputs of a GPS/INS/Odometer based positioning system and a 1:500 digital map of the testing site. Accuracy and efficiency of the method in measuring 3-D urban scene is demonstrated.

The Panchromatic Remote sensing Instrument for Stereo Mapping (PRISM) sensor that will be carried on the Advanced Land Observing Satellite (ALOS) includes creation of digital elevation maps as one of its purposes. PRISM is capable of high spatial resolution and along-track stereoscopic observation. In order to provide software to create a Digital Elevation Model (DEM) semi-routinely at the Earth Observation Research Center (EORC), development of a DEM generation algorithm and software was carried out. The authors considered the basic function of DEM generation software, various processing algorithms for DEM generation were examined, and a trial production and evaluation which used test data was performed This paper reports the resulting processing method, and evaluation of results, of DEM generated from test data consisting of the nadir and forward images from JERS-1/OPS.

This article focuses on developing a predictive method to simulate the distribution of agricultural land use patterns based on remote sensing and a multiple choice logit model. A multiple choice model is developed to determine land use options such as rice cropland, maize cropland, wheat cropland, soybean cropland and so on. The independent variables are grouped under two categories: basic environment variables and socioeconomic variables. The basic environment variables include natural factors and anthropogenic factors. Natural factors include basic conditions mainly to estimate crop productivity, such as temperature, precipitation, soil physical and chemical properties, and terrain variables. A GIS-based EPIC (Erosion Productivity Impact Calculation) model, which is developed to simulate land productivity for different crops in each grid at regional level, is embedded in multiple choice logit model. Remote sensing is used to collect the sample points of agricultural land use. Examinations show that the results in our test are acceptable.

A useful analyzer system has been developed for walking stability. It is composed of two sub-systems. One acquires time series data of joints' movements using a newly developed, synchronized system of two accelerometers (SSA); the other is a system for analyzing the person's walking stability. It utilizes the impulse response by autoregressive (AR) modeling. The impulse response of movement factors enables the analysis of their dynamic relations. The biofeedback control analysis of walking stability using this analyzer system suggests new potential for the understanding of medical recovery and kinesitherapy in rehabilitation and physical fitness in daily movement.

Presents a useful walking analyser system (WAS). The set-up for acquiring data is a small wearable personal computer (WPC) assisted system. It employs a lithium battery-powered gyro-sensor. While walking, the WPC is worn like a wrist-watch, the small sensor is fixed to the person's back, near the body's centre of gravity, and the batteries are inserted into a waist pouch. When the WPC switch is turned on, three relative angles (i.e. roll, pitch and yaw) and accelerations in the X, Y and Z axes of the person's back are recorded at 30 Hz. People who wear dentures were selected as subjects at the department of geriatric dentistry in a dental hospital. It was observed that, in the case of people wearing dentures, the angular displacements and the accelerations of the back in walking had a better rhythm compared to those without dentures. As an effective analyser system for walking stability, power spectral analysis is used, utilizing autoregressive modelling. In the case of people not wearing dentures, their power spectrum densities of the three angles and accelerations of back fluctuation were wavy, showing unstable walking. In the case of people wearing dentures, their power spectrum densities were approximately close to a 1/f/sup 3/ fluctuation (f = frequency), i.e. they were stable when walking in all directions. Using this WAS, it is easy to discover people's walking stability in rehabilitation and physical fitness.

This paper presents a useful analyzer system for human movement stability, which is composed of two developed sub-systems. One is a system of acquiring time series data of person's movement factors, the other is a system of analyzing the person's movement stability. The setup of acquiring data system, which is a personal computer-assisted, employs two CCD digital video cameras and a force-plate sensor system, which are controlled simultaneously. Time series data of 3D coordinates on major joints are measured from a video recorder system using image processing and videometry. Time series data of each foot pressure are measured from force-plate sensor system. For a spectral analysis, utilizing an autoregressive model, the power spectrum and impulse response of movement factors can used to analyse the dynamical relations between the body centre of gravity and foot pressure. This human movement stability analysis system provides new medical evaluation in rehabilitation and physical fitness.

The developed system is composed of two CCD video cameras and a force plate sensor, which are controlled simultaneously by a personal computer. By this system, time-series data of 3D coordinates of joints and each foot's pressure enable to provide efficient bio-feedback control analysis of postural stability utilizing autoregressive modeling. After therapy, in the case of 80% of patients, the power spectrum of the body's center of gravity fluctuations became inversely proportional to frequency, and impulse response recovered.