In the context of smart cities, there is a growing demand for risk-free navigation systems that help citizens avoid congestion and enjoy outdoor activities in clean environments. Such systems require the ability to predict traffic and environmental hotspots by analyzing spatial-temporal big data from various sources, including IoT devices, stations, CCTV, and personal device networks. However, analyzing and forecasting spatial-temporal data presents significant challenges due to the complex interplay of spatial and temporal dependencies. To address these challenges, we introduce a novel dual attention mechanism graph transformer that leverages both spatial and temporal information to capture intricate patterns in spatial-temporal data. We evaluate our model on two forecasting tasks: air pollution and traffic flow. Our results demonstrate superior performance compared to other graph neural network models. Consequently, this model has been integrated into a traffic-risk navigator application, which will be evaluated in Yokohama, Japan, using real data collected from air pollution stations and traffic monitors.

This paper presents an innovative approach to predicting near-miss accidents, vital for proactive accident prevention. Leveraging dashcam footage and weather sensor data, our model integrates camera calibration, collision point prediction, heuristic knowledge, and analysis of near-miss accident patterns. We propose a comprehensive method to detect and predict potential collisions within the ego-vehicle's safe zone, utilizing a combination of machine learning techniques including DeepHough, YOLOv8, and LSTM. Furthermore, we introduce heuristic rules to handle sudden changes in object behavior and enhance object detection accuracy under challenging conditions like low visibility. Our approach identifies common near-miss accident patterns and achieves a prediction accuracy of 96.01% with support from hard brake detection. Comparative analysis demonstrates the superior performance of our method against existing benchmarks. Moreover, our lightweight model is designed for deployment on edge clients, ensuring real-time assistance to drivers. Collaboratively developed with government and industry stakeholders, our approach contributes to creating cost-effective smart driving assistance systems with wide-ranging applications in traffic safety and accident analysis.

This paper introduces MM-TrafficRisk, an innovative fleet management application that harnesses dashcam video data, environmental data, and physiological data to forecast, mitigate, and investigate traffic-risk events while uncovering traffic-risk patterns. To provide a comprehensive overview of this application, we outline its system architecture, encompassing a database, ETL processes, UI/UX components, and a fine-grained text-video search engine. Additionally, we present a groundbreaking two-stage near-miss accident prediction model designed to identify near-miss incidents within dashcam video databases, and the text-to-video search engine, facilitating rapid searches for traffic-risk events based on textual queries. These models and search engines are rigorously evaluated in a controlled laboratory environment to showcase their advantages. Moreover, we highlight several essential functions of the MM-TrafficRisk application through snapshots, emphasizing the collaborative efforts between a government agency and industrial companies to develop and deploy this application in practical settings. We also delve into our future endeavors, focusing on multi-modal deep learning event prediction and the adaptability of our application to Edge AI environments.

The paper aims to present a collaboration between the industry and government sectors, focusing on creating a benchmarking platform for predicting urban risk transportation through the utilization of multimodal data. In this collaboration, the industry partner contributes datasets and customer preference surveys obtained from its business operations. On the other hand, government partners curate open datasets sourced from non-profit organizations in both private and public domains. Furthermore, the government provides an accessible platform that allows individuals to conveniently access and leverage resources for the purpose of advancing application development and engaging in research endeavors. Throughout the collaborative effort, a variety of techniques have been under development for forecasting urban risk transportation through the analysis of weather patterns, congestion levels, and people flow data. The core objective of this partnership is to formulate two foundational prediction methods. These methods are intended to serve as benchmarks, offering future users a dependable means to assess the performance of their own approaches in terms of both time-series and datapoints analytics methodologies.

Driver reaction is of vital importance in risk scenarios. Drivers can take correct evasive maneuver at proper cushion time to avoid the potential traffic crashes, but this reaction process is highly experience-dependent and requires various levels of driving skills. To improve driving safety and avoid the traffic accidents, it is necessary to provide all road drivers with on-board driving assistance. This study explores the plausibility of case-based reasoning (CBR) as the inference paradigm underlying the choice of personalized crash evasive maneuvers and the cushion time, by leveraging the wealthy of human driving experience from the steady stream of traffic cases, which have been rarely explored in previous studies. To this end, in this paper, we propose an open evolving framework for generating personalized on-board driving assistance. In particular, we present the FFMTE model with high performance to model the traffic events and build the case database; A tailored CBR-based method is then proposed to retrieve, reuse and revise the existing cases to generate the assistance. We take the 100-Car Naturalistic Driving Study dataset as an example to build and test our framework; the experiments show reasonable results, providing the drivers with valuable evasive information to avoid the potential crashes in different scenarios.

Sleep plays a vital role in our physical, cognitive, and psychological well-being. Despite its importance, long-term monitoring of personalized sleep quality (SQ) in real-world contexts is still challenging. Many sleep researches are still developing clinically and far from accessible to the general public. Fortunately, wearables and IoT devices provide the potential to explore the sleep insights from multimodal data, and have been used in some SQ researches. However, most of these studies analyze the sleep related data and present the results in a delayed manner (i.e., today’s SQ obtained from last night’s data), it is sill difficult for individuals to know how their sleep will be before they go to bed and how they can proactively improve it. To this end, this paper proposes a computational framework to monitor the individual SQ based on both the objective and subjective data from multiple sources, and moves a step further towards providing the personalized feedback to improve the SQ in a data-driven manner. The feedback is implemented by referring the insights from the PMData dataset based on the discovered patterns between life events and different levels of SQ. The deep learning based personal SQ model (PerSQ), using the long-term heterogeneous data and considering the carry-over effect, achieves higher prediction performance compared with baseline models. A case study also shows reasonable results for an individual to monitor and improve the SQ in the future.

Split learning (SL) is a popular distributed machine learning (ML) method used to enable ML. It divides a neural network based model into subnetworks. Then, it separately trains the subnetworks on distributed parties (e.g., client and server). In distributed ML, data are generated and collected on the client-side. In contrast, the collected data are processed using an application deployed on the server side. However, when applied in practice using Internet of things systems and clients, numerous obstacles occur because of limited configuration and resources. Dividing neural networks in the SL is the biggest problem and an open question in numerous studies. This study introduces splitDyn, which is a new dynamic SL solution to solve the aforementioned problems. This method provides a solution for eliminating their inherent drawbacks. The main idea is to apply a Round-Robin schedule to select the client for the training process. Then, the next idea is to use the Hungarian optimization algorithm to assign a layer to a client and enhance the accuracy. The proposed method reasonably achieved better accuracy and reduced processing time than the other learning models. Furthermore, it applies the incident datasets to predict the incident event and in edge computing for edge artificial intelligence (AI) applications.

Federated learning (FL) has been used to protect data contributors’ privacy by allowing training at clients and then feeding back machine learning models to servers for aggregation. Conventional methods of FL aggregation methods average model weights to produce a fused global model. However, in real-world applications in cyber-space systems, which often have heterogeneous Internet of Things data configuration and collection, FL encounters obstacles with non-independent and identically distributed (Non-IID) data. The main problem is the aggregated global models deviating from the optimal model trained on centralized servers. According to recent research, most Non-IID FL aggregation methods attempt to direct the movement of gradients to the optimal one using differentiation from trained models. In this paper, we propose a framework for using feature probability distribution in aggregation calculation. The proposed aggregation algorithm shows robustness on different Non-IID datasets and outperforms state-of-the-art methods in various FL experiments.

A geo-referenced time series database represents the data generated by a set of fixed locations (or spatial items) observing a particular phenomenon over time. This data hides valuable information that can help users progress in their social and economic lives. This paper presents a new model of Geo-referenced Periodic-Frequent Patterns (GPFPs) that might be in these databases. A GPFP is a set of frequently occurring items close to each other and seen in a database at regular intervals. Three constraints have been used to figure out how interesting a pattern is in a geo-referenced time series database: maximum distance (maxDist), minimum support (minSup), and maximum periodicity (maxPer). The maxDist controls how far apart the items in a pattern can be. The minimum number of times a pattern must appear in the data is controlled by the minSup. Lastly, the maxPer variable specifies how many times a pattern must repeat before it is considered periodic in the data. Each pattern that satisfies these three requirements will be returned. An effective method known as the Geo-referenced Periodic-Frequent Pattern-Miner (GPFP-Miner) has been proposed to discover all GPFPs included inside a geo-referenced time series database. GPFP-Miner uses an innovative, smart depth-first search approach to uncover required patterns efficiently. The findings of the experiments support the contention that the proposed algorithm is effective. In addition, we present two case studies in which we utilise our methodology to extract meaningful information from databases pertaining to air pollution and traffic congestion.

Dashcam video has become popular recently due to the safety of both individuals and communities. While an individual can have undeniable evidence for legal and insurance, communities can benefit from sharing these dashcam videos for further traffic education and criminal investigation. Moreover, relying on recent computer vision and AI development, few companies have launched the so-called AI dashcam that can alert drivers to near-risk accidents (e.g.., following distance detection, forward collision warning), forwarding to improving driver’s safety. However, even though dashcam videos create a driver’s travel log (i.e., traveling diary), little research focuses on creating a valuable and friendly tool to find any incident or event with few described sketches by users. Besides, most incident detection models have been built using a traditional supervised learning approach (i.e., collecting and labeling data for a new incident class). That prevents the quick and customized development of a new incident class. Inspired from these observations, we introduce an interactive incident detection and retrieval system for first-view travel-log data, namely MM-trafficEvent, that can (1) online defined-incident detection, (2) offline fine-grained incident retrieval for both defined and undefined incidents, (3) offline automatically new incident class creating using user’s queries. Moreover, the system gives promising results when being evaluated on several public datasets.

Due to the rapid recent development of cloud-edge networks, smart devices can facilitate rapid access to patients’ health information. Success has been achieved in the healthcare sector with the training of a federated learning (FL) model on large amounts of the personal data of users. However, some challenges remain that other FL models have not yet addressed. Firstly, FL models with computational parameters are very complex, which results in a high communication cost in the cloud-edge network. Furthermore, trained models in the cloud are not personalized. If personalization is present, the models do not provide practical solutions to fine-tune parameters in order to accurately predict performance in health monitoring. To address the above challenges, this paper presents the Fed xData framework for contextual health monitoring in cloud-edge networks. The Fed xData framework introduces a continuous data balancing supplemented structure using the RandomOverSample method, which solves all data classes. The FL model is an encode depth convolutional network (EDCN) model designed for both server and client. It solves various problems, for instance by using the fine-tuning model to increase personalization and solving not independent and identically (Non-IID) distribution problems regarding user health. Test results based on human activity recognition indicate that Fed xData is far superior to others for use in general centralized learning models and FL models.

Currently, air pollution has become the tremendous problem humankind has ever faced. Countries governments have struggled with conflicting demands from industries benefits and people/natural health. Scientists have investigated to find solutions that can balance these demands. Along with these directions, this research introduces a convenient and economical solution for estimating PM2.5 at the current time and predicting PM2.5 in a short- and medium-term period just by using images captured from personal devices (e.g., smartphones, cameras, lifelog cameras). The proposed method aims to leverage the association between urban nature (e.g., street greenness, street building), urban traffic (e.g., the volume of vehicles), and air pollution (e.g., PM2.5) to discover a set of periodic-frequent patterns and to build the PM2.5 estimation model. The estimated PM2.5, together with a set of patterns, is utilized to predict the PM2.5 in the short-term future. Evaluation running on different datasets collected from India and Vietnam shows the productivity of the proposed method.

Partial periodic pattern mining is an important model in data mining with many real-world applications. However, this model’s successful industrial application was hindered by the problem of combinatorial explosion of patterns, which involves generating too many patterns, most of which might be redundant or uninteresting to the user. Furthermore, the problem of combinatorial explosion increases the memory, runtime, and the energy requirements of a mining algorithm. This paper aims to tackle this challenging problem by proposing a novel model of maximal partial periodic pattern that may exist in a database. We also present a new tree structure and a pattern-growth algorithm, called Maximal Partial Periodic Pattern-growth (max3P-growth), to find all desired patterns effectively. Experimental results demonstrate that the proposed model prunes many redundant patterns, and the max3P-growth is efficient and scalable. Finally, we show the usefulness of our model with a case study on traffic congestion analytics.

Air pollution prediction for smart city applications has been attracted in artificial intelligence research to overcome problems to the health of citizen. Conventionally, environmental IoT data is gathered from monitoring station sensors then is sent to servers for centralized predictive model training at a whole region. That causes latency issues in data transmission from IoT devices to cloud servers. This paper describes federated learning paradigm approach for air pollution prediction model training on environmental monitoring sensor data. In the research, we design distributed learning framework that assists cooperative training among participants from different spatial areas such as cities and prefectures. At each area, Convolutional Recurrent Neural Networks (CRNN) are trained locally aiming to predict local Oxidant warning level while aggregated global model enhances distilled knowledge from all areas of a region. The research illustrates that designed common parts of CRNN can be fused globally meanwhile adaptive structure at predictive part of the deep neural network model can capture different environmental monitoring stations configuration at local areas. Some experiment results also hint methods to keep balance between federated learning synchronous training rounds and local deep neural network training epochs to maximize accuracy of the whole federated learning system. The results also prove that new participating areas can train and quickly obtain optimized local models by using transferred common global model.

Spatial High Utility Itemset Mining (SHUIM) is an important knowledge discovery technique with many real-world applications. It involves discovering all itemsets that satisfy the user-specified m inimum u tility (minUtil) i n a q uantitative spatiotemporal database. The popular adoption and the successful industrial application of this technique have been hindered by the following two limitations: (i) Since the rationale of SHUIM is to find all itemsets that satisfy the minUtil constraint, it often produces too many patterns, most of which may be redundant or uninteresting to the user. (ii) Specifying a right minUtil value is an open research problem in SHUIM. This paper tackles these two problems by proposing a novel model of top-k spatial high utility itemsets that may exist in a database. A new constraint, called dynamic minimum utility (dMinUtil), was explored to reduce the search space effectively. This constraint is based on a greedy search, where we raise its value through five thresholdraising strategies. An efficient single scan algorithm that employs depth-first search to find all top-k spatial high utility itemsets was also presented in this paper. Experimental results demonstrate that our algorithm is memory and runtime efficient. We will also demonstrate the usefulness of our algorithm with two real-world case studies.

Nowadays, resolving chaotic traffic situations, which usually link to traffic congestion, is an essential need. It poses many risks to commuters like traffic accidents, especially during bad weather situations. Besides, owing to the exponential growth of IoT technologies, it is easier than ever to collect a huge amount of urban sensing data. Therefore, building a system to anticipate congestion from the collected data could enhance public safety and give traffic police forces enough time to handle traffic flows in potentially dangerous areas. Moreover, if we can discover patterns in which predicted congestion usually happens, we can build reaction plans with various alert codes. They create dynamic risk maps that can provide useful knowledge to both authorities and travelers to make rescue and travel plans effectively. This paper proposes a novel framework to address these problems. The proposed framework employs the Enhanced-Fusion-3DCNN deep learning model to predict future long-term traffic congestion on a particular mesh-code at a particular time instance. The predicted traffic congestion data is later transformed into a temporal database and feed to the maximal periodic-frequent pattern algorithm to identify the sets of mesh-code in which regular congestion may happen in the predicted data. Experimental results on real-world traffic congestion data demonstrate that the proposed framework is efficient.

Finding Spatial High Utility Itemsets (SHUIs) in a spatiotemporal database is a challenging problem of great importance in many real-world applications. Most previous works focused on the sequential discovery of SHUIs in a database running on a single machine. Consequently, these works are not suitable for big data (or cloud-based) applications as they suffer from the scalability and fault tolerant problems. This paper proposes several novel pruning techniques to reduce the search space and present a more flexible distributed algorithm to find all desired itemsets from the database using Spark in-memory computing architecture. Our algorithm inherits several advantages of Spark, including low communication cost, fault tolerance, and high scalability. Experimental results demonstrate that the proposed algorithm has good scalability and performance on very large databases. Finally, we present a real-world navigation application in which SHUIs generated from the traffic congestion data have been employed to recommend alternative routes to the users.

Periodic-frequent pattern mining (PFPM) is an important data mining model having many real-world applications. However, this model's prosperous industrial use has been hindered by the problem of combinatorial explosion of patterns, which is the generation of too many redundant patterns, most of which may be useless to the user. We propose a novel model of closed periodic-frequent patterns that may exist in a temporal database to address this problem. Closed periodic-frequent patterns represent a concise lossless subset that uniquely preserves the complete information of all periodic-frequent patterns in a database. An efficient depth-first search algorithm, called Closed Periodic-Frequent Pattern Miner (CPFP-Miner), has been introduced to find all the database's desired patterns. Experimental results demonstrate that CPFP-Miner is not only memory, runtime, and energy-efficient, but also highly scalable. The usefulness of our model has also been shown with a case study on traffic congestion analytics.

Large events with many attendees cause congestion in the traffic network around the venue. To avoid accidents or delays due to this kind of unexpected congestion, it is important to predict the level of congestion in advance of the event. This study aimed to forecast congestion triggered by large events. However, historical congestion information alone is insufficient to forecast congestion at large venues when non-recurrent events are held there. To address this problem, we utilize microblog posts that refer to future events as an indicator of event attendance. We propose a regression model that is trained with microblog posts and historical congestion information to accurately forecast congestion at large venues. Experiments on next 24-hour congestion forecasting using real-world traffic and Twitter data demonstrate that our model reduces the prediction errors over those of the baseline models (autoregressive and long short term memory) by 20% - 50%.

In recent years, the importance of air pollution issues has been increasingly discussed by the public and the government. Air pollution prediction has become a crucial reference to rely on when the government needs to formulate environmental policies. Considering the sparsity of atmospheric monitoring stations in the spatial distribution, in this study, we consider the atmospheric data as a spatio-temporal structure graph for better utilization of spatio-temporal information. Accordingly, we propose using multi-attention spatio-temporal graph networks (MASTGN) to exploit the graph structure atmospheric data for air pollution prediction tasks. The MASTGN model allows better mining the high-level spatial, temporal, and physical features corresponding to the atmospheric data through the multi-attention mechanism of the spatial, temporal, and channel attention. We conduct experiments on two datasets gathered in Japan and China to predict the concentration of PM2.5, O3, and PM10. The results indicate that the proposed MASTGN model outperforms the considered baseline approaches on prediction accuracy.

Periodic-frequent pattern mining (PFPM) is an important data mining model having many real-world applications. However, the successful industrial application of this model has been hindered by the problem of combinatorial explosion of patterns, that is the generation of too many redundant patterns, most of which may be useless to the user. To address this problem, this paper proposes a novel model of maximal periodic- frequent pattern that may exist in a temporal database. A new pattern-growth algorithm, called Maximum Periodic-Frequent Pattern-growth (maxPFP-growth), has also been introduced to efficiently find all desired patterns in the data. Experimental results demonstrate that maxPFP-growth is not only memory and runtime efficient, but also highly scalable as well. The usefulness of our model has also been demonstrated with a case study on traffic congestion analytics.

Frequent pattern mining is an important model in data mining. It involves finding all patterns in a transactional database that satisfy the user-specified minimum support (minSup) constraint. The minSup controls the minimum number of transactions that a pattern must cover in a transactional database. Since only minSup is used to evaluate a pattern's interestingness, the frequent pattern model implicitly assumes that spatial information of the items will not impact the interestingness of a pattern in the database. This assumption limits the applicability of the frequent pattern model in many real-world applications. It is because patterns whose items are close to each other are typically more attractive to the user than the patterns whose items are far from each other in a coordinate system. With this motivation, this paper proposes a novel model of frequent spatial pattern that may exist in a spatiotemporal database. An efficient pattern-growth algorithm, called Frequent Spatial Pattern-growth (FSP-growth), has also been presented to mine all desired patterns in a database. Experimental results demonstrate that our algorithm is efficient. The usefulness of the proposed patterns has also been shown with a real-world application.

Lifelogging is known as the new trend of writing diary digitally where both the surrounding environment and personal physiological data and cognition are collected at the same time under the first perspective. Exploring and exploiting these lifelog (i.e., data created by lifelogging) can provide useful insights for human beings, including healthcare, work, entertainment, and family, to name a few. Unfortunately, having a valuable tool working on lifelog to discover these insights is still a tough challenge. To meet this requirement, we introduce an interactive multimodal retrieval system that aims to provide people with two functions, memory assistant and life organized support, with a friendly and easy-to-use web UI. The output of the former function is a video with footages expressing all instances of events people want to recall. The latter function generates a statistical report of each event so that people can have more information to balance their lifestyle. The system relies on two major algorithms that try to match keywords/phrases to images and to run a cluster-based query using a watershed-based approach.

Having an effective and efficient model to predict traffic congestion using multi-sources data has challenged researchers for decades, especially when the number of data sources and data volume increase dramatically. In this research, we propose a new CNN-based approach that can absorb and wrap multi-sources data into a 2D/3D raster-image to predict traffic congestion. Thanks to the raster-image-based wrapping technique, the spatial-temporal correlation is conserved entirely. The proposed approach can (1) accurately predict traffic congestion over multi-scale areas at different time horizons, (2) additionally consider external factors that could affect traffic flows and might cause traffic congestion afterwards by using an immediate fusion strategy. Traffic congestion, precipitation data, posts on a social networking platform collected in Kobe city, Japan during the summer and fall of the two consecutive years are used to evaluate the proposed approach. The comparison to other methods working on the same topic is also conducted to confirm the advantage of the proposed approach. Last but not least, some insights regarding the consequences of spatial and temporal dimensions as well as external factors to different time windows are also discussed.

Finding partial periodic patterns in very large databases is a challenging problem of great importance in many real-world applications. Most previous work focused on finding these patterns in temporal (or transactional) databases and did not recognize the spatial characteristics of items. In this paper, we propose a more flexible model of partial periodic spatial pattern that may be present in spatiotemporal database. Three constraints, maximum inter-arrival time(maxIAT), minimum period-support(minPS) and maximum distance(maxDist), have been employed to determine the interestingness of a pattern in a spatiotemporal database. The maxIAT controls the maximum duration in which a pattern must reappear to consider its occurrence as periodic within the data. The minPS controls the minimum number of periodic occurrences of a pattern within the data. The maxDist controls the maximum distance between the items in a pattern. All patterns satisfying these three constraints are returned. An efficient algorithm, called SpatioTemporal-Equivalence CLAss Transformation (ST-ECLAT), has also been described to discover all partial periodic spatial patterns in a spatiotemporal database. This algorithm employs a novel smart depth-first search technique to discover desired patterns effectively. Experimental results demonstrate that the proposed algorithm is efficient. We also present a case study in which we apply our model to find useful information in the air pollution database.

Predictive analytics are concerned as a type of complex event processing where a complex event can be predicted by utilizing insights extracted from a set of related events. This paper introduces a new complex event analysis for traffic risk prediction using 3D-CNN and a set of related events detected from multi-sources urban sensing data (e.g., congestion, traffic accident, precipitation). The contribution of this paper involves (1) the spatio-temporal information of multi-sources urban sensing data is reserved and wrapped into 3D raster images towards being able to leverage recent developments of 3D-CNN to conduct predictive analytics, (2) the imbalanced data problem which could severely affect the performance of deep learning models is tackled by straightening curved geographic chains, (3) traffic risks can be predicted well in both short-term and medium-term time horizons, and (4) The influence of related events detected from extra factors on a complex event can be explained explicitly. The proposed method is evaluated on the real dataset collected in Kobe, Japan during 2014 and 2015. The comparisons to baseline methods such as historical average and 2D-CNN show the advantage of the proposed method as well.

Air Quality Index (AQI) is an indicator of the rank of air pollution that is very vital for the environmental impacts to the public health. In this paper, we propose an association model between visual feature and AQI rank of lifelog data. Visual data (i.e., environmental pictures) and numerical data (i.e., environmental AQI measurements) of lifelog are utilized for the data training stage. The features of the visual data are extracted using a CNN-based method, where the latter are calculated using the standard AQI ranking. The extracted visual features and ranked AQI are combined as the input data for a deep neural network MLP (Multi-layer Perception) to study the association relationship between visual feature and AQI rank. The experimental results show that the proposed method can provide accurate predictions of good or unhealthy AQI ranks from lifelog visual data.

Personal air quality is an important indicator when assessing the impact of air pollution on personal health. Because personal air quality data are collected manually, it difficult to collect such data in large quantities. The main challenge facing personal air quality predictions is building an effective prediction model with a small amount of training data. Moreover, public atmospheric monitoring stations in urban areas have collected large quantities of air quality data. Therefore, we focus on using atmospheric monitoring data with a transfer-learning method to predict personal air quality. In this paper, we design a transferlearning framework based on an encoder-decoder structure. This transfer-learning framework uses the Wasserstein distance to match the heterogeneous distribution of the source domain (the data from the atmospheric monitoring stations) and the target domain (the personal air quality); we refer to this as decoder transfer learning (DTL). We use data from public atmospheric monitoring stations, collected by the Atmospheric Environmental Regional Observation System (AEROS) of Japan, as the source domain dataset and private datasets collected in Fujisawa, Japan, and Tokyo, Japan, as the target domain datasets to evaluate this approach. The experimental results demonstrate that compared with the inverse distance weighting (IDW), IDW with linear regression, and typical transfer-learning models, the proposed DTL framework demonstrates a significant improvement in prediction performance.

In real-world databases, high utility itemset (HUI) is an important class of regularities. Most previous studies have focused on mining HUIs in transactional databases and did not consider the spatiotemporal characteristics of items. In this study, a more flexible model of spatial HUIs (SHUIs) that exist in spatiotemporal databases is proposed. In a spatiotemporal database (STD), an itemset is said to be an SHUI if its utility is not less than a user-specified minimum utility and the distance between any two of its items is not more than a user-specified maximum distance. Identifying SHUIs is very challenging because the generated itemsets do not satisfy the anti-monotonic property. In this study, we present two novel pruning techniques for reducing computational costs. Moreover, a fast single scan algorithm is presented for effectively evaluating all SHUIs in a STD. Furthermore, two case studies are presented, in which the proposed model is used to identify useful information in traffic congestion data and air pollution data.

Torrential rains, the complicated network of roads, and the high density of vehicles contribute partly the number of traffic accidents. In order to understand the association between these factors towards building a risk map that can alert drivers of dangerous zones, the visual patterns reasoning system is proposed. By converting sensing data collected from different factors to raster images, the associations can be treated as visual patterns that can conserve their spatiotemporal information. Deep convolutional neural networks (Deep-CNN) are utilized to build a model based on these raster images towards detecting accidents based on the association between factors. Image clustering is applied to learn a representation of each type of associations. Thus, the visual pattern of high-probability traffic accidents can be reasoned in the natural language format. Both 2D and 3D raster images are investigated to examine the spatial and spatiotemporal associations between factors with Deep-CNN models. We use both transfer learning and fine-tuning approach to build our model due to the small size of the positive samples dataset. The evaluation shows the initial but promising results of our method. We also discuss the potential applications and various research directions can be investigated using our proposed method.

Spatiotemporal co-occurrence patterns represent subsets of event features that are often located together in space and time. However, such spatiotemporal co-occurrence patterns can fail to capture disaster-related events that often occur unexpectedly and in limited regions and limited time intervals. In addition, previous studies for discovering co-occurrence patterns do not consider using patterns for prediction problem. In this paper, we define the problem of discovering co-occurrence patterns, each of which is annotated with a valid spatial and temporal subregion. We also define a new interest measure of cooccurrence patterns for prediction problem and then based on this measure, we propose a method for discovering such co-occurrence patterns in form of association rules by incorporating repeatedly spatiotemporal clusterings to remove spatiotemporal bias. Our algorithm is suitable to large datasets. We evaluate our method for real-world datasets by discovering and then predicting traffic disaster events co-occurring with torrential rain events in Kansai area, Japan. By only using 80% most interesting discovered patterns, our experimental result shows 24% improvement of prediction performance on F-measure against a baseline.

ETL (Extraction-Transform-Load) tools, traditionally developed to operate offline on historical data for feeding Datawarehouses, need to be enhanced to deal with big and fresh data and be executed at network level during data streams acquisition.In this paper, we present StreamLoader, a Web application for the specification of conceptual ETL dataflows on heterogeneous sensor data that leverages the peculiarities of network configuration, data stream management, and specification and deployment of ETL operations in a programmable network.It can be used for feeding traditional/ real-time data-warehouses or visual analytic tools.

We can collect, store, and analyze a huge amount of information about human mobility and social interaction activities due to the emergence of information and communication technologies and location-enabled mobile devices under cyber physical system frameworks. The high spatial resolution of population data on a multi-temporal scale is required by transport planners, human geographers, social scientists, and emergency management teams. In this study, we build a space-time multiple regression model to estimate grid-based (500 m × 500 m) spatial resolution at multi-temporal scale (30-min intervals) population data based on the space-time relationship among geospatially enabled person trip (PT) survey data and incorporate both mobile call (MC) and geotagged Twitter (GT) data. Since using geospatially enabled PT survey data as dependent variables enables us to acquire actual population amounts, which strongly depend on MCs and social interaction activities. Although many grids have a strong correlation between PT and MC/GT, some show fewer correlation results, especially where the grids have factories, schools, and workshops in which fewer MCs are found but a large population is presented. Although GT data are sparser than MCs, people from amusement and tourist areas can be detected by GT data. The space-time multiple regression model can also estimate the different amounts of populations based on human travel behavior that changes over space and time. According to accuracy assessments, the night-time estimated results, especially between 00:00 and 06:30, strongly correlate with national census data except in places where the grids have railway and subway stations.

In this paper, we introduce a new method that explores spatio-temporal-theme correlations between physical and social streaming data for event detection and pattern interpretation from heterogeneous sensors. Particularly, we employ a basic two-phase framework in pattern recognition (i.e. feature extraction and detection) with the novel improvement that concerns the use of semantic information acquired from social sensors to automatically label the low-level features extracted from physical sensors. Moreover, by symbolizing the trend component of time-series data, the proposed method has an ability to interpret event's patterns to help users get insights of how events happen. Differentiating from conventional supervised learning methods whose training data are labeled manually and in an off-line mode, the proposed method can collect labels for training data automatically and in an on-line mode. Moreover, after running for a certain time, a training stage can run parallel with the detecting stage when an event model is totally built. After that, the training stage continues learning to increase the accuracy of the event model by nonstop collecting new samples with labels from streaming data. The problem of environmental factors and particularly air pollution impacts on asthma exacerbation is considered for evaluating the proposed method. The experimental results show that the proposed method can probably detect the prevalence of asthma risks in a specific spatio-temporal context as well as help users understand how a change in the surrounding environment (e.g. weather condition and air pollution) can influence their health (e.g. asthma attack) by interpreting detected event's patterns.

We consider constrained label placement problem considering touch interface such as smartphone or tablet. For scientific dataset search, the search results are shown on the global map based on spatial information. There spatial region are often unevenly distributed and most of them are overlapped each other. To select non-overlapped regions from overlapped regions can be considered as a combinational optimization problem which is known as NP-hard. Also, the applications for touch interface should be designed considering finger pad size. In this paper, we propose rectangular label placement method in configuration space for scientific dataset search with touch interface. The idea of configuration is often used for path planning for robots to avoid obstacles. The proposed method apply the idea of configuration space to find the un-overlapped regions to show the selectable regions for touch interface. Furthermore, the relevance between search query and each datasets are considered, so that the user can select more relevant datasets from original search results. This method can be applied not only for scientific dataset search system but also any other applications which shows their results in global map. The experimental evaluations show that the proposed method achieves superior performance to the compared methods.

Cyber-Physical Social Systems (CPSS) are transforming how we live and interact with the physical world by semantically linking devices, data, and people. However, while tremendous progress have been made in modelling various CPSS physical components such as sensors and actuators, the involvement of humans in CPSS poses additional challenges for conceptual modelling experts and systems designers. Part of the problem is due to the fact that, even though humans sense, actuate, and process information like other CPSS component, they do these things differently. Furthermore, it is difficult to exactly know, in advance, how the human entity will interact with a CPSS. In this paper, we present a framework for modelling human's involvement in CPSS. Use cases, lessons learnt, and challenges we face in using the framework to develop a CPSS for research and development in the environmental sciences are also discussed.

In this paper, we introduce a Deep Recurrent Neural Network (DRNN) that is trained using a novel autoencoder pre-training method especially designed for the task of time series prediction. Our main objective is to perform predictions of environmental monitoring data using open sensors with improved accuracy over the currently employed methods. The numerical experiments show that our proposed pre-training method is superior that a canonical and a state-of-the-art auto-encoder training method when applied to time series prediction. On the specific case of fine particulate matter (PM2.5) forecasting in Japan, the experiments confirm that when compared against the PM2.5prediction system VENUS employed by the Japanese Government, our technique improves the accuracy of PM2.5concentration level predictions that are being reported in Japan.

We are living in the Internet of Things (IoT) era where all the (smart) objects around us are connected and communicated with each other to serve our life better without the need of explicit instruction. Soon we have to cope with trillions of heterogeneous data streams coming from IoT. Since data is not information, methods for discovering useful and correlative information from data and utilising them for the better life, in real-time mode, are the utmost requirements.

In this paper, a novel cross-database search system (Cross-DB) is proposed. The aim of Cross-DB is to facilitate the search of interdisciplinary-correlated datasets from large-scale, multi-domain and heterogeneous data repositories. With conventional systems or portals for searching scientific datasets, the scientists must know the relation between the datasets in advance or must find their relations manually. In Cross-DB, the datasets search process is based on discovering an optimal combination of their multiple and latent associations such as spatio-temporal, ontological, and citational correlations based on evolutionary computing. The basic concepts of Cross-DB are introduced as well as its main components. Comparisons with an existing search engine based on a massive datasets repository demonstrate the feasibility and the correctness of the proposed framework. We show that offering to the user a full set composed of correlated datasets is a useful alternative to the classical ranking methods. Experimental result shows that our system can overachieve conventional portal search in terms of relevance and novelty.

On behalf of the rapidly and widely disseminated smartphone technology into the public, lots of social network sites and location-based social applications are accumulating a huge volume of massive crowd's daily experiences and thoughts in an unprecedented scale. We can regard them as novel data sources for accomplishing various social analytics, which have usually required lots of efforts to collect crowds' opinion and behavioral data. Thus, we can take advantages of abundant social datasets by integrating them appropriately. However, when we integrate disparate sources to derive a comprehensive view for a survey, it is necessary to know intrinsic exclusive values of each data source compared to others in an intuitive and succinct way. In fact, lots of efforts and time are wasted to overview various datasets consequently to confidently choose a dataset to be integrated in a final result. In this paper, we propose a complementarity index, which can estimate the exclusive usefulness of data sources in terms of spatial and topical coverage when selecting data sources for social analytics purposes. We conducted an experiment about complementarity measurement with two real social datasets from Twitter and VoiceTra; the latter is a speech-to-speech translation app, with which we can additionally obtain crowds' verbal translation logs. With the proposed complementarity index, we can measure the capability of a dataset comparing to others before integrating datasets, thus enabling analysts to examine much more datasets from as many related data sources as possible by focusing on exclusive coverage and relative strength of relevant topics.

Network-based mobile services, such as speech-to-speech translation and voice search, enable the construction of large-scale log database including speech. We have developed a smartphone application called VoiceTra for speech-to-speech translation and have collected 10,000,000 utterances so far. This huge corpus is unique in size and spatio-temporal information; it contains information on anonymized user locations. This spatiotemporal corpus can be used for improving the accuracy of its speech recognition and machine translation, and it will open the door for the study of the location dependency of vocabulary and new applications for location-based services. This paper first analyzes the corpus and then presents a novel method for classifying utterances using linguistic and non-linguistic information. L2-regularized Logistic Regression is used for utterance classification. Our experiments performed on the VoiceTra log corpus revealed that our proposed method outperformed baseline methods in terms of F measure.

We propose a structural approach to service composition using a relational model. We focus on structural aspects of service composition and apply the relational model. Services in our model are defined and organized by relations, and service manipulations such as service invoke and service composition, are achieved by relational operations to the corresponding relations. Since conventional relational algebra is insufficient for service management, we extend relational algebra and introduce a new θ-operator to deal with services.

With being coupled with geographic location, microblogging messages have become more important information resources to share observations and opinions about the real world via social media. As a result, we are getting much interested in comprehending situations related to natural and/or social events using those messages. In the recent, certain works have focused on showing trending topics that can represent snapshots of certain situations with spatial and temporal contexts on the basis of geo-tagged microblogging messages. They, however, have difficulty in tracking and comparing topic changes and movements in a spatiotemporal domain because of the separated geo-spatial and temporal components. This demonstration introduces mTrend, which constructs and visualizes spatiotemporal trends of topics, named as "topic movements", on the basis of the geo-tagged Tweets. In particular, its interactive visual mining tool allows users to intuitively understand the differences between topic movements over space and time. In the demonstration, we present the comparisons of topic movements using a few keywords related to the Great East Japan Earthquake.

Geosocial networking services allow users to create,use, and share information and to communicate with other people regarding geographical locations and time. Geosocial networking services generate large amounts of spatiotemporal contents that mainly include information about personal interests, activities, or real-life experiences. This paper discusses a new geovisualization tool named StickViz that helps retrieve k-neighbors by considering the spatial, temporal, and thematic interests of users over geosocial networking services.In particular, we define phenomena of interest for combining user interests with respect to location, time, and topic and propose a phenomenon-based k-neighbor query to find people having similar interests in spatial, temporal, and thematical dimensions. Moreover, we use a three-dimensional (two spatial dimensions and one temporal dimension) space so that the queries and spatiotemporal contents can be represented in the same space. Finally, we present a prototype of StickViz to navigate spatiotemporal contents and search k-neighbors on the basis of the predefined phenomena of interest through three-dimensional visual interfaces.

With the advance of ubiquitous computing and mobile environments, we have begun to continuously monitor changes in real-world condition and environment through wireless sensor networks. Opportunities also exist for people to create information related to the world around them by using mobile phones equipped with sensing devices, and share that information online with others. In this paper, we propose a novel approach for the interconnection of earth observation data and spatiotemporal web contents on the basis of spatiotemporal and thematic relationships. In particular, we use the concept of moving phenomena of interests to link between measurement sensing data and people-centric contents on the basis of spatiotemporal proximity and thematic relevance. This paper also shows a simple application that automatically generates semantic tags with respect to natural geographic phenomena, such as typhoons, climate changes, and air pollution, on the basis of our interconnection approach. We are able to easily understand qualitative meanings with respect to a certain phenomenon expressed by quantitative numeric conditions.

Various knowledge resources are spread to a world-wide scope. Unfortunately, most of them are community-based and never thought to be used among different communities. That makes it difficult to gain “connection merits” in a web-scale information space. This paper presents a three-layered system architecture for computing dynamic associations of events to related knowledge resources. The important feature of our system is to realize dynamic interconnection among heterogeneous knowledge resources by event-driven and event-centric computing with resolvers for uncertainties existing among those resources. This system navigates various associated data including heterogeneous data-types and fields depending on user's purpose and standpoint. It also leads to effective use for the sensor data because the sensor data can be interconnected with those knowledge resources. This paper also represents application to the space weather sensor data.

The internet and web applications have changed business and human life. Nowadays everybody is used to obtain data through the internet. Most applications are still Web 1.0 applications. Web 2.0 community collaboration and annotated data on the basis of Web 3.0 technologies supports new businesses and applications. The quality dimension of the web is however one of the main challenges. Knowledge systems target at high-quality data on safe grounds, with a good reference to established science and technology and with data adaptation to user's needs and demands. Knowledge system can be build based on existing and novel technologies. This paper discusses the challenges, two solutions and the fundamentals of knowledge system environments.

This paper presents an interconnection method for heterogeneous knowledge bases depending on user's interests as a context. Various knowledge bases have been created in each field by using collaborative working environments such as Wiki. One of the important issues is how to interconnect these knowledge bases and represent the relationships between various concepts in heterogeneous fields. An event affects various aspects of an area, field, or community. In order to understand an arbitrary event or concept, it is necessary to find various relationships over heterogeneous fields. Generally, the relationships over heterogeneous fields strongly depend on contexts and situations. It is important to realize the dynamic interconnection of knowledge bases depending on contexts and situations. Therefore, we design an interrelation management function (IMF) that defines the operator for interconnection data. In this paper, we propose a framework for a context-dependent dynamic interconnection method by using the interrelation management function.

With the advance of the Web technologies, people can more easily access the geographic information and generate various types of user contents including geo-information on the Web. Consequently, geo-web and geo-communities have been infrastructures to share and connect information on the Web for many years, and people start to describe a specific phenomenon on places by own representation methods such as text, images, videos, etc. In this demonstration, we introduce a new type of location-based services, called Sticker, which can search and aggregate the relevant user generated contents/media to moving phenomena such as hurricanes, flooding, and global warming. In particular, the Sticker navigates user-generated contents with three dimensional view of space-time(2D+1D) and allows users to retrieve related information with the moving phenomena in a spatiotemporal domain as well as interesting keywords.

Utilization of global information resources as a part of risk management is insufficient. The authorities are maintaining information systems mainly for their own purposes, without access to high quality public information sources in Internet and without interoperability between systems of different authorities. Beneficial use of all available information resources would provide an opportunity to create knowledge based on different pieces of information. However, powerful distributed knowledge management, mining of the information items, analysing the quality of them, is needed to create new information to be utilized. The distributed operations needs support of complex network architectures, models supporting mutual understanding over the cultures and language borders, and ability to recognize the context and adapt the results to the new context. This paper opens discussion from different viewpoints to the topic of global risk management. Architectural solutions supporting interoperability, quality of data in wide networks, ubiquity and mobility as well as time dimension of the information space are covered.

This paper described our development dialog system on Kyoto tourist information assistance. Dialog part of our system helped user to make an appropriate query. Information analysis part would be assisted for user to select the retrieved information. Nowadays we can get most information through the Internet. However, we have a trouble to pick up expected information from the huge results with conventional search engines. Especially in mobile terminal, we are confronted with great difficulties for two factors. One is that most of users cannot make an appropriate query because their request is vague with theirselves. The other is that the retrieved information has huge variation and mobile terminal has small area for displaying them. Therefore, we aim to develop technologies for the users to input their requests by familiar way and clarify what they want to know with displaying the retrieved information with suitable method.

This paper proposes a general framework of for a system with a semantic browsing and visualization interface called Knowledge Communication, Collaboration and Creation Browser (KC3 Browser) integrates multimedia contests and web services on the grid networks, and makes a semantic mash-up called knowledge workspace (k-workspace) with various visual gadgets according to user's contexts (e.g. their interests, purpose and computational environments).

We developed a mechanism for photographing people and annotating their behavior along with nearby elements using multiple embedded cameras. In addition, we developed a method of dynamically integrating and presenting the recorded content. As conventional cameras are used to photograph objects selected by users, taking pictures that include users while holding the camera is difficult. Security camera systems take pictures that include people and nearby elements, but such systems cannot show the intentions of the people being photographed. After detecting the intentions and behaviors of subjects with radio frequency identification (RFID) tags, our system selects the best camera from cameras located in an area, and then the camera photographs the subject and the surrounding area. In addition, these photos can be annotated with information about the context and movement history of the subject. We created a prototype of our system and determined its effectiveness experimentally.

We developed a mechanism for photographing people and annotating their behavior along with nearby elements using multiple embedded cameras. In addition, we developed a method of dynamically integrating and presenting the recorded content. As a conventional camera is used to photograph objects selected by a photographer, taking pictures that include him while holding the camera is difficult. Security camera systems take pictures that include people and nearby elements, but such systems cannot take the user aspect of intentions and interest of the people. After detecting the intentions and behaviors of subjects using radio frequency identification (RFID) tags, our system selects the best camera from cameras located in an area, and then the camera photographs the subject and the surrounding area. In addition, these photos can be annotated with meta datainformation about the context and interest of the subject.

The Web is a vast playground for the propagation of information by individuals. The capability of Web users to leverage the value of Web content, in terms of factors such as usefulness, reputation, or reliability, has emerged as a requirement. The most significant characteristics of the Web are hyperlinks, which enables an author to refer to Web content published by other people. As a result, Web content can refer to other content on the Web in various contexts. The references provide important clues to understanding the roles or reputations of various Web pages according to the viewpoints of third parties. In this paper, we propose an approach to mining referential contexts on the Web.