The 3D Variable Scene Graph (3DVSG) is a newly emerging representation for modeling dynamic environments, extending scene graphs by introducing a node-level property called variability, which quantifies the likelihood of semantic change over time. In this work, we explore the integration of pre-trained language models (PLMs) into variability estimation. This is of significant practical importance because variability estimation suffers from data scarcity and severe class imbalance. PLMs provide a rich general semantic knowledge that can enhance representation learning in such settings. We systematically evaluate PLM embeddings across different graph neural networks (GNNs). We introduce a template-based text structuring (TTS) to understand the effect of input formatting. Our experiments show that PLM embeddings significantly improve variability estimation performance, with effectiveness influenced by both embedding and GNN choices. Also, we demonstrate that text structure can significantly affect embedding quality. Lastly, we demonstrate that PLM embeddings yield reliable gains in variability estimation and downstream active change detection.

Abstract Commonsense explanation generation refers to reasoning and explaining why a commonsense statement contradicts commonsense knowledge, such as why the statement “My dad grew volleyballs in his garden” is nonsensical. While such reasoning is trivial for humans, it remains a challenge for AI systems. Despite their notable performance in tasks like text generation and reasoning, large language models (LLMs) often fall short of consistently generating coherent and accurate commonsense explanations. To bridge this gap, we propose a novel Two-stage Identification and Prompting (TIP) framework for enhancing LLMs’ ability to handle the task of commonsense explanation generation. Specifically, in the first stage, TIP identifies the nonsensical concept in the given statement, pinpointing the specific element that contradicts commonsense knowledge. In the second stage, TIP generates implicit knowledge based on the identified nonsensical concept and then leverages this implicit knowledge to guide the adopted LLMs in generating explanations. In order to demonstrate the effectiveness of the proposed TIP framework for commonsense explanation generation, we conducted extensive experiments based on the ComVE dataset and a newly constructed CSE dataset, where a variety of LLMs are evaluated. The experimental results show that TIP consistently outperforms all baseline methods across multiple metrics, demonstrating its effectiveness in improving LLMs’ commonsense reasoning and explanation generation capabilities.

This paper presents a novel approach to optimizing semantic joinability in heterogeneous data, leveraging embedding techniques and deep learning in the context of big data environments. We propose two distinct embedding strategies: a text-based approach using DistilBERT and an image-based approach utilizing ResNet50, both fine-tuned using a triplet data structure and circle loss functions to enhance joinability predictions from data lakes. By transforming tabular data into semantic embeddings, our method facilitates more effective integration of large, diverse datasets. Experiments conducted on large-scale datasets show that the fine-tuned models significantly outperform baseline approaches in accuracy and robustness. Furthermore, incorporating computer vision techniques via the image-based method demonstrates the versatility of embedding strategies across different data types. The results suggest that pre-trained models, when fine-tuned for specific joinability tasks, can provide a scalable and efficient solution for extensive data integration. Future work will explore expanding this approach to additional data modalities and optimizing model performance in large-scale applications.

Recent advances in location-aware devices have resulted in an exponential increase in the trajectory data streams. A number of applications require real-time processing and analysis of massive moving objects' trajectories. For instance, route guidance in emergency evacuation, patients tracking, etc. Existing scalable trajectory management systems lack support for real-time processing, while the real-time systems do not natively support spatial trajectory processing. This work presents TStream, a real-time and scalable trajectory stream processing and analysis framework. TStream utilizes grid index to support efficient processing of continuous range, kNN and join queries.

Forecasting incident occurrences (e.g. crime, EMS, traffic accident) is a crucial task for emergency service providers and transportation agencies in performing response time optimization and dynamic fleet management. However, such events are by nature rare and sparse, which causes the label imbalance problem and inferior performance of models relying on data sufficiency. The existing studies circumvent, instead of truly solving, this issue by defining the incident prediction problem in a coarse-grained temporal (e.g. daily) setting, which leaves the proposed models unrobust to fine-grained dynamics and trivial for the real-world decision making. In this paper, we tackle the temporally fine-grained incident prediction problem in a sparse setting by explicitly exploiting the behind-the-scene chainlike triggering mechanism. Moreover, this chain effect roots in multiple domains (i.e. spatial, categorical), which further entangles with the temporal dimension and happens to be time-variant. To be specific, we propose a novel deep learning framework, namely Spatio-Temporal-Categorical Graph Neural Networks (STC-GNN), to handle the multidimensional and dynamic chain effect for performing fine-grained multi-incident co-prediction. Extensive experiments on three real-world city-level incident datasets verify the insightfulness of our perspective and effectiveness of the proposed model.

Forecasting regional ambulance demand plays a fundamental part in dynamic fleet allocation and redeployment. This topic has been gaining increasing significance, as virtually every country is experiencing an aging population, with generally higher level of vulnerability and demand for the emergency medical service (EMS). Although exploring the spatial and temporal correlations in EMS historical records, the existing methods principally consider the former time-invariant, which does not necessarily hold in reality. Moreover, this assumption ignores the fact that the behind-the-scenes dynamics are people, whose demographic profiles and activity patterns could be determinants of regional EMS demands. In this paper, we are therefore motivated to mine the collective daily routines in human mobility, to further represent the evolving spatial correlations. Particularly, we model profiled mobility groups as multiple random walkers and propose a novel bicomponent neural network, including a heterogeneous multi-graph convolution layer and spatio-temporal interlacing attention module, to perform the prediction task. Experimental results on the real-world data verify the effectiveness of introducing dynamic human mobility and the advantage of our approach over the state-of-the-art models.

Abstract Analyzing social media has become a common way for capturing and understanding people's opinions, sentiments, interests, and reactions to ongoing events. Social media has thus become a rich and real‐time source for various kinds of public opinion and sentiment studies. According to psychology and neuroscience, human emotions are known to be strongly dependent on sensory perceptions. Although sensation is the most fundamental antecedent of human emotions, prior works have not looked into their relation to emotions based on social media texts. In this paper, we report the results of our study on sensation effects that underlie human emotions as revealed in social media. We focus on the key five types of sensations: sight, hearing, touch, smell, and taste. We first establish a correlation between emotion and sensation in terms of linguistic expressions. Then, in the second part of the paper, we define novel features useful for extracting sensation information from social media. Finally, we design a method to classify texts into ones associated with different types of sensations. The sensation dataset resulting from this research is opened to the public to foster further studies.

Apache Flink is an open-source system for scalable processing of batch and streaming data. Flink does not natively support efficient processing of spatial data streams, which is a requirement of many applications dealing with spatial data. Besides Flink, other scalable spatial data processing platforms including GeoSpark, Spatial Hadoop, etc. do not support streaming workloads and can only handle static/batch workloads. To fill this gap, we present GeoFlink, which extends Apache Flink to support spatial data types, indexes and continuous queries over spatial data streams. To enable efficient processing of spatial continuous queries and for the effective data distribution across Flink cluster nodes, a gird-based index is introduced. GeoFlink currently supports spatial range, spatial kNN and spatial join queries on point data type. An experimental study on real spatial data streams shows that GeoFlink achieves significantly higher query throughput than ordinary Flink processing.

With the development of Light Detection and Ranging (LiDAR) technology, point cloud data is a valuable resource to build three-dimensional (3D) models of digital twins. The geospatial 3D model is the principal element to abstract a geographic feature with geometric and semantic properties. The 3D model data provides more efficiency to handle, retrieve, exchange, and visualize geographic features compared to point clouds. However, the construction of 3D models, especially indoor space where various objects exist, usually necessitates expensive time and manual labor resources to organize and extract the geometry information by authoring tools.

With the increase in the use of 3D scanner to sample the earth surface, there is a surge in the availability of 3D spatial data. 3D spatial data contains a wealth of information and can be of potential use if integrated, processed and analyzed in real-time. The 3D spatial data is generated as continuous data stream, however due to its size, velocity and inherent noise, it is processed offline. Many applications require real-time processing and analysis of spatial stream, for-instance, forest fire management, real-time road traffic analysis, disaster engulfed areas monitoring, etc., however they suffer from slow offline processing of traditional systems. This paper presents and demonstrates a robust and scalable pipeline for the real-time processing and analysis of 3D spatial streams. An experimental evaluation is also presented to prove the effectiveness of the proposed framework.

Event crowd management has been a significant research topic with high social impact. When some big events happen such as an earthquake, typhoon, and national festival, crowd management becomes the first priority for governments (e.g. police) and public service operators (e.g. subway/bus operator) to protect people's safety or maintain the operation of public infrastructures. However, under such event situations, human behavior will become very different from daily routines, which makes prediction of crowd dynamics at big events become highly challenging, especially at a citywide level. Therefore in this study, we aim to extract the deep trend only from the current momentary observations and generate an accurate prediction for the trend in the short future, which is considered to be an effective way to deal with the event situations. Motivated by these, we build an online system called DeepUrbanEvent which can iteratively take citywide crowd dynamics from the current one hour as input and report the prediction results for the next one hour as output. A novel deep learning architecture built with recurrent neural networks is designed to effectively model these highly-complex sequential data in an analogous manner to video prediction tasks. Experimental results demonstrate the superior performance of our proposed methodology to the existing approaches. Lastly, we apply our prototype system to multiple big real-world events and show that it is highly deployable as an online crowd management system.

We propose a general view that demonstrates the relationship between network embedding approaches and matrix factorization. Unlike previous works that present the equivalence for the approaches from a skip-gram model perspective, we provide a more fundamental connection from an optimization (objective function) perspective. We demonstrate that matrix factorization is equivalent to optimizing two objectives: one is for bringing together the embeddings of similar nodes; the other is for separating the embeddings of distant nodes. The matrix to be factorized has a general form: S-β. The elements of $\mathbfS $ indicate pairwise node similarities. They can be based on any user-defined similarity/distance measure or learned from random walks on networks. The shift number β is related to a parameter that balances the two objectives. More importantly, the resulting embeddings are sensitive to β and we can improve the embeddings by tuning β. Experiments show that matrix factorization based on a new proposed similarity measure and β-tuning strategy significantly outperforms existing matrix factorization approaches on a range of benchmark networks.

Three-dimensional (3D) Semantic segmentation of aerial derived point cloud aims at assigning each point to a semantic class such as building, tree, road, and so on. Accurate 3D-segmentation results can be used as an essential information for constructing 3D city models, for assessing the urban expansion and economical condition. However, the fine-grained semantic segmentation is a challenge in high-resolution point cloud due to irregularly distributed points unlike regular pixels of image. In this demonstration, we present a case study to apply PointNet, a novel deep learning network, to outdoor aerial survey derived point clouds by considering intensity (depth) as well as spectral information (RGB). PointNet was basically designed for indoor point cloud data based on the permutation invariance of 3D points. We firstly fuse two surveying datasets of Light Detection and ranging (LiDAR) and aerial images for generating multi-sourced aerial point clouds (RGB-DI). Then, each point of fused data is classified into a semantic class of ordinary building, public facility, apartment, factory, transportation network, park, and water by reworking PointNet. The result of our approach by using deep learning shows about 0.88 accuracy and 0.64 F-measure of semantic segmentation with the RGB-DI data we have fused. It outperforms a Support Vector Machine(SVM) approach based on geometric features of linearity, planarity, scattering, and verticality of a set of 3D points.

With the development of social network services, various phenomena can be shared easily and rapidly through human natural language, including not only natural, but also social-cultural phenomena. Consequently, analyses of social media have appreciated in value for understanding human behaviors to grasp public interests or sentiments, as both the medium and outcome of human experiences. From the state of the art psychology and neuroscience, human behaviors, regarding both physical and linguistic aspects, are mostly dependent on sensory perceptions under the realm of the subconscious. Even though sensation is the most fundamental element to understand human behaviors, the rack of background resources make it hard to study the social sensation comparing with the sentimental or opinion mining. This paper focuses on building sensation knowledges to obtain useful human perceptual experiences in natural language expressions, as a requisite for the social sensation analysis. We try to approach the constructing lexicons as a sensation knowledge from two viewpoints, such as a deep learning and lexicon based methods. Then we classify social media text based on the lexicons with considering a part of speech as well as semantic meanings of each word. Finally, we identify which knowledge has a good performance to distinguish sensation expressions from social media data in terms of accuracy and and F-score.

With the development of position tracking technologies and the increasing usage of mobile devices, the analysis of moving objects, such as pedestrians, vehicles, drones, and hurricanes has become an important topic in various applications including intelligent transportation, disaster management, and urban planning. Many of existing studies have focused on managing and analyzing only time-varying locations of point-based objects. However, real-world moving phenomena are space-time continua occupying volumes, having an area at a time; even more, they contain dynamic attributes depending on time and space, such as the velocity of vehicles or the average of wind speed of hurricanes. In this demonstration, we introduce a comprehensive data format to represent various types of temporal geometries and dynamic properties of moving objects based on OGC® Moving Features. Moreover, we present a visual extension of Cesium to visualize moving objects in a space-time cube by cooperating with a data server that manages moving objects in a Cassandra database via RESTful APIs. This demonstration presents how to analyze a correlation between typhoon trajectories and geo-tagged Twitter messages with our systems.

Computing for human experience has become more important for understanding all of aspects of any interaction of human beings in the cyber, physical, and social environments. In particular, artificial intelligent technologies based on big data enable to understand natural language, enhance day to day human experience, and make a better decision. In this paper, we propose a method to classify unstructured text data on the Web into the five types of sensation features: sight (ophthalmoception), hearing (audioception), touch (tactioception), smell (olfacception), and taste (gustaoception). Even though sensation is the first process of human experience against the environments, the study of sensation information extraction is neglected due to lack of sensory expression and knowledge comparing with the sentimental analysis or opinion mining. We first define the sensation measurement that is assigned to each feature. Then, we identify which sensation feature has a strong influence on human perceptual experience in a specific topic of corpus. Finally, we evaluate our method by comparing with several baselines in terms of the accuracy.

Social networks have played a crucial role as information channels for people to understanding their daily lives beyond merely being communication tools. In particular, coupling social networks with geographic location has boosted the worth of social media to not only enable comprehension of the effects of natural phenomena such as global warming and disasters, but also the social patterns of human societies. However, the high rate of social data generation and the large amounts of noisy data makes it difficult to directly apply social media to decision-making processes. This article proposes a new system of analyzing the spatio-temporal patterns of social phenomena in real time and the discovery of local topics based on their latent spatio-temporal relationships. We will first describe a model that represents the local patterns of populations of geo-tagged social media. We will then define a local topic whose keywords share a region in space and time and present a system implementation based on existing open source technologies. We evaluated the model of local topics with several ways of visualization in experiments and demonstrated a certain social pattern from a dataset of daily Twitter streams. The results obtained from experiments revealed certain keywords had a strong spatio-temporal proximity even though they did not occur in the same message.

We are living in the Internet of Things (IoT) era where all the (smart) objects around us are connected and communicated with each other to serve our life better without the need of explicit instruction. Soon we have to cope with trillions of heterogeneous data streams coming from IoT. Since data is not information, methods for discovering useful and correlative information from data and utilising them for the better life, in real-time mode, are the utmost requirements.

On behalf of the rapidly and widely disseminated smartphone technology into the public, lots of social network sites and location-based social applications are accumulating a huge volume of massive crowd's daily experiences and thoughts in an unprecedented scale. We can regard them as novel data sources for accomplishing various social analytics, which have usually required lots of efforts to collect crowds' opinion and behavioral data. Thus, we can take advantages of abundant social datasets by integrating them appropriately. However, when we integrate disparate sources to derive a comprehensive view for a survey, it is necessary to know intrinsic exclusive values of each data source compared to others in an intuitive and succinct way. In fact, lots of efforts and time are wasted to overview various datasets consequently to confidently choose a dataset to be integrated in a final result. In this paper, we propose a complementarity index, which can estimate the exclusive usefulness of data sources in terms of spatial and topical coverage when selecting data sources for social analytics purposes. We conducted an experiment about complementarity measurement with two real social datasets from Twitter and VoiceTra; the latter is a speech-to-speech translation app, with which we can additionally obtain crowds' verbal translation logs. With the proposed complementarity index, we can measure the capability of a dataset comparing to others before integrating datasets, thus enabling analysts to examine much more datasets from as many related data sources as possible by focusing on exclusive coverage and relative strength of relevant topics.

With being coupled with geographic location, microblogging messages have become more important information resources to share observations and opinions about the real world via social media. As a result, we are getting much interested in comprehending situations related to natural and/or social events using those messages. In the recent, certain works have focused on showing trending topics that can represent snapshots of certain situations with spatial and temporal contexts on the basis of geo-tagged microblogging messages. They, however, have difficulty in tracking and comparing topic changes and movements in a spatiotemporal domain because of the separated geo-spatial and temporal components. This demonstration introduces mTrend, which constructs and visualizes spatiotemporal trends of topics, named as "topic movements", on the basis of the geo-tagged Tweets. In particular, its interactive visual mining tool allows users to intuitively understand the differences between topic movements over space and time. In the demonstration, we present the comparisons of topic movements using a few keywords related to the Great East Japan Earthquake.

Geosocial networking services allow users to create,use, and share information and to communicate with other people regarding geographical locations and time. Geosocial networking services generate large amounts of spatiotemporal contents that mainly include information about personal interests, activities, or real-life experiences. This paper discusses a new geovisualization tool named StickViz that helps retrieve k-neighbors by considering the spatial, temporal, and thematic interests of users over geosocial networking services.In particular, we define phenomena of interest for combining user interests with respect to location, time, and topic and propose a phenomenon-based k-neighbor query to find people having similar interests in spatial, temporal, and thematical dimensions. Moreover, we use a three-dimensional (two spatial dimensions and one temporal dimension) space so that the queries and spatiotemporal contents can be represented in the same space. Finally, we present a prototype of StickViz to navigate spatiotemporal contents and search k-neighbors on the basis of the predefined phenomena of interest through three-dimensional visual interfaces.

Context is a multi-dimensional concept. It is hard to define context generally for computer science. Which information is considered as context, which is not? Why are the certain context elements relevant for a certain case, but irrelevant for another? How to explain this to computers? Can computers learn these issues as humans do? In our paper we present different viewpoints to the concept of context and to context modelling starting from requirements engineering and ending up to multi-disciplinary education. Based on context related literature research and discussions in our paper, we can summarize that a complete and comprehensive definition and model of context is difficult to achieve and may not even be appropriate at all. However we can conclude that there is a common understanding that context always relates to an entity, context is used to solve a problem, context depends on the domain of use, context depends on time and context is evolutionary.

With the advance of ubiquitous computing and mobile environments, we have begun to continuously monitor changes in real-world condition and environment through wireless sensor networks. Opportunities also exist for people to create information related to the world around them by using mobile phones equipped with sensing devices, and share that information online with others. In this paper, we propose a novel approach for the interconnection of earth observation data and spatiotemporal web contents on the basis of spatiotemporal and thematic relationships. In particular, we use the concept of moving phenomena of interests to link between measurement sensing data and people-centric contents on the basis of spatiotemporal proximity and thematic relevance. This paper also shows a simple application that automatically generates semantic tags with respect to natural geographic phenomena, such as typhoons, climate changes, and air pollution, on the basis of our interconnection approach. We are able to easily understand qualitative meanings with respect to a certain phenomenon expressed by quantitative numeric conditions.

Various knowledge resources are spread to a world-wide scope. Unfortunately, most of them are community-based and never thought to be used among different communities. That makes it difficult to gain “connection merits” in a web-scale information space. This paper presents a three-layered system architecture for computing dynamic associations of events to related knowledge resources. The important feature of our system is to realize dynamic interconnection among heterogeneous knowledge resources by event-driven and event-centric computing with resolvers for uncertainties existing among those resources. This system navigates various associated data including heterogeneous data-types and fields depending on user's purpose and standpoint. It also leads to effective use for the sensor data because the sensor data can be interconnected with those knowledge resources. This paper also represents application to the space weather sensor data.

With the advance of the Web technologies, people can more easily access the geographic information and generate various types of user contents including geo-information on the Web. Consequently, geo-web and geo-communities have been infrastructures to share and connect information on the Web for many years, and people start to describe a specific phenomenon on places by own representation methods such as text, images, videos, etc. In this demonstration, we introduce a new type of location-based services, called Sticker, which can search and aggregate the relevant user generated contents/media to moving phenomena such as hurricanes, flooding, and global warming. In particular, the Sticker navigates user-generated contents with three dimensional view of space-time(2D+1D) and allows users to retrieve related information with the moving phenomena in a spatiotemporal domain as well as interesting keywords.