With the rapid growth of sensors, communication technologies, and social networks, it is now easy to collect large amounts of data from people and their surrounding environments, such as wearable devices, lifelog cameras, and ambient sensors. These data provide both personal and external views of human activities, but most existing work still focuses on analyzing each type of data separately. As a result, there is a clear gap in understanding how to combine and use cross-data across different sources. This workshop provides a platform for researchers from academia and industry to explore cross-data analysis and retrieval, with a focus on practical challenges such as integrating different data types, handling distributed data, and ensuring data security, aiming to support the development of a smart and sustainable society.

The proliferation of online misinformation has underscored the urgent need for robust, comprehensive verification tools. Existing approaches often lack the capacity for multi-modal analysis and evidence-based explainability, limiting their effectiveness in diverse, real-world scenarios. DeepVerify presents a complete end-to-end software solution for evidence-based online information verification, integrating previously published modules from ACMMM 2025 and supporting multi-lingual users. The system introduces a novel algorithm for detecting AI-generated content in images and text, extending prior capabilities that focused solely on video. The core idea of the algorithm is to compare the similarity of the input with information available online, allowing for the detection of AI-generated content. As a result, the system not only determines whether the data is created by humans or machines, but also provides specific evidence to support this decision. Unlike previous approaches that merely output a confidence score, this method offers concrete supporting evidence for its verdict, enhancing transparency and reliability in the verification process. Experiments conducted on benchmark datasets demonstrate that DeepVerify consistently outperforms current state-of-the-art methods, offering enhanced accuracy and transparent reasoning across multiple modalities. This work sets a new standard for explainable, reliable online information verification suitable for global audiences.

Worldwide image geolocalization, which aims to predict the GPS coordinates of any image on Earth, remains challenging due to global visual diversity. Recent generative approaches based on Retrieval-Augmented Generation (RAG) and Large Multimodal Models (LMMs) leverage candidates retrieved from fixed databases for reasoning, but often struggle with scenes that are absent from the reference set. In this work, we propose GeoSearch, an open-world geolocation framework that integrates web-scale reverse image search into the RAG pipeline. GeoSearch augments LMM prompts with database-retrieved coordinates and textual evidence extracted from web pages. To mitigate noise from irrelevant content, we introduce a two-layer filtering mechanism consisting of image matching, followed by confidence-based gating. Experiments on standard benchmarks Im2GPS3k and YFCC4k demonstrate the superiority of GeoSearch under leakage-aware evaluation. Our code and data are publicly available to support reproducibility.

The sixth edition of the Intelligent Cross-Data Analysis and Retrieval (ICDAR) workshop continues to serve as a forum for researchers and practitioners addressing the integration, analysis, and retrieval of heterogeneous data sources. While individual modalities such as wearable sensors, lifelogging cameras, and social media have been well studied, analyzing cross-data that incorporates multiple perspectives remains a crucial yet challenging task for advancing human-centered applications. In 2025, the workshop received 19 submissions, of which 7 were accepted following a careful peer-review process, resulting in an acceptance rate of 37%. The accepted papers covered a wide range of topics, including zero-shot composed image retrieval, vision-language scene understanding, adaptive modality fusion, lightweight fine-tuning with truncated SVD, and real-world federated split learning on mobile devices. By fostering interdisciplinary collaboration across domains such as well-being, disaster mitigation, mobility, food computing, and smart cities, the workshop continues to highlight emerging challenges and solutions for building intelligent, sustainable, and human-centric systems driven by cross-modal and multimodal data analytics.

In the context of smart cities, there is a growing demand for risk-free navigation systems that help citizens avoid congestion and enjoy outdoor activities in clean environments. Such systems require the ability to predict traffic and environmental hotspots by analyzing spatial-temporal big data from various sources, including IoT devices, stations, CCTV, and personal device networks. However, analyzing and forecasting spatial-temporal data presents significant challenges due to the complex interplay of spatial and temporal dependencies. To address these challenges, we introduce a novel dual attention mechanism graph transformer that leverages both spatial and temporal information to capture intricate patterns in spatial-temporal data. We evaluate our model on two forecasting tasks: air pollution and traffic flow. Our results demonstrate superior performance compared to other graph neural network models. Consequently, this model has been integrated into a traffic-risk navigator application, which will be evaluated in Yokohama, Japan, using real data collected from air pollution stations and traffic monitors.

Traffic predictions play a crucial role in intelligent transportation systems. The rapid development of IoT devices allows us to collect different kinds of data with high correlations to traffic predictions, fostering the development of efficient multi-modal traffic prediction models. Until now, there are few studies focusing on utilizing advantages of multi-modal data for traffic predictions. In this paper, we introduce a novel temporal attentive cross-modality transformer model for long-term traffic predictions, namely xMTrans, with capability of exploring the temporal correlations between the data of two modalities: one target modality (for prediction, e.g., traffic congestion) and one support modality (e.g., people flow). We conducted extensive experiments to evaluate our proposed model on traffic congestion and taxi demand predictions using real-world datasets. The results showed the superiority of xMTrans against recent state-of-the-art methods on long-term traffic predictions. In addition, we also conducted a comprehensive ablation study to further analyze the effectiveness of each module in xMTrans.

Information disorder is one of the most typical challenges in the current era of science and technology. The amount of information on the internet is increasing, but its correctness and authenticity are not always guaranteed, leading to false information, fake news, etc. The mentioned problem negatively affects users' reception and use of information. Unlike deepfake, cheapfake is created using simple techniques and does not rely on AI to produce fake multimedia. Cheapfake is becoming increasingly popular due to its ease of creation. Thus, there is a growing need to develop techniques that can detect cheapfake content. Following previous events, the Grand Challenge on Detecting Cheapfakes at ACM ICMR 2024 continues to seek contributions from researchers on cheapfake detection with the goal of improving effectiveness and creativity in approach, and understanding the limitations of the current dataset. This challenge has accepted 6 new proposed methods from participants with the highest private test accuracies achieved at 72.2% for Task 1 and 54.84% for Task 2. The highest public test accuracies for the two tasks are 95.6% and 93% respectively. These new methods focus on incorporating new AI models such as Stable Diffusion, LLM. These new findings represent the latest advancements in cheapfake detection research and introduce new potential approaches for future research.

Our workshop aims to provide a platform for both academic and industrial professionals engaged in the analysis and retrieval of cross-data from diverse perspectives, with a particular emphasis on wearable and ambient sensors, lifelog cameras, social networks, and surrounding sensors.Despite numerous studies exploring individual viewpoints, there remains a significant gap in the analysis and retrieval of cross-data to maximize benefits for humanity.Additionally, challenges such as data security and distributed learning for cross-modal model training and inference arise when dealing with large and distributed datasets.We invite researchers to contribute to this initiative, with the overarching goal of fostering the development of a smart and sustainable society through the efficient utilization of intelligent cross-data analysis and retrieval techniques.

This paper presents an innovative approach to predicting near-miss accidents, vital for proactive accident prevention. Leveraging dashcam footage and weather sensor data, our model integrates camera calibration, collision point prediction, heuristic knowledge, and analysis of near-miss accident patterns. We propose a comprehensive method to detect and predict potential collisions within the ego-vehicle's safe zone, utilizing a combination of machine learning techniques including DeepHough, YOLOv8, and LSTM. Furthermore, we introduce heuristic rules to handle sudden changes in object behavior and enhance object detection accuracy under challenging conditions like low visibility. Our approach identifies common near-miss accident patterns and achieves a prediction accuracy of 96.01% with support from hard brake detection. Comparative analysis demonstrates the superior performance of our method against existing benchmarks. Moreover, our lightweight model is designed for deployment on edge clients, ensuring real-time assistance to drivers. Collaboratively developed with government and industry stakeholders, our approach contributes to creating cost-effective smart driving assistance systems with wide-ranging applications in traffic safety and accident analysis.

The rise of social media enables access to valuable information but also fuels the spread of fake news and misinformation. Cheapfake is a type of misinformation created through simple techniques, often involving the use of unaltered images with misleading captions. To distinguish between Out-of-Context (OOC) and Not-Out-of-Context (NOOC) image-caption pairs, prior research has used text-to-image generative models to generate images from captions and then extract correlations between the generated images and the original images. Despite being unable to identify contradictions in the caption pairs, the aforementioned work has demonstrated promising potential for using generative models in cheapfake detection. In this paper, we introduce a novel framework that leverages a generative model to join the contents of an original image and a caption into a new image, referred to as a context-synthetic image. To convert the quantitative difference between an original image and a context-synthetic image, termed as the contextual deviation value, into OOC and NOOC labels, we train a classification model on a newly curated dataset of 7144 context-synthetic images generated using the Stable Diffusion model. We believe that our work offers valuable insights into the use of generative models for cheapfake detection, paving the way for future advancements in this field.

Cheapfakes is a new term for fake media that is made without AI, but with simple tools or captions that deceive or mislead. Cheapfakes include photos, videos, audio recordings, or any media that has been changed to distort its original meaning or context. The "ACM ICMR 2024 Grand Challenge on Detecting Cheapfakes" focuses on the challenge of finding out-of-context (OOC) media, which can assist fact-checkers in their work. By finding OOC media, we can narrow down the search space and raise the chances of discovering cheapfakes. To cope with this challenge, in this research paper, our team propose a novel approach that combines Context Generation and Text Classification as the key components in detecting cheapfakes. Our method was evaluated using 80% of the public test set from the COSMOS dataset, which includes two context labels: out-of-context and not-out-of-context. The results demonstrate an accuracy of 85.9%, validating the effectiveness and reliability of our method in identifying cheapfakes.

This paper introduces MM-TrafficRisk, an innovative fleet management application that harnesses dashcam video data, environmental data, and physiological data to forecast, mitigate, and investigate traffic-risk events while uncovering traffic-risk patterns. To provide a comprehensive overview of this application, we outline its system architecture, encompassing a database, ETL processes, UI/UX components, and a fine-grained text-video search engine. Additionally, we present a groundbreaking two-stage near-miss accident prediction model designed to identify near-miss incidents within dashcam video databases, and the text-to-video search engine, facilitating rapid searches for traffic-risk events based on textual queries. These models and search engines are rigorously evaluated in a controlled laboratory environment to showcase their advantages. Moreover, we highlight several essential functions of the MM-TrafficRisk application through snapshots, emphasizing the collaborative efforts between a government agency and industrial companies to develop and deploy this application in practical settings. We also delve into our future endeavors, focusing on multi-modal deep learning event prediction and the adaptability of our application to Edge AI environments.

The paper aims to present a collaboration between the industry and government sectors, focusing on creating a benchmarking platform for predicting urban risk transportation through the utilization of multimodal data. In this collaboration, the industry partner contributes datasets and customer preference surveys obtained from its business operations. On the other hand, government partners curate open datasets sourced from non-profit organizations in both private and public domains. Furthermore, the government provides an accessible platform that allows individuals to conveniently access and leverage resources for the purpose of advancing application development and engaging in research endeavors. Throughout the collaborative effort, a variety of techniques have been under development for forecasting urban risk transportation through the analysis of weather patterns, congestion levels, and people flow data. The core objective of this partnership is to formulate two foundational prediction methods. These methods are intended to serve as benchmarks, offering future users a dependable means to assess the performance of their own approaches in terms of both time-series and datapoints analytics methodologies.

Recently, there has been an increased interest in cross-data research problems, such as predicting air quality using life logging images, predicting congestion using weather and tweets data, and predicting sleep quality using daily exercises and meals. Although several research focusing on multimodal data analytics have been performed, few studies have been conducted on cross-data research (e.g., cross-modal data, cross-domain, cross-platform). The article collection “Intelligent Cross-Data Analysis and Retrieval” aims to encourage research in intelligent cross-data analytics and retrieval and contribute to the creation of a sustainable society. Researchers from diverse domains such as well-being, disaster prevention and mitigation, mobility, climate, tourism and healthcare are welcome to contribute to this Research Topic.

Driver reaction is of vital importance in risk scenarios. Drivers can take correct evasive maneuver at proper cushion time to avoid the potential traffic crashes, but this reaction process is highly experience-dependent and requires various levels of driving skills. To improve driving safety and avoid the traffic accidents, it is necessary to provide all road drivers with on-board driving assistance. This study explores the plausibility of case-based reasoning (CBR) as the inference paradigm underlying the choice of personalized crash evasive maneuvers and the cushion time, by leveraging the wealthy of human driving experience from the steady stream of traffic cases, which have been rarely explored in previous studies. To this end, in this paper, we propose an open evolving framework for generating personalized on-board driving assistance. In particular, we present the FFMTE model with high performance to model the traffic events and build the case database; A tailored CBR-based method is then proposed to retrieve, reuse and revise the existing cases to generate the assistance. We take the 100-Car Naturalistic Driving Study dataset as an example to build and test our framework; the experiments show reasonable results, providing the drivers with valuable evasive information to avoid the potential crashes in different scenarios.

Sleep plays a vital role in our physical, cognitive, and psychological well-being. Despite its importance, long-term monitoring of personalized sleep quality (SQ) in real-world contexts is still challenging. Many sleep researches are still developing clinically and far from accessible to the general public. Fortunately, wearables and IoT devices provide the potential to explore the sleep insights from multimodal data, and have been used in some SQ researches. However, most of these studies analyze the sleep related data and present the results in a delayed manner (i.e., today’s SQ obtained from last night’s data), it is sill difficult for individuals to know how their sleep will be before they go to bed and how they can proactively improve it. To this end, this paper proposes a computational framework to monitor the individual SQ based on both the objective and subjective data from multiple sources, and moves a step further towards providing the personalized feedback to improve the SQ in a data-driven manner. The feedback is implemented by referring the insights from the PMData dataset based on the discovered patterns between life events and different levels of SQ. The deep learning based personal SQ model (PerSQ), using the long-term heterogeneous data and considering the carry-over effect, achieves higher prediction performance compared with baseline models. A case study also shows reasonable results for an individual to monitor and improve the SQ in the future.

Split learning (SL) is a popular distributed machine learning (ML) method used to enable ML. It divides a neural network based model into subnetworks. Then, it separately trains the subnetworks on distributed parties (e.g., client and server). In distributed ML, data are generated and collected on the client-side. In contrast, the collected data are processed using an application deployed on the server side. However, when applied in practice using Internet of things systems and clients, numerous obstacles occur because of limited configuration and resources. Dividing neural networks in the SL is the biggest problem and an open question in numerous studies. This study introduces splitDyn, which is a new dynamic SL solution to solve the aforementioned problems. This method provides a solution for eliminating their inherent drawbacks. The main idea is to apply a Round-Robin schedule to select the client for the training process. Then, the next idea is to use the Hungarian optimization algorithm to assign a layer to a client and enhance the accuracy. The proposed method reasonably achieved better accuracy and reduced processing time than the other learning models. Furthermore, it applies the incident datasets to predict the incident event and in edge computing for edge artificial intelligence (AI) applications.

Traffic forecasting has been an important research topic in the intelligent transportation system for smooth and safe transportation. Among research directions, spatiotemporal-graph-based traffic forecasting has been known as one of the popular directions. Unfortunately, many requirements, such as lightweight architecture, low resource consumption, and flexible transfer learning, have not had satisfactory answers. Hence, we propose a new method called 3D Spatial Temporal Graph Point-wise Convolutional Network (3D-STGPCN) to address these requirements. Differing from others, we proceed with spatial and temporal information simultaneously (i.e., wrap them into one structure). We evaluate and compare our model on three public traffic network datasets, METR-LA, PEMS-BAY, and STREETS, with different methods in the same domain. The experimental results show the advantage of our model over others. We publish our source code at https://github.com/dophanh-26/3D-STGPCN.

We have witnessed the rise of cross-data against multimodal data problems recently. The cross-modal retrieval system uses a textual query to look for images; the air quality index can be predicted using lifelogging images; the congestion can be predicted using weather and tweets data; daily exercises and meals can help to predict the sleeping quality are some examples of this research direction. Although vast investigations focusing on multimodal data analytics have been developed, few cross-data (e.g., cross-modal data, cross-domain, cross-platform) research has been carried on. In order to promote intelligent cross-data analytics and retrieval research and to bring a smart, sustainable society to human beings, the specific article collection on "Intelligent Cross-Data Analysis and Retrieval" is introduced. This Research Topic welcomes those who come from diverse research domains and disciplines such as well-being, disaster prevention and mitigation, mobility, climate change, tourism, healthcare, and food computing

Dashcam video has become popular recently due to the safety of both individuals and communities. While an individual can have undeniable evidence for legal and insurance, communities can benefit from sharing these dashcam videos for further traffic education and criminal investigation. Moreover, relying on recent computer vision and AI development, few companies have launched the so-called AI dashcam that can alert drivers to near-risk accidents (e.g.., following distance detection, forward collision warning), forwarding to improving driver’s safety. However, even though dashcam videos create a driver’s travel log (i.e., traveling diary), little research focuses on creating a valuable and friendly tool to find any incident or event with few described sketches by users. Besides, most incident detection models have been built using a traditional supervised learning approach (i.e., collecting and labeling data for a new incident class). That prevents the quick and customized development of a new incident class. Inspired from these observations, we introduce an interactive incident detection and retrieval system for first-view travel-log data, namely MM-trafficEvent, that can (1) online defined-incident detection, (2) offline fine-grained incident retrieval for both defined and undefined incidents, (3) offline automatically new incident class creating using user’s queries. Moreover, the system gives promising results when being evaluated on several public datasets.

Due to the rapid recent development of cloud-edge networks, smart devices can facilitate rapid access to patients’ health information. Success has been achieved in the healthcare sector with the training of a federated learning (FL) model on large amounts of the personal data of users. However, some challenges remain that other FL models have not yet addressed. Firstly, FL models with computational parameters are very complex, which results in a high communication cost in the cloud-edge network. Furthermore, trained models in the cloud are not personalized. If personalization is present, the models do not provide practical solutions to fine-tune parameters in order to accurately predict performance in health monitoring. To address the above challenges, this paper presents the Fed xData framework for contextual health monitoring in cloud-edge networks. The Fed xData framework introduces a continuous data balancing supplemented structure using the RandomOverSample method, which solves all data classes. The FL model is an encode depth convolutional network (EDCN) model designed for both server and client. It solves various problems, for instance by using the fine-tuning model to increase personalization and solving not independent and identically (Non-IID) distribution problems regarding user health. Test results based on human activity recognition indicate that Fed xData is far superior to others for use in general centralized learning models and FL models.

Currently, air pollution has become the tremendous problem humankind has ever faced. Countries governments have struggled with conflicting demands from industries benefits and people/natural health. Scientists have investigated to find solutions that can balance these demands. Along with these directions, this research introduces a convenient and economical solution for estimating PM2.5 at the current time and predicting PM2.5 in a short- and medium-term period just by using images captured from personal devices (e.g., smartphones, cameras, lifelog cameras). The proposed method aims to leverage the association between urban nature (e.g., street greenness, street building), urban traffic (e.g., the volume of vehicles), and air pollution (e.g., PM2.5) to discover a set of periodic-frequent patterns and to build the PM2.5 estimation model. The estimated PM2.5, together with a set of patterns, is utilized to predict the PM2.5 in the short-term future. Evaluation running on different datasets collected from India and Vietnam shows the productivity of the proposed method.

Cross-data analytics and retrieval have gained significant improvement recently. People can now extract more data insights precisely and quickly towards having many excellent applications serving human lives. Since people create multimedia and other types of data that reflect the diverse perspectives of human lives, these data are just pieces of the puzzle of the world's pictures. Hence, it is necessary to assembly all these pieces towards having a better solution for human-centered problems. Hence, the workshop welcomes those who work with multimedia and others and come from diverse research domains and disciplines to work on intelligent cross-data analytics and retrieval to bring a smart, sustainable society to human beings. The research domain can vary from well-being, disaster prevention and mitigation, mobility to food computing, to name a few.

Nowadays, resolving chaotic traffic situations, which usually link to traffic congestion, is an essential need. It poses many risks to commuters like traffic accidents, especially during bad weather situations. Besides, owing to the exponential growth of IoT technologies, it is easier than ever to collect a huge amount of urban sensing data. Therefore, building a system to anticipate congestion from the collected data could enhance public safety and give traffic police forces enough time to handle traffic flows in potentially dangerous areas. Moreover, if we can discover patterns in which predicted congestion usually happens, we can build reaction plans with various alert codes. They create dynamic risk maps that can provide useful knowledge to both authorities and travelers to make rescue and travel plans effectively. This paper proposes a novel framework to address these problems. The proposed framework employs the Enhanced-Fusion-3DCNN deep learning model to predict future long-term traffic congestion on a particular mesh-code at a particular time instance. The predicted traffic congestion data is later transformed into a temporal database and feed to the maximal periodic-frequent pattern algorithm to identify the sets of mesh-code in which regular congestion may happen in the predicted data. Experimental results on real-world traffic congestion data demonstrate that the proposed framework is efficient.

In many countries worldwide, effectively estimating AQI values and levels is essential for better monitoring the air pollution around the living area. This problem has become one of the interesting research subjects for many years, and there are many applications developed for personal usages. In this work, we aim to investigate a multi-source machine learning approach to approximate the local AQI scores at users' location in a big city. We conduct different experiments on three primary data sets: "SEPHLA-MediaEval 2019", "MNR-Air-HCM," and "MNR-HCM," collected in Ho Chi Minh City (Vietnam) and Fukuoka city (Japan). From the data sets provided, we extract different types of useful attributes for the problem: the timestamp information, the geographical data, sensor data (humidity and temperature), users' emotion tags (such as greenness, calmness, etc.), the semantic features from images captured by users as well as the public weather data (including temperature, dew point, humidity, wind speed, and pressure) of the related cities. After that, we compare five distinct machine learning models for estimating the local AQI score and level, including Support Vector Machine [1], Random Forest [2], Extreme Gradient Boosting [3], LightGBM [4] and CatBoost [5]. We use RMSE, MAE, and R2for measuring the performance of these approaches. The experimental results show that using random forest with sensor data, combined with public weather data, the results in AQI values regression and AQI ranks prediction can be the highest in many cases.

Finding Spatial High Utility Itemsets (SHUIs) in a spatiotemporal database is a challenging problem of great importance in many real-world applications. Most previous works focused on the sequential discovery of SHUIs in a database running on a single machine. Consequently, these works are not suitable for big data (or cloud-based) applications as they suffer from the scalability and fault tolerant problems. This paper proposes several novel pruning techniques to reduce the search space and present a more flexible distributed algorithm to find all desired itemsets from the database using Spark in-memory computing architecture. Our algorithm inherits several advantages of Spark, including low communication cost, fault tolerance, and high scalability. Experimental results demonstrate that the proposed algorithm has good scalability and performance on very large databases. Finally, we present a real-world navigation application in which SHUIs generated from the traffic congestion data have been employed to recommend alternative routes to the users.

Nowadays, recommendation systems play an indispensable role in many fields, including e-commerce, finance, economy, and gaming. There is emerging research on publication venue recommendation systems to support researchers when submitting their scientific work. Several publishers such as IEEE, Springer, and Elsevier have implemented their submission recommendation systems only to help researchers choose appropriate conferences or journals for submission. In this work, we present a demo framework to construct an effective recommendation system for paper submission. With the input data (the title, the abstract, and the list of possible keywords) of a given manuscript, the system recommends the list of top relevant journals or conferences to authors. By using state-of-the-art techniques in natural language understanding, we combine the features extracted with other useful handcrafted features. We utilize deep learning models to build an efficient recommendation engine for the proposed system. Finally, we present the User Interface (UI) and the architecture of our paper submission recommendation system for later usage by researchers.

The First International Workshop on "Intelligence Cross-Data Analytics and Retrieval" (ICDAR'20) welcomes any theoretical and practical works on intelligence cross-data analytics and retrieval to bring the smart-sustainable society to human beings. We have witnessed the era of big data where almost any event that happens is recorded and stored either distributedly or centrally. The utmost requirement here is that data came from different sources, and various domains must be harmonically analyzed to get their insights immediately towards giving the ability to be retrieved thoroughly. These emerging requirements lead to the need for interdisciplinary and multidisciplinary contributions that address different aspects of the problem, such as data collection, storage, protection, processing, and transmission, as well as knowledge discovery, retrieval, and security and privacy. Hence, the goal of the workshop is to attract researchers and experts in the areas of multimedia information retrieval, machine learning, AI, data science, event-based processing and analysis, multimodal multimedia content analysis, lifelog data analysis, urban computing, environmental science, atmospheric science, and security and privacy to tackle the issues as mentioned earlier.

With the rapid growth of users' data in SaaS (Software-as-a-service) platforms using micro-services, it becomes essential to detect duplicated entities for ensuring the integrity and consistency of data in many companies and businesses (primarily multinational corporations). Due to the large volume of databases today, the expected duplicate detection algorithms need to be not only accurate but also practical, which means that it can release the detection results as fast as possible for a given request. Among existing algorithms for the deduplicate detection problem, using Siamese neural networks with the triplet loss has become one of the robust ways to measure the similarity of two entities (texts, paragraphs, or documents) for identifying all possible duplicated items. In this paper, we first propose a practical framework for building a duplicate detection system in a SaaS platform. Second, we present a new active learning schema for training and updating duplicate detection algorithms. In this schema, we not only allow the crowd to provide more annotated data for enhancing the chosen learning model but also use the Siamese neural networks as well as the triplet loss to construct an efficient model for the problem. Finally, we design a user interface of our proposed deduplicate detection system, which can easily apply for empirical applications in different companies.

Lifelogging is known as the new trend of writing diary digitally where both the surrounding environment and personal physiological data and cognition are collected at the same time under the first perspective. Exploring and exploiting these lifelog (i.e., data created by lifelogging) can provide useful insights for human beings, including healthcare, work, entertainment, and family, to name a few. Unfortunately, having a valuable tool working on lifelog to discover these insights is still a tough challenge. To meet this requirement, we introduce an interactive multimodal retrieval system that aims to provide people with two functions, memory assistant and life organized support, with a friendly and easy-to-use web UI. The output of the former function is a video with footages expressing all instances of events people want to recall. The latter function generates a statistical report of each event so that people can have more information to balance their lifestyle. The system relies on two major algorithms that try to match keywords/phrases to images and to run a cluster-based query using a watershed-based approach.

Having an effective and efficient model to predict traffic congestion using multi-sources data has challenged researchers for decades, especially when the number of data sources and data volume increase dramatically. In this research, we propose a new CNN-based approach that can absorb and wrap multi-sources data into a 2D/3D raster-image to predict traffic congestion. Thanks to the raster-image-based wrapping technique, the spatial-temporal correlation is conserved entirely. The proposed approach can (1) accurately predict traffic congestion over multi-scale areas at different time horizons, (2) additionally consider external factors that could affect traffic flows and might cause traffic congestion afterwards by using an immediate fusion strategy. Traffic congestion, precipitation data, posts on a social networking platform collected in Kobe city, Japan during the summer and fall of the two consecutive years are used to evaluate the proposed approach. The comparison to other methods working on the same topic is also conducted to confirm the advantage of the proposed approach. Last but not least, some insights regarding the consequences of spatial and temporal dimensions as well as external factors to different time windows are also discussed.

Predictive analytics are concerned as a type of complex event processing where a complex event can be predicted by utilizing insights extracted from a set of related events. This paper introduces a new complex event analysis for traffic risk prediction using 3D-CNN and a set of related events detected from multi-sources urban sensing data (e.g., congestion, traffic accident, precipitation). The contribution of this paper involves (1) the spatio-temporal information of multi-sources urban sensing data is reserved and wrapped into 3D raster images towards being able to leverage recent developments of 3D-CNN to conduct predictive analytics, (2) the imbalanced data problem which could severely affect the performance of deep learning models is tackled by straightening curved geographic chains, (3) traffic risks can be predicted well in both short-term and medium-term time horizons, and (4) The influence of related events detected from extra factors on a complex event can be explained explicitly. The proposed method is evaluated on the real dataset collected in Kobe, Japan during 2014 and 2015. The comparisons to baseline methods such as historical average and 2D-CNN show the advantage of the proposed method as well.

Air Quality Index (AQI) is an indicator of the rank of air pollution that is very vital for the environmental impacts to the public health. In this paper, we propose an association model between visual feature and AQI rank of lifelog data. Visual data (i.e., environmental pictures) and numerical data (i.e., environmental AQI measurements) of lifelog are utilized for the data training stage. The features of the visual data are extracted using a CNN-based method, where the latter are calculated using the standard AQI ranking. The extracted visual features and ranked AQI are combined as the input data for a deep neural network MLP (Multi-layer Perception) to study the association relationship between visual feature and AQI rank. The experimental results show that the proposed method can provide accurate predictions of good or unhealthy AQI ranks from lifelog visual data.

Torrential rains, the complicated network of roads, and the high density of vehicles contribute partly the number of traffic accidents. In order to understand the association between these factors towards building a risk map that can alert drivers of dangerous zones, the visual patterns reasoning system is proposed. By converting sensing data collected from different factors to raster images, the associations can be treated as visual patterns that can conserve their spatiotemporal information. Deep convolutional neural networks (Deep-CNN) are utilized to build a model based on these raster images towards detecting accidents based on the association between factors. Image clustering is applied to learn a representation of each type of associations. Thus, the visual pattern of high-probability traffic accidents can be reasoned in the natural language format. Both 2D and 3D raster images are investigated to examine the spatial and spatiotemporal associations between factors with Deep-CNN models. We use both transfer learning and fine-tuning approach to build our model due to the small size of the positive samples dataset. The evaluation shows the initial but promising results of our method. We also discuss the potential applications and various research directions can be investigated using our proposed method.

Natural disasters, especially those related to flooding, are global issues that attract a lot of attention in many parts of the world. A series of research ideas focusing on combining heterogeneous data sources to monitor natural disasters have been proposed, including multi-modal image retrieval. Among these data sources, social media streams are considered of high importance due to the fast and localized updates on disaster situations. Unfortunately, the social media itself contains several factors that limit the accuracy of this process such as noisy data, unsynchronized content between image and collateral text, and untrusted information, to name a few. In this research work, we introduce a context-aware late-fusion approach for disaster image retrieval from social media. Several known techniques based on context-aware criteria are integrated, namely late fusion, tuning, ensemble learning, object detection and scene classification using deep learning. We have developed a method for image-text content synchronization and spatial-temporal-context event confirmation, and evaluated the role of using different types of features extracted from internal and external data sources. We evaluated our approach using the dataset and evaluation tool offered by MediaEval2017: Emergency Response for Flooding Events Task. We have also compared our approach with other methods introduced by MediaEval2017's participants. The experimental results show that our approach is the best one when taking the image-text content synchronization and spatial-temporal-context event confirmation into account.

ETL (Extraction-Transform-Load) tools, traditionally developed to operate offline on historical data for feeding Datawarehouses, need to be enhanced to deal with big and fresh data and be executed at network level during data streams acquisition.In this paper, we present StreamLoader, a Web application for the specification of conceptual ETL dataflows on heterogeneous sensor data that leverages the peculiarities of network configuration, data stream management, and specification and deployment of ETL operations in a programmable network.It can be used for feeding traditional/ real-time data-warehouses or visual analytic tools.

In this paper, we introduce a new method that explores spatio-temporal-theme correlations between physical and social streaming data for event detection and pattern interpretation from heterogeneous sensors. Particularly, we employ a basic two-phase framework in pattern recognition (i.e. feature extraction and detection) with the novel improvement that concerns the use of semantic information acquired from social sensors to automatically label the low-level features extracted from physical sensors. Moreover, by symbolizing the trend component of time-series data, the proposed method has an ability to interpret event's patterns to help users get insights of how events happen. Differentiating from conventional supervised learning methods whose training data are labeled manually and in an off-line mode, the proposed method can collect labels for training data automatically and in an on-line mode. Moreover, after running for a certain time, a training stage can run parallel with the detecting stage when an event model is totally built. After that, the training stage continues learning to increase the accuracy of the event model by nonstop collecting new samples with labels from streaming data. The problem of environmental factors and particularly air pollution impacts on asthma exacerbation is considered for evaluating the proposed method. The experimental results show that the proposed method can probably detect the prevalence of asthma risks in a specific spatio-temporal context as well as help users understand how a change in the surrounding environment (e.g. weather condition and air pollution) can influence their health (e.g. asthma attack) by interpreting detected event's patterns.

We are living in the Internet of Things (IoT) era where all the (smart) objects around us are connected and communicated with each other to serve our life better without the need of explicit instruction. Soon we have to cope with trillions of heterogeneous data streams coming from IoT. Since data is not information, methods for discovering useful and correlative information from data and utilising them for the better life, in real-time mode, are the utmost requirements.

In this contribution, we propose a watershed-based method with support from external data sources and visual information to detect social events in web multimedia. The idea is based on two main observations: (1) people cannot be involved in more than one event at the same time, and (2) people tend to introduce similar annotations for all images associated to the same event. Based on these observations, the metadata is turned to an image so that each row contains all records belonging to one user; and these records are sorted by time. Thus, the social event detection is turned to watershed-based image segmentation, where Markers are generated by using (keyword, location, visual) features with support of external data sources, and the Flood progress is carried on by taking into account (tags set, time, visual) features. We test our algorithm on the MediaEval 2012 dataset both using only external data but also introducing visual information.

Events are becoming very popular as a tool to organize and access large media collections. An unsolved problem however, is how to define event models. Most part of the approaches so far proposed in the literature are based on a-priori knowledge, and translate into hierarchical data structures or taxonomies a more or less intuitive definition of what a given type of event is. The association of media and event models is then a consequent process, in which one tries to learn the distinctive characteristics of media associated to a certain event or sub-event. In this paper, we attempt to reverse this paradigm, inferring from a set of media collections belonging to the same event class the underlying taxonomy in an unconstrained way. As a result we obtain a hierarchy of natural clusters, largely shared by the different collections, which capture the essence of the event itself. Although it is not possible to compare the proposed approach with state-of-the-art method based on a-priori event structures, experimental results demonstrate that this approach may become an effective support for discovering and defining event models and managing event-related data collections.