Effective monitoring of essential road infrastructure is critical to ensure economic growth and community connectivity, as deterioration from climate change and intensive use can compromise safety and hamper emergency services. To address these critical issues, we propose optimized approaches for road damage detection using YOLOv5, YOLOv8, and YOLOv10 models, with fine-tuning and hyperparameter optimization to improve detection accuracy. Our approaches achieve good overall accuracy (in terms of F1-score) by combining models of different sizes and frameworks using ensemble methods. Moreover, to enhance the speed of inference, parallel processing techniques are used. In order to obtain fast detection performance, small models, such as YOLOv5-small and YOLOv8-nano were given priority in modeling to balance between detection accuracy and inference speed. In the context of the 2024 IEEE BigData Optimized Road Damage Detection Challenge (ORDDC’2024), our approach achieved an F1 score greater than 0.7 with an inference speed of 0.0432 seconds per image, meeting the ORDDC’s accuracy and inference speed goal.

Automated monitoring systems have been utilized for effective road maintenance in order to eliminate time-consuming and manual inspection by road administration employees. Image-based technology has grown as an important option since a variety of images, such as from surveillance cameras or on-dash cameras, are widely obtainable. Towards that solution, this paper introduces a deep learning-based approach for detecting and classifying road damages in diverse images collected from different countries. Our approach integrates individual models trained per country and a general model trained for all countries. Our approach were evaluated thoroughly using the 2022 IEEE BigData Crowdsensing-based Road Damage Detection Challenge (CRDDC) Datasets. Experimental results show that our approach achieved an F1 score of up to 0.73.

With the availability of massive amounts of visual data covering wide geographical regions, various image learning applications have emerged, including classifying the street cleanliness level, detecting forest fires or road hazards. Such applications share similar characteristics as they need to 1) detect specific objects or events (what), 2) associate the detected object with a location (where), and 3) know the time that the event happened (when). Advancements in image-based machine learning (ML) benefit these applications as they can automate the detection of objects of interest. Along with the edge computing (EC) paradigm, the processing cost is offloaded to the devices, hence reducing latency and communication cost. Moreover, sensors on the edge devices (e.g., GPS) enrich the collected data with metadata. However, a shortcoming of existing approaches is that they rely on pre-trained "static" models. Nonetheless, crowdsourced data at diverse locations can be leveraged to iteratively improve the robustness of a model. We refer to the aforementioned strategy as "spatial crowd-based learning".To showcase this class of applications, we present FloraVision, an end-to-end system that integrates ML, crowdsourcing, and EC to automate the detection, mapping, and exploration of California Native Plants. FloraVision implements a pipeline to collect and clean publicly available image data, train a lightweight MobileNet-based classification model, and then deploy the model on mobile devices. It leverages spatial crowd-based learning to iteratively evolve the initial model from crowdsourced data. Its mobile application facilitates detecting plants and mapping their geolocations. Finally, it allows end-users to submit ad hoc spatio-temporal nearest neighbor queries and visualizes the results in an augmented reality user interface. Although our application focuses on plants, several other applications follow similar architectural patterns.

For efficient road maintenance, an automated monitoring system is required to avoid laboriously and time-consuming manual inspection by road administration crews. One potential solution is to utilize image processing-based technologies, especially, as various sources of images have readily been available, e.g., surveillance cameras, in-vehicle cameras, or smartphones. Such image-based solutions enable detecting and classifying road damages. This paper introduces deep learning-based image analysis for road damage detection and classification. Our ensemble learning approaches with test time augmentation were thoroughly evaluated using the 2020 IEEE Big Data Global Road Damage Detection Challenge Dataset. Experimental results show that our approaches achieved an F1 score of up to 0.67, allowing us to win the Challenge.

Due to the prevalence of GPS-equipped cameras (e.g., smartphones and surveillance cameras), massive amounts of geo-tagged images capturing urban streets are increasingly being collected. Consequently, many smart city applications have emerged, relying on efficient image search. Such searches include spatial-visual queries in which spatial and visual properties are used in tandem to retrieve similar images to a given query image within a given geographical region. Towards this end, new index structures that organize images based on both spatial and visual properties are needed to efficiently execute such queries. Based on our observation that street images are typically similar in the same spatial locality, index structures for spatial-visual queries can be effectively built on a spatial index (i.e., R*-tree). Therefore, we propose a class of R*-tree indexes, particularly, by associating each node with two separate minimum bounding rectangles (MBR), one for spatial and the other for (dimension-reduced) visual properties of their contained images, and adapting the R*-tree optimization criteria to both property types.

Due to the ubiquity of GPS-equipped cameras such as smartphones, more photos are getting automatically tagged with camera locations (referred to as geo-tagged images) so large-scale geo-tagged image datasets are available on the Web. And a significant portion of online images such as travel and surveillance may not be meaningful without their location information. Thus, image localization for untagged images has been studied. However, the point camera location of an image might be quite different from the location of the scene depicted in the image (referred to as scene location) rendering image localization inaccurate. To address this problem, we propose a data-centric framework for image scene localization using a CNN-based classification in three steps. First, the framework provides two mechanisms for constructing a reference image dataset tagged with scene locations. Second, a spatial-visual classification approach organizes a dataset spatially using R-tree to generate a set of geographical regions tightly bounding the image scene locations. Then, we train a classifier based on the classes of images corresponding to the generated regions. Finally, to enhance the classification accuracy, we train a set of hierarchical classification models utilizing the spatial hierarchical structure of the R-tree where the trained models enable learning the visual features of images at different geographical granularities. We evaluate our framework using a geo-tagged image dataset obtained from Google Street View and demonstrate that the utilization of scene locations enables localizing images far more accurately as compared with camera location based localization.

With recent advances in technology, it is feasible to conveniently monitor urban roads using various cameras, such as surveillance cameras, in-vehicle cameras, or smartphones, and recognize their conditions by detecting specific types of road damages in order to plan maintenance resources efficiently based on the identified spots. This paper describes a road damage type detection and classification solution submitted to the IEEE BigData Cup Challenge 2018. Our solution is based on the state-of-the-art deep learning methods for an object detection task. In particular, our approach utilizes an object detection algorithm to detect various types of road damages by training the detector on different image examples categorized into a set of damages defined by Japan Road Association. We evaluated our approach thoroughly using different versions of trained models. Our experiments show that our approach was able to achieve an F1 score up to 0.62.

Sentiment analysis of disaster-related posts in social media can contribute to the situation awareness and better understanding of the dynamics of disaster events by identifying the polarity of sentiments expressed by the public. However, Even though many sentiment analysis techniques have been developed and available, there are still limitations in reliably using sentiment analysis since there is no dominantly accepted technique in disasters. Taking advantage of existing state-of-the-art sentiment classifiers, this paper proposes a novel framework for geo-spatial sentiment analysis of disaster-related social media data objects. Our framework addresses three types of challenges: the inaccuracy and discrepancy associated with various text and image sentiment classifiers, the geo-sentiment discrepancy among data objects in a local geographical area, and observing diverse sentiments from multimedia data objects (i.e., text and image). The extracted sentiments are aggregated geographically for the purpose of extracting more accurate local regional insights. For the evaluation of the framework, we explored Twitter and Flickr datasets at the time of Hurricane Sandy and Napa Earthquake and showed how our approach can provide a better understanding of disaster events.

Recently spatial crowd sourcing was introduced as a natural extension to traditional crowd sourcing allowing for tasks to have a geospatial component, i.e., A task can only be performed if a worker is physically present at the location of the task. The problem of assigning spatial tasks to workers in a spatial crowd sourcing system can be formulated as a weighted bipartite b-matching graph problem that can be solved optimally by existing methods for the minimum cost maximum flow problem. However, these methods are still too complex to run repeatedly for an online system, especially when the number of incoming workers and tasks increases. Hence, we propose a class of approaches that utilizes an online partitioning method to reduce the problem space across a set of cloud servers to construct independent bipartite graphs and solve the assignment problem in parallel. Our approaches solve the spatial task assignment approximately but competitive to the exact solution. We experimentally verify that our approximate approaches outperform the centralized and Map Reduce version of the exact approach with acceptable accuracy and thus suitable for online spatial crowd sourcing at scale.