Large image datasets have driven innovations in image-based machine learning, especially supervised object detection. However, manual annotation of such datasets has been challenging due to the issues related to labeling time, the subjectivity of human perception, and the scalability of dataset size. To that end, semi-supervised object detection (SSOD) methods have emerged that leverage a small amount of manually labeled and much larger unlabeled data. However, existing SSOD methods have been trained over clean, noise-free datasets such as MS-COCO and PASCAL VOC. They thus may fail to capture the reality, i.e., the heterogeneity and nuances of objects in the real world, which contain noises such as occlusion, illumination, and irregularity in the object’s shape. This study conducts an empirical analysis of the performance of semi-supervised object detection on a real-world custom dataset of homeless encampments. Results show that certain noise conditions may be ignored during the manual annotation process without causing a significant drop-off in SSOD performance across various metrics.

Effective monitoring of essential road infrastructure is critical to ensure economic growth and community connectivity, as deterioration from climate change and intensive use can compromise safety and hamper emergency services. To address these critical issues, we propose optimized approaches for road damage detection using YOLOv5, YOLOv8, and YOLOv10 models, with fine-tuning and hyperparameter optimization to improve detection accuracy. Our approaches achieve good overall accuracy (in terms of F1-score) by combining models of different sizes and frameworks using ensemble methods. Moreover, to enhance the speed of inference, parallel processing techniques are used. In order to obtain fast detection performance, small models, such as YOLOv5-small and YOLOv8-nano were given priority in modeling to balance between detection accuracy and inference speed. In the context of the 2024 IEEE BigData Optimized Road Damage Detection Challenge (ORDDC’2024), our approach achieved an F1 score greater than 0.7 with an inference speed of 0.0432 seconds per image, meeting the ORDDC’s accuracy and inference speed goal.

The analysis of trajectory datasets has numerous applications ranging from urban planning to human mobility understanding, but to protect the privacy of individuals trajectory datasets are rarely released to researchers. And even when they are, they are limited in size and spatio-temporal coverage. To address these issues a number of methods for generating synthetic yet realistic trajectory datasets have been proposed. These existing methods either require a lot of complex parameters to be calibrated (simulators) or rely on existing trajectory datasets (generative models). In this paper, we propose Data-Driven Trajectory Generator, dubbed DDTG, a data-driven, model-free, and parameter-less algorithm for generating realistic synthetic vehicle trajectory datasets. Unlike existing approaches, DDTG relies on aggregate origin-destination and traffic data, both of which are publicly available and free of privacy concerns. Furthermore, we show that our method is orthogonal to the existing approaches with which DDTG can be combined to generate synthetic datasets of higher quality. Our experiments with real-world trajectory and traffic data show that the datasets generated by DDTG follow distributions that are very close to the distributions of real trajectory datasets.

Automated monitoring systems have been utilized for effective road maintenance in order to eliminate time-consuming and manual inspection by road administration employees. Image-based technology has grown as an important option since a variety of images, such as from surveillance cameras or on-dash cameras, are widely obtainable. Towards that solution, this paper introduces a deep learning-based approach for detecting and classifying road damages in diverse images collected from different countries. Our approach integrates individual models trained per country and a general model trained for all countries. Our approach were evaluated thoroughly using the 2022 IEEE BigData Crowdsensing-based Road Damage Detection Challenge (CRDDC) Datasets. Experimental results show that our approach achieved an F1 score of up to 0.73.

The pervasive deployment of IoT devices along with the advancements in Deep Neural Network (DNN) models have enabled video analytics at the edge, the so-called Edge AI systems, in support of various large smart-city applications such as automatic road damage evaluation and fire detection. Current solutions require the model developer to make the placement decision by manually assigning models to edge devices. However, an Edge AI solution could entail hundreds of mobile edge devices operating in a large geographical region (e.g., installed on vehicles) with various resource capabilities and different DNN models, hence rendering manual placement ineffective. This paper presents alternative methods to automatically place various models on a diverse set of edge devices, considering the geospatial coverage of video data, resource capabilities of edge devices, and the characteristics of the trained models. First, we mathematically formulate the model placement as an optimization problem which is proven to be NP-Hard. We then propose several heuristics to solve it efficiently and evaluate them with a real-world dataset collected along the 165 bus route trajectories in the City of San Francisco. Our placement algorithm yields a higher recall in object detection and is more robust to the uncertainty of the underlying location context, without sacrificing much utilization cost.

With the availability of massive amounts of visual data covering wide geographical regions, various image learning applications have emerged, including classifying the street cleanliness level, detecting forest fires or road hazards. Such applications share similar characteristics as they need to 1) detect specific objects or events (what), 2) associate the detected object with a location (where), and 3) know the time that the event happened (when). Advancements in image-based machine learning (ML) benefit these applications as they can automate the detection of objects of interest. Along with the edge computing (EC) paradigm, the processing cost is offloaded to the devices, hence reducing latency and communication cost. Moreover, sensors on the edge devices (e.g., GPS) enrich the collected data with metadata. However, a shortcoming of existing approaches is that they rely on pre-trained "static" models. Nonetheless, crowdsourced data at diverse locations can be leveraged to iteratively improve the robustness of a model. We refer to the aforementioned strategy as "spatial crowd-based learning".To showcase this class of applications, we present FloraVision, an end-to-end system that integrates ML, crowdsourcing, and EC to automate the detection, mapping, and exploration of California Native Plants. FloraVision implements a pipeline to collect and clean publicly available image data, train a lightweight MobileNet-based classification model, and then deploy the model on mobile devices. It leverages spatial crowd-based learning to iteratively evolve the initial model from crowdsourced data. Its mobile application facilitates detecting plants and mapping their geolocations. Finally, it allows end-users to submit ad hoc spatio-temporal nearest neighbor queries and visualizes the results in an augmented reality user interface. Although our application focuses on plants, several other applications follow similar architectural patterns.

Generating hyper-local news at scale is challenging because publicly available data is not provided at the desired spatial and temporal granularity. Besides, there is a lack of automated analytical and publishing tools. Crosstown Foundry, which is being actively developed and used by engineers and journalists, is a novel data-driven system that leverages a massive multi-modal dataset to generate personalized newsletters for Los Angeles County readers.

For efficient road maintenance, an automated monitoring system is required to avoid laboriously and time-consuming manual inspection by road administration crews. One potential solution is to utilize image processing-based technologies, especially, as various sources of images have readily been available, e.g., surveillance cameras, in-vehicle cameras, or smartphones. Such image-based solutions enable detecting and classifying road damages. This paper introduces deep learning-based image analysis for road damage detection and classification. Our ensemble learning approaches with test time augmentation were thoroughly evaluated using the 2020 IEEE Big Data Global Road Damage Detection Challenge Dataset. Experimental results show that our approaches achieved an F1 score of up to 0.67, allowing us to win the Challenge.

Due to the prevalence of GPS-equipped cameras (e.g., smartphones and surveillance cameras), massive amounts of geo-tagged images capturing urban streets are increasingly being collected. Consequently, many smart city applications have emerged, relying on efficient image search. Such searches include spatial-visual queries in which spatial and visual properties are used in tandem to retrieve similar images to a given query image within a given geographical region. Towards this end, new index structures that organize images based on both spatial and visual properties are needed to efficiently execute such queries. Based on our observation that street images are typically similar in the same spatial locality, index structures for spatial-visual queries can be effectively built on a spatial index (i.e., R*-tree). Therefore, we propose a class of R*-tree indexes, particularly, by associating each node with two separate minimum bounding rectangles (MBR), one for spatial and the other for (dimension-reduced) visual properties of their contained images, and adapting the R*-tree optimization criteria to both property types.

An isochrone is generally defined as a curve drawn on a map connecting points at which moving objects (e.g., cars) arrive at the same time. Their construction is an important task in many application domains. As an example, in urban planning, isochrones are essential when assessing the placement of public services like hospitals and fire departments. In this study, we formally define the isochrone and reverse isochrone problems, describe our approach to solving them and provide a fully functional system that is capable of visualizing the reachability in various ways. Unlike other studies, our approach is purely data-driven and does not depend on the underlying road network for computing the isochrone. Instead, we focus on directly processing trajectory data. Our system processes two real-world taxi datasets to visualize the reachability of the cities of Seoul and Xi'an. As our experiments show, our approach outperforms the traditional graph-theory techniques while eliminating the expensive need of preprocessing the data.

Due to the ubiquity of GPS-equipped cameras such as smartphones, more photos are getting automatically tagged with camera locations (referred to as geo-tagged images) so large-scale geo-tagged image datasets are available on the Web. And a significant portion of online images such as travel and surveillance may not be meaningful without their location information. Thus, image localization for untagged images has been studied. However, the point camera location of an image might be quite different from the location of the scene depicted in the image (referred to as scene location) rendering image localization inaccurate. To address this problem, we propose a data-centric framework for image scene localization using a CNN-based classification in three steps. First, the framework provides two mechanisms for constructing a reference image dataset tagged with scene locations. Second, a spatial-visual classification approach organizes a dataset spatially using R-tree to generate a set of geographical regions tightly bounding the image scene locations. Then, we train a classifier based on the classes of images corresponding to the generated regions. Finally, to enhance the classification accuracy, we train a set of hierarchical classification models utilizing the spatial hierarchical structure of the R-tree where the trained models enable learning the visual features of images at different geographical granularities. We evaluate our framework using a geo-tagged image dataset obtained from Google Street View and demonstrate that the utilization of scene locations enables localizing images far more accurately as compared with camera location based localization.

With recent advances in technology, it is feasible to conveniently monitor urban roads using various cameras, such as surveillance cameras, in-vehicle cameras, or smartphones, and recognize their conditions by detecting specific types of road damages in order to plan maintenance resources efficiently based on the identified spots. This paper describes a road damage type detection and classification solution submitted to the IEEE BigData Cup Challenge 2018. Our solution is based on the state-of-the-art deep learning methods for an object detection task. In particular, our approach utilizes an object detection algorithm to detect various types of road damages by training the detector on different image examples categorized into a set of damages defined by Japan Road Association. We evaluated our approach thoroughly using different versions of trained models. Our experiments show that our approach was able to achieve an F1 score up to 0.62.

Sentiment analysis of disaster-related posts in social media can contribute to the situation awareness and better understanding of the dynamics of disaster events by identifying the polarity of sentiments expressed by the public. However, Even though many sentiment analysis techniques have been developed and available, there are still limitations in reliably using sentiment analysis since there is no dominantly accepted technique in disasters. Taking advantage of existing state-of-the-art sentiment classifiers, this paper proposes a novel framework for geo-spatial sentiment analysis of disaster-related social media data objects. Our framework addresses three types of challenges: the inaccuracy and discrepancy associated with various text and image sentiment classifiers, the geo-sentiment discrepancy among data objects in a local geographical area, and observing diverse sentiments from multimedia data objects (i.e., text and image). The extracted sentiments are aggregated geographically for the purpose of extracting more accurate local regional insights. For the evaluation of the framework, we explored Twitter and Flickr datasets at the time of Hurricane Sandy and Napa Earthquake and showed how our approach can provide a better understanding of disaster events.

Efficient and thorough data collection and its timely analysis are critical for disaster response and recovery in order to save peoples lives during disasters. However, access to comprehensive data in disaster areas and their quick analysis to transform the data to actionable knowledge are challenging. With the popularity and pervasiveness of mobile devices, crowdsourcing data collection and analysis has emerged as an effective and scalable solution. This paper addresses the problem of crowdsourcing mobile videos for disasters by identifying two unique challenges of 1) prioritizing visualdata collection and transmission under bandwidth scarcity caused by damaged communication networks and 2) analyzing the acquired data in a timely manner. We introduce a new crowdsourcing framework for acquiring and analyzing the mobile videos utilizing fine granularity spatial metadata of videos for a rapidly changing disaster situation. We also develop an analytical model to quantify the visual awareness of a video based on its metadata and propose the visual awareness maximization problem for acquiring the most relevant data under bandwidth constraints. The collected videos are evenly distributed to off-site analysts to collectively minimize crowdsourcing efforts for analysis. Our simulation results demonstrate the effectiveness and feasibility of the proposed framework.

An unprecedented number of user-generated videos (UGVs) are currently being collected by mobile devices, however, such unstructured data are very hard to index and search. Due to recent development, UGVs can be geo-tagged, e.g., GPS locations and compass directions, at the acquisition time at a very fine spatial granularity. Ideally, each video frame can be tagged by the spatial extent of its coverage area, termed Field-Of-View (FOV). In this paper, we focus on the challenges of spatial indexing and querying of FOVs in a large repository. Since FOVs contain both location and orientation information, and their distribution is non-uniform, conventional spatial indexes (e.g., R-tree, Grid) cannot index them efficiently. We propose a class of new R-tree-based index structures that effectively harness FOVs' camera locations, orientations and view-distances, in tandem, for both filtering and optimization. In addition, we present novel search strategies and algorithms for efficient range and directional queries on FOVs utilizing our indexes. Our experiments with a real-world dataset and a large synthetic video dataset (over 30 years worth of videos) demonstrate the scalability and efficiency of our proposed indexes and search algorithms and their superiority over the competitors.

GIS applications now increasingly make use of geo-located multimedia data such as images and videos. Furthermore, the wide-spread availablity of smartphones allows the acquisition of user-generated videos that are annotated with geo-properties. The sensor meta-data, e.g., GPS and digital compass values, are considerably smaller in size than the visual content and are helpful in effectively and efficiently manage and search through large repositories of videos. However, a major practical issue is the noisy nature of such sensor data. For example, due to sensor data inaccuracies the visual coverage described by the meta-data may not exactly match the actual video scene, which leads to imprecise search results and positional disagreements on map overlays. Obstructions between the camera and its captured objects make these situations worse. Therefore, robust error-tolerance is an essential feature of any geo-tagged video search application.

Recently various sensors, such as GPS and compass devices, can be cost-effectively manufactured and this allows their deployment in conjunction with mobile video cameras. Hence, recorded clips can automatically be annotated with geospatial information and the resulting georeferenced videos may be used in various Geographic Information System (GIS) applications. However, the research community is lacking large-scale and realistic test datasets of such sensor-fused information to evaluate their techniques since collecting real-world test data requires considerable time and effort. To fill this void, we propose an approach for generating synthetic video meta-data with realistic geospatial properties for mobile video management research. We highlight the essential aspects of the georeferenced video meta-data and present an approach to simulate the behavioral patterns of mobile cameras in the synthetic data. The data generation process can be customized through user parameters for a variety of GIS applications that use mobile videos. We demonstrate the feasibility and applicability of the proposed approach by providing comparisons with real-world data.

Many Geographic Information Systems (GIS) handle large geospatial datasets stored in raster representation. Spatial joins over raster data are important queries in GIS for data analysis and decision support. However, evaluating spatial joins can be very time intensive due to the size of these datasets. In this paper we propose a new interactive framework that allows users to get approximate answers in near instantaneous time, thus allowing for truly interactive data exploration. Our method utilizes two proposed statistical approaches: probabilistic join and sampling based join. Our probabilistic join method provides speedup of two orders of magnitude with no correctness guarantee, while our sampling based method provides an order of magnitude improvement over the full quad-tree join and also provides running confidence intervals. We propose a framework that combines the two approaches to allow end users to tradeoff speed versus bounded accuracy. The two approaches are evaluated empirically with real and synthetic datasets.

A number of recent technological trends have made data intensive applications such as continuous media (audio and video) servers a reality. These servers are expected to play an important role in applications such as video-on-demand, digital library, news-on-demand, distance learning, etc. Continuous media applications are data intensive and might require storage subsystems that consist of hundreds of (multi-zone) disk drives. With the current technological trends, a homogeneous disk subsystem might evolve to consist of a heterogeneous collection of disk drives. Given such a storage subsystem, the system must continue to support a hiccup-free display of audio and video clips. This study describes extensions of four continuous display techniques for multi-zone disk drives to a heterogeneous platform. These techniques include IBM's Logical Track [21], HP's Track Pairing [4], and USC's FIXB [9] and deadline driven techniques [10]. We quantify the performance tradeoff associated with these techniques using analytical models and simulation studies. The obtained results demonstrate tradeoffs between the cost per simultaneous stream supported by a technique, the wasted disk space, and the incurred startup latency.