Various computer vision techniques based on deep neural networks have been proposed to detect objects accurately and fast. However, due to the privacy, security and communication bandwidth restrictions of diverse participating parties, it is sometimes prohibitive to train such models on a centralized machine. Federated Learning (FL) provides a promising solution to learn a model from decentralized data. Despite the advances in FL, the diversity of client regions in which they operate and the Non-IID nature of the crowdsourced datasets reduces the accuracy of object detection models significantly. In this paper, we introduce a novel FL object detection system to efficiently train models with heterogeneous client datasets. We propose lightweight client selection methods to learn object detection models faster. Our client selection methods based on the object data distribution at clients achieves up to 74% reduction in required federated rounds compared to conventional approaches. We further extend this method by leveraging the metadata of the training images (e.g., location, direction, depth), to select clients which maximize the coverage of diverse geographical regions. We report on extensive experiments with real datasets.

The pervasive deployment of IoT devices along with the advancements in Deep Neural Network (DNN) models have enabled video analytics at the edge, the so-called Edge AI systems, in support of various large smart-city applications such as automatic road damage evaluation and fire detection. Current solutions require the model developer to make the placement decision by manually assigning models to edge devices. However, an Edge AI solution could entail hundreds of mobile edge devices operating in a large geographical region (e.g., installed on vehicles) with various resource capabilities and different DNN models, hence rendering manual placement ineffective. This paper presents alternative methods to automatically place various models on a diverse set of edge devices, considering the geospatial coverage of video data, resource capabilities of edge devices, and the characteristics of the trained models. First, we mathematically formulate the model placement as an optimization problem which is proven to be NP-Hard. We then propose several heuristics to solve it efficiently and evaluate them with a real-world dataset collected along the 165 bus route trajectories in the City of San Francisco. Our placement algorithm yields a higher recall in object detection and is more robust to the uncertainty of the underlying location context, without sacrificing much utilization cost.

With the availability of massive amounts of visual data covering wide geographical regions, various image learning applications have emerged, including classifying the street cleanliness level, detecting forest fires or road hazards. Such applications share similar characteristics as they need to 1) detect specific objects or events (what), 2) associate the detected object with a location (where), and 3) know the time that the event happened (when). Advancements in image-based machine learning (ML) benefit these applications as they can automate the detection of objects of interest. Along with the edge computing (EC) paradigm, the processing cost is offloaded to the devices, hence reducing latency and communication cost. Moreover, sensors on the edge devices (e.g., GPS) enrich the collected data with metadata. However, a shortcoming of existing approaches is that they rely on pre-trained "static" models. Nonetheless, crowdsourced data at diverse locations can be leveraged to iteratively improve the robustness of a model. We refer to the aforementioned strategy as "spatial crowd-based learning".To showcase this class of applications, we present FloraVision, an end-to-end system that integrates ML, crowdsourcing, and EC to automate the detection, mapping, and exploration of California Native Plants. FloraVision implements a pipeline to collect and clean publicly available image data, train a lightweight MobileNet-based classification model, and then deploy the model on mobile devices. It leverages spatial crowd-based learning to iteratively evolve the initial model from crowdsourced data. Its mobile application facilitates detecting plants and mapping their geolocations. Finally, it allows end-users to submit ad hoc spatio-temporal nearest neighbor queries and visualizes the results in an augmented reality user interface. Although our application focuses on plants, several other applications follow similar architectural patterns.

Generating hyper-local news at scale is challenging because publicly available data is not provided at the desired spatial and temporal granularity. Besides, there is a lack of automated analytical and publishing tools. Crosstown Foundry, which is being actively developed and used by engineers and journalists, is a novel data-driven system that leverages a massive multi-modal dataset to generate personalized newsletters for Los Angeles County readers.

Advances in federated learning and edge computing advocate for deep learning models to run at edge devices for video analysis. However, the captured video frame rate is too high to be processed at the edge in real-time with a typical model such as CNN. Any approach to consecutively feed frames to the model compromises both the quality (by missing important frames) and the efficiency (by processing redundantly similar frames) of analysis. Focusing on outdoor urban videos, we utilize the spatial metadata of frames to select an optimal subset of frames that maximizes the coverage area of the footage. The spatial keyframe extraction is formulated as an optimization problem, with the number of selected frames as the restriction and the maximized coverage as the objective. We prove this problem is NP-hard and devise various heuristics to solve it efficiently. Our approach is shown to yield much better hit-ratio than conventional ones.

Several applications in urban planning, ride-sharing or marketing, require access to the location popularity of a geographical area (e.g., city block, city, county) in near real-time and at different resolutions. To conceptualize such an access, imagine a visualization tool to view a heatmap of location popularity of a region on-the-fly as a user interacts seamlessly by zooming in and out. The access method required to enable such a seamless visualization must support: 1) updating the heatmap cells frequently as the raw data (e.g., check-ins) arrives at a high rate in a streaming fashion, and 2) splitting and merging the adjacent cells quickly to support zooming in and out, respectively. This is challenging because the most useful metric for location popularity, location entropy, requires counting the number of unique visits per user, and hence: 1) a large data structure should be maintained and updated per cell, and 2) the adjacent cells must be aggregated/disaggregated quickly while the unique visits are not additive. Due to these challenges, the previous techniques for OLAP cubes, streaming sketches and index structures are not effective. In this paper, we propose a new index structure called MR-Cube that approximates the popularity by maintaining sketches of streamed data per cell, supports time-decay for older visits and aggregates the non-additive location popularity quickly and accurately at different resolutions. We evaluate the accuracy and efficiency of MR-Cube using real-world and synthetic datasets and show its utility for our application.

In this demonstration paper, we present an indoor positioning system developed for Android smartphones, coined Airplace. To infer the unknown user location we rely on ubiquitous WLANs and exploit Received Signal Strength (RSS) values from neighboring Access Points (AP) that are constantly monitored by the mobile devices under normal operation. Our system follows a mobile-based network-assisted architecture to eliminate the communication overhead and respect user privacy. In a typical scenario, when a user walks inside a building a smartphone client conducts a single communication with our Distribution Server to receive the RSS radiomap and is then able to position itself independently using the observed RSS values. Moreover, we have implemented an Android application to facilitate the collection of RSS values by users that may contribute their data to our system for constructing and updating the radiomap through crowdsourcing1. We will demonstrate the real-time positioning capabilities of the system during the conference by allowing attendees to carry an Android tablet in order to view their position on a floorplan map, while walking around inside the demo area (interactive scenario). Moreover, we will illustrate how to evaluate the performance of different positioning algorithms using profiled data in a trace-driven scenario. Our objective is to highlight the effectiveness and applicability of our system and at the same time the participants will be able to appreciate the potential of indoor location-oriented services and applications.

In this demo paper, we present the Airplace indoor positioning platform developed for Android smartphones [1]. Airplace relies on existing WLAN infrastructure and exploits Received Signal Strength (RSS) values from neighboring Access Points (AP) to infer the unknown user location. Our system utilizes a number of RSS fingerprints collected a priori to build the so-called radiomap. Location is then estimated by finding the best match between the currently measured fingerprint and fingerprints in the radiomap [2].