Deep learning Processor Unit (DPU) is a highly configurable CNN accelerator that supports a variety of CNNs and can be implemented with multiple instances on the same FPGA. Many applications deploy concurrent execution of different CNNs and in such a setting, an execution time predictor can help “optimize” the DPU configurations to meet the performance requirements of different tasks. We characterize CNN execution on DPUs and reduce the variability in execution time due to interference from the operating system. Subsequently, we propose a machine learning-based framework (EXPRESS) to predict the execution time of any given CNN on a DPU configuration, considering CNN, DPU, and bus characteristics. We improvise EXPRESS to support heterogeneous CNNs in EXPRESS-2.0 by making features independent of the number of CNNs. Our entire experimentation is based on data from a real FPGA board for 16 standard CNNs. Our frameworks, EXPRESS and EXPRESS-2.0, significantly outperform state-of-the-art by achieving an average execution time prediction error of 2.2% and 0.7%, respectively. We illustrate the effectiveness of this low prediction error for design space exploration, which is very useful for embedded system application developers.

IoT devices (e.g., voice assistants) that execute real-time speech commands are proliferating fast in our daily lives. In such a device, detecting the correct keyword spoken as a command triggers the supported function, and hence keyword spotting (KWS) using a machine learning (ML) model is the pivotal task in their functioning. However, KWS is vulnerable to adversarial machine learning (AML)-based attacks through which an adversary can craft an adversarial audio sample that sounds like a benign keyword to a human, but is detected as a different keyword by the KWS pipeline. In this paper, we propose SpotOn, a novel KWS pipeline that both recovers from AML attacks, as well as detects whether an attacker is using the device to generate AML noise. Using the Google speech command dataset, we demonstrate that SpotOn provides reasonable accuracy in correctly detecting keywords in the absence or presence of AML attacks. Through careful optimizations, we enable SpotOn to process streaming speech input on resource-constrained IoT devices. Overall, the design of SpotOn provides critical insights into making voice-controlled IoT devices suitable for safety-critical systems.

Effective traffic intersection control is crucial for urban sustainability. State of the art research seeking Artificial Intelligence (AI), for example Deep Reinforcement Learning (DRL) based traffic control requires environment states through various Computer Vision methods, where the collective state of multiple cameras across an intersection constitute the single state for AI. This brings in serious robustness or fault-tolerance concerns on the deployed system. Camera systems are highly susceptible to faults due to multiple possible points of failure. A single fault collapses the AI state and hence the capacity of AI controller to manage the traffic is gone. Also, infrastructure deployment and maintenance is a slow bureaucratic process in the developing countries, which makes camera faults a regular event. In the given paper, we build WebLight (https://github.com/sachin-iitd/WebLight), a web based, independent and alternative, traffic state processing method which can replace the camera dependency completely, or support as a backup mechanism until the camera system is back online, making the AI intersection control robust to camera failures.

Rapid advancements in edge devices have led to a large deployment of deep neural network (DNN) based workloads. To utilize the resources at the edge effectively, many DNN compilers are proposed that efficiently map the high level DNN models developed in frameworks like PyTorch, Tensorflow, Caffe, and so on into minimum deployable lightweight execution engines. For real time applications like ADAS, these compiler optimized engines should give precise, reproducible, and predictable inferences, both in-terms of runtime and output consistency. This article is the first effort in empirically auditing state-of-the-art DNN compilers viz TensorRT, AutoTVM, and AutoScheduler. We characterize the NN compilers based on their performance predictability w.r.t inference latency, output reproducibility, hardware utilization, and so on and based on that provide various recommendations. Our methodology and findings can potentially help the application developers, in making informed decision about the choice of DNN compiler, in a real time safety critical setting.

Machine learning (ML) applications have experienced remarkable growth and integration into various domains. However, challenges with cloud-based deployments, such as latency, privacy, reliability, bandwidth and connectivity, have driven the popularity of deploying ML on edge devices. ML application deployment stack consists of various components such as neural network models, input frameworks, software runtime libraries and hardware architecture. Understanding the impact of different components in the ML stack on deployment effectiveness, particularly in terms of cost effectiveness, remains a challenge. In this work, we systematically analyze the diverse choices available for each component of the ML stack and their influence on deployment performance. We empirically evaluate eight heterogeneous edge platforms and eight software runtime libraries, considering various hardware components like CPUs, GPUs, NPUs, and VPUs for ML inference. Our findings contribute to a better understanding of optimizing cost effectiveness in ML deployments on edge platforms, aiding decision-making for application developers and stakeholders.

Air pollution poses serious health concerns in developing countries, such as India, necessitating large-scale measurement for correlation analysis, policy recommendations, and informed decision-making. However, fine-grained data collection is costly. Specifically, static sensors for pollution measurement cost several thousand dollars per unit, leading to inadequate deployment and coverage. To complement the existing sparse static sensor network, we propose a mobile sensor network utilizing lower-cost PM2.5 sensors mounted on public buses in the Delhi-NCR region of India. Through this exercise, we introduce a novel dataset AirDelhi comprising PM2.5 and PM10 measurements. This dataset is made publicly available, at https://www.cse.iitd.ac.in/pollutiondata, serving as a valuable resource for machine learning (ML) researchers and environmentalists. We present three key contributions with the release of this dataset. Firstly, through in-depth statistical analysis, we demonstrate that the released dataset significantly differs from existing pollution datasets, highlighting its uniqueness and potential for new insights. Secondly, the dataset quality been validated against existing expensive sensors. Thirdly, we conduct a benchmarking exercise (https://github.com/sachin-iitd/DelhiPMDatasetBenchmark), evaluating state-of-the-art methods for interpolation, feature imputation, and forecasting on this dataset, which is the largest publicly available PM dataset to date. The results of the benchmarking exercise underscore the substantial disparities in accuracy between the proposed dataset and other publicly available datasets. This finding highlights the complexity and richness of our dataset, emphasizing its value for advancing research in the field of air pollution.

Privacy-preserving machine learning (PPML) promises to train machine learning (ML) models by combining data spread across multiple data silos. Theoretically, secure multiparty computation (MPC) allows multiple data owners to train models on their joint data without revealing the data to each other. However, the prior implementations of this secure training using MPC have three limitations: they have only been evaluated on CNNs, and LSTMs have been ignored; fixed point approximations have affected training accuracies compared to training in floating point; and due to significant latency overheads of secure training via MPC, its relevance for practical tasks with streaming data remains unclear. The motivation of this work is to report our experience of addressing the practical problem of secure training and inference of models for urban sensing problems, e.g., traffic congestion estimation, or air pollution monitoring in large cities, where data can be contributed by rival fleet companies while balancing the privacy-accuracy trade-offs using MPC-based techniques.Our first contribution is to design a custom ML model for this task that can be efficiently trained with MPC within a desirable latency. In particular, we design a GCN-LSTM and securely train it on time-series sensor data for accurate forecasting, within 7 minutes per epoch. As our second contribution, we build an end-to-end system of private training and inference that provably matches the training accuracy of cleartext ML training. This work is the first to securely train a model with LSTM cells. Third, this trained model is kept secret-shared between the fleet companies and allows clients to make sensitive queries to this model while carefully handling potentially invalid queries. Our custom protocols allow clients to query predictions from privately trained models in milliseconds, all the while maintaining accuracy and cryptographic security.

Developing countries are home to the most polluted cities in the world. Particulate Matter (PM), one of the most serious air pollutants, needs to be measured at scale across urban areas in such countries. Factors potentially affecting PM like road traffic, green cover, industrial emissions etc., also need to be quantified, to enable fine-grained correlation analyses among PM and its causes. This paper presents an IoT platform with multiple sensors, latest deep neural network based edge-computing, local storage and communication support – to measure PM and its associated factors. Through real world deployments, the first in depth empirical analysis of a government enforced traffic control policy for pollution control, is presented as a use case of our IoT platform. We demonstrate the potential of IoT and edge computing in urban sustainability questions in this paper, especially in a developing region context. At the same time, we show how complex a real system like Particulate Matter’s factor analyses can be, and urge environmentalists to use sensors networks and fine-grained empirical datasets as ours in future, for more nuanced and data-driven policy discussions.

Road traffic congestion increases vehicular emissions and air pollution. Traffic rule violation causes road accidents. Both pollution and accidents take tremendous social and economic toll worldwide, and more so in developing countries where the skewed vehicle to road infrastructure ratio amplifies the problems. Automating traffic intersection management to detect and penalize traffic rule violations and reduce traffic congestion, is the focus of this paper, using state-of-the-art Convolutional Neural Network (CNN) on traffic camera feeds. There are however non-trivial challenges in handling the chaotic, non-laned traffic scenes in developing countries. Maintaining high throughput is one of the challenges, as broadband connectivity to remote GPU servers is absent in developing countries, and embedded GPU platforms on roads need to be low cost due to budget constraints. Additionally, ambient temperatures in developing country cities can go to 45-50 degree Celsius in summer, where continuous embedded processing can lead to lower lifetimes of the embedded platforms. In this paper, we present DynCNN, an application dynamism and ambient temperature aware controller for Neural Network concurrency. DynCNN effectively uses processor heterogeneity to control the number of threads and frequencies on the accelerator to manage application utility under strict thermal and power thresholds. We evaluate the efficiency of DynCNN on three different commercially available embedded GPUs (Jetson TX2TM, Xavier NXTM and Xavier AGXTM) using a real traffic intersection’s 40 days’ dataset. Experimental results show that in comparison to all existing state-of-the art- GPU governors for two different CPU settings, DynCNN reduces the average temperature and power by ~12°C and 68.82% respectively for one CPU setting (Baseline1) and similarly, it improves the performance by around 31.2% compared to the other CPU setting (Baseline2).

Deep learning Processor Units (DPUs) from Xilinx are design-time configurable CNN accelerators for FPGAs. We propose EXPRESS, which predicts the execution time of any given CNN on a DPU. EXPRESS incorporates the effect of bus connections into prediction. As a DPU is invoked by a host CPU to process a CNN layer by layer, EXPRESS considers the CPU and the DPU execution time for predicting the end-to-end processing time. EXPRESS has an average prediction error of 2.2% and significantly outperforms state-of-the-art.

Machine Learning (EdgeML) algorithms on edge devices facilitate safety-critical applications like building security management and smart city interventions. However, their wired/wireless connections with the Internet make such platforms vulnerable to attacks compromising the embedded software. We find that in the prior works, the issue of regular runtime integrity assessment of the deployed software with negligible EdgeML performance degradation is still unresolved. In this paper, we present PracAttest, a practical runtime attestation framework for embedded devices running compute-heavy EdgeML applications. Unlike the conventional remote attestation schemes that check the entire software in each attestation event, PracAttest segments the software and randomizes the integrity check of these segments over short random attestation intervals. The segmentation coupled with the randomization leads to a novel performance-vs-security trade-off that can be tuned per the EdgeML application’s performance requirements. Additionally, we implement three realistic EdgeML benchmarks for pollution measurement, traffic intersection control, and face identification, using state-of-the-art neural network and computer vision algorithms. We specify and verify security properties for these benchmarks and evaluate the efficacy of PracAttest in attesting the verified software. PracAttest provides 50x-80x speedup over the state-of-the-art baseline in terms of mean attestation time, with negligible impact on application performance. We believe that the novel performance-vs-security trade-off facilitated by PracAttest will expedite the adoption of runtime attestation on edge platforms.

Convolutional Neural Networks (CNNs) are increasingly becoming popular in embedded and energy limited mobile applications. Hardware designers have proposed various accelerators to speed up the execution of CNNs on embedded platforms. Deep Learning Processor Unit (DPU) is one such generic CNN accelerator for Xilinx platforms that can execute any CNN on one or more DPUs configured on an FPGA. In a period of rapid growth in CNN algorithms and the availability of multiple configurations of CNN accelerators (like DPU), the design space is expanding fast. These design points show significant trade-off in execution time, energy consumption and application performance measured in terms of accuracy. To be able to perform this trade-off, we propose a methodology for energy estimation of a CNN running on a DPU. We build an energy model using characteristics of few CNNs and use this model for energy prediction of other unseen CNNs. We evaluate our approach using 16 different standard and popular CNNs with an average prediction error of 9.9%. Energy estimation can be useful in various scheduling applications where one can choose from multiple CNNs based on its energy consumption. We demonstrate the utility of our approach in a drone that is deployed for detecting objects on the ground.

Effective intersection control can play an important role in reducing traffic congestion and associated vehicular emissions. This is vitally needed in developing countries, where air pollution is reaching life threatening levels. This paper presents EcoLight intersection control for developing regions, where budget is constrained and network connectivity is very poor. EcoLight learns effective control offline using state-of-the-art Deep Reinforcement Learning methods, but deploys highly efficient runtime control algorithms on low cost embedded devices that work stand-alone on road without server connectivity. EcoLight optimizes both average case and worst case values of throughput, travel time and other metrics, as evaluated on open-source datasets from New York and on a custom developing region dataset.

Classifying and counting vehicles in road traffic has numerous applications in the transportation engineering domain. However, the wide variety of vehicles (two-wheelers, three-wheelers, cars, buses, trucks etc.) plying on roads of developing regions without any lane discipline, makes vehicle classification and counting a hard problem to automate. In this paper, we use state of the art Convolutional Neural Network (CNN) based object detection models and train them for multiple vehicle classes using data from Delhi roads. We get upto 75% MAP on an 80-20 train-test split using 5562 video frames from four different locations. As robust network connectivity is scarce in developing regions for continuous video transmissions from the road to cloud servers, we also evaluate the latency, energy and hardware cost of embedded implementations of our CNN model based inferences.

This poster describes a low-cost and robust embedded platform, designed for vehicle mounted sensing of particulate matter (PM2.5 and PM10). The prototype is specifically designed to be mounted on the Delhi Integrated Multi-Modal Transit System (DIMTS) buses. Movement of the buses adds noise to pollution data. Error in GPS measurement causes issues in detecting moving vs. stationary state of the buses, useful to filter out noisy pollution data collected in the moving state. Intermittent cellular network connectivity causes frequent disconnects with the remote server. Our prototype is designed to handle such real world deployment challenges. Pilot deployment with this platform is currently ongoing. Preliminary data analysis from the pilot deployment will be discussed as part of the poster presentation, along with demonstration of the prototype sensor platform. This hardware prototype has the potential of creating locality wise, dense air pollution data providing crucial insights into the causes of air pollution.

Reliable on-off control of peripherals on smart devices is a key to security and privacy in many scenarios. Journalists want to reliably turn off radios to protect their sources during investigative reporting. Users wish to ensure cameras and microphones are reliably off during private meetings. In this paper, we present SeCloak, an ARM TrustZone-based solution that ensures reliable on-off control of peripherals even when the platform software is compromised. We design a secure kernel that co-exists with software running on mobile devices (e.g., Android and Linux) without requiring any code modifications. An Android prototype demonstrates that mobile peripherals like radios, cameras, and microphones can be controlled reliably with a very small trusted computing base and with minimal performance overhead.

In this paper we analyze Facebook's Free Basics program, which provides free Internet access to a restricted set of web services. As the program grows to 60+ developing countries, an independent and data-driven audit of its scope and outreach is highly relevant to the ICTD community.

The ubiquity of portable mobile devices equipped with built-in cameras have led to a transformation in how and when digital images are captured, shared, and archived. Photographs and videos from social gatherings, public events, and even crime scenes are commonplace online. While the spontaneity afforded by these devices have led to new personal and creative outlets, privacy concerns of bystanders (and indeed, in some cases, unwilling subjects) have remained largely unaddressed. We present I-Pic, a trusted software platform that integrates digital capture with user-defined privacy. In I-Pic, users choose alevel of privacy (e.g., image capture allowed or not) based upon social context (e.g., out in public vs. with friends vs. at workplace). Privacy choices of nearby users are advertised via short-range radio, and I-Pic-compliant capture platforms generate edited media to conform to privacy choices of image subjects. I-Pic uses secure multiparty computation to ensure that users' visual features and privacy choices are not revealed publicly, regardless of whether they are the subjects of an image capture. Just as importantly, I-Pic preserves the ease-of-use and spontaneous nature of capture and sharing between trusted users. Our evaluation of I-Pic shows that a practical, energy-efficient system that conforms to the privacy choices of many users within a scene can be built and deployed using current hardware.

Developing region road traffic has a unique characteristic of high heterogeneity in vehicle types. In this paper, we describe VividhaVahana, a smartphone sensor based system to categorize road vehicles into four predominant categories: two-wheeler bikes, three-wheeler auto-rickshaws, four-wheeler car

Detecting the group context of an individual (i.e., whether an individual is alone or part of a group) in crowded public spaces, such as shopping malls, is an important goal with many practical applications. However, in crowded indoor spaces, understanding the group-dependent movement behavior is a non-trivial problem as: (1) detecting groups is hard as the density ensures that at any location, a large number of people are moving together, (2) location tracking in many real-world venues is either absent or not very accurate, and (3) indoor mobility models that take into account group attributes (such as group size) are rare. In this paper, we first introduce GruMon, a platform for near real-time group monitoring in dense, public spaces, and then demonstrate how the movement & residency properties of individuals are significantly affected when they are in groups.

Real-time monitoring of groups and their rich contexts will be a key building block for futuristic, group-aware mobile services. In this paper, we propose GruMon, a fast and accurate group monitoring system for dense and complex urban spaces. GruMon meets the performance criteria of precise group detection at low latencies by overcoming two critical challenges of practical urban spaces, namely (a) the high density of crowds, and (b) the imprecise location information available indoors. Using a host of novel features extracted from commodity smartphone sensors, GruMon can detect over 80% of the groups, with 97% precision, using 10 minutes latency windows, even in venues with limited or no location information. Moreover, in venues where location information is available, GruMon improves the detection latency by up to 20% using semantic information and additional sensors to complement traditional spatio-temporal clustering approaches. We evaluated GruMon on data collected from 258 shopping episodes from 154 real participants, in two large shopping complexes in Korea and Singapore. We also tested GruMon on a large-scale dataset from an international airport (containing ≈37K+ unlabelled location traces per day) and a live deployment at our university, and showed both GruMon's potential performance at scale and various scalability challenges for real-world dense environment deployments.

An unprecedented rate of growth in the number of vehicles has resulted in acute road congestion problems worldwide, especially in many developing countries. In this article, we present Road-RFSense, a practical RF sensing--based road traffic estimation system for developing regions. Our first contribution is a new mechanism to sense road occupancy, based on variation in RF link characteristics, when line of sight between a transmitter-receiver pair is obstructed. We design algorithms to classify traffic states into two classes, free-flow versus congested, at timescales of 20 seconds with greater than 90% accuracy. We also present a traffic queue length measurement system, where a network of RF sensors can correlate the traffic state classification decisions of individual sensors and detect traffic queue length in real time. Deployment of our system on a Mumbai road gives correct estimates, validated against 9 hours of image-based ground truth. Our third contribution is a large-scale data-driven study, in collaboration with city traffic authorities, to answer questions regarding road-specific classification model training. Finally, we explore multilevel classification into seven different traffic states using a larger set of RF-based features and careful choice of classification algorithms.

Radio Frequency (RF) fingerprinting, based onWiFi or cellular signals, has been a popular approach to indoor localization. However, its adoption in the real world has been stymied by the need for sitespecific calibration, i.e., the creation of a training data set comprising WiFi measurements at known locations in the space of interest. While efforts have been made to reduce this calibration effort using modeling, the need for measurements from known locations still remains a bottleneck. In this paper, we present Zee -- a system that makes the calibration zero-effort, by enabling training data to be crowdsourced without any explicit effort on the part of users. Zee leverages the inertial sensors (e.g., accelerometer, compass, gyroscope) present in the mobile devices such as smartphones carried by users, to track them as they traverse an indoor environment, while simultaneously performing WiFi scans. Zee is designed to run in the background on a device without requiring any explicit user participation. The only site-specific input that Zee depends on is a map showing the pathways (e.g., hallways) and barriers (e.g., walls). A significant challenge that Zee surmounts is to track users without any a priori, user-specific knowledge such as the user's initial location, stride-length, or phone placement. Zee employs a suite of novel techniques to infer location over time: (a) placement-independent step counting and orientation estimation, (b) augmented particle filtering to simultaneously estimate location and user-specific walk characteristics such as the stride length,(c) back propagation to go back and improve the accuracy of ocalization in the past, and (d) WiFi-based particle initialization to enable faster convergence. We present an evaluation of Zee in a large office building.

Unprecedented rate of growth in the number of vehicles has resulted in acute road congestion problems worldwide. Better traffic flow management, based on enhanced traffic monitoring, is being tried by city authorities. In many developing countries, the situation is worse because of greater skew in growth of traffic vs the road infrastructure. Further, the existing traffic monitoring techniques perform poorly in the chaotic non-lane based traffic here.

Road congestion is a common problem all over the world. In many developed countries, automated congestion detection techniques have been deployed, that are used in road travel assisting applications. But these techniques are mostly inapplicable in many developing regions due to high cost and their assumptions of orderly traffic. Efforts in developing regions have been few. In this paper, we present RoadSoundSense, an acoustic sensing based technique, for near real time congestion monitoring on chaotic roads, at a moderate cost. We present the detailed design of an acoustic sensing hardware prototype, which has to be deployed by the side of the road to be monitored. This unit samples and processes road noise to compute various metrics like amount of vehicular honks and vehicle speed distribution, with speeds calculated from honks using differential Doppler shift. The metrics are sent to a remote server over GPRS every alternate minute. Based on the metric values, the server can decide the traffic condition on the road. Data from deployment of this prototype in six different Mumbai roads, validated against manually observed ground truth, shows feasibility of per minute congestion monitoring from a remote server. K-means clustering gives on average 90% accuracy to group unlabeled data on a new road into two clusters of congested and free-flow. Deployment data from one road for six days shows the temporal variation in traffic state for that road. Though we test our technique in Mumbai, we believe that most of our claims and experimental results can be extended to city roads of other developing regions as well.

Road congestion is a common problem worldwide. Existing Intelligent Transport Systems (ITS) are mostly inapplicable in developing regions due to high cost and assumptions of orderly traffic. In this work, we develop a low-cost technique to estimate vehicular speed, based on vehicular honks. Honks are a characteristic feature of the chaotic road conditions common in many developing regions like India and South-East Asia.