This paper proposes a new method, that we call VisualBackProp, for visualizing which sets of pixels of the input image contribute most to the predictions made by the convolutional neural network (CNN). The method heavily hinges on exploring the intuition that the feature maps contain less and less irrelevant information to the prediction decision when moving deeper into the network. The technique we propose is dedicated for CNN-based systems for steering self-driving cars and is therefore required to run in real-time. This makes the proposed visualization method a valuable debugging tool which can be easily used during both training and inference. We justify our approach with theoretical arguments and confirm that the proposed method identifies sets of input pixels, rather than individual pixels, that collaboratively contribute to the prediction. We utilize the proposed visualization tool in the NVIDIA neural-network-based end-to-end learning system for autonomous driving, known as PilotNet. We demonstrate that VisualBackProp determines which elements in the road image most influence PilotNet's steering decision and indeed captures relevant objects on the road. The empirical evaluation furthermore shows the plausibility of the proposed approach on public road video data as well as in other applications and reveals that it compares favorably to the layer-wise relevance propagation approach, i.e. it obtains similar visualization results and achieves order of magnitude speed-ups.

The number of small wireless devices is rapidly increasing, making the radio channel efficiency in limited geographic areas (individual rooms or buildings) an important metric for MAC protocols. Many of these emerging devices have use-cases that are difficult to satisfy with current hardware solutions and channel access methods; for instance device mobility, small energy reserves, and requirements for low cost and small form factors. However, for most of these applications, such as health care monitoring or sensing, feedback to the radio device is unnecessary and unidirectional communication techniques are not only sufficient, but can also be advantageous. We propose an efficient, reliable technique for unidirectional communication, called Transmit Only (TO), that satisfies these requirements while maintaining packet throughput guarantees and reducing energy consumption. In this paper we will demonstrate the feasibility and performance of this kind of highly asymmetric, transmit-only protocol through theoretical, simulated, and experimental results.

Radio frequency based device-free localization has been proposed as an alternative localization technique. Unlike its active localization counterpart, it does not require subjects to wear any radio device, but tries to determine the subject's location by observing how much the subject disturbs the radio propagation patterns. This problem is very challenging due to the well known multipath effect, especially in a complex indoor environment where it is impractical to accurately model the effects of a subject on the surrounding radio links. In this article, we formulate the device-free localization problem using probabilistic classification approaches that are based on discriminant analysis.To boost the localization accuracies, we adopt methods to mitigate errors caused by the multipath effect, as well as methods to automatically recalibrate training data so that accuracy can be maintained as the environment evolves. We validate our method in a one-bedroom apartment that consists of 32 cells, using eight fixed transmitters and eight fixed receivers. When the space has a single occupant, our method can correctly estimate the occupied cell with a likelihood as high as 97.2 percent. Further, we show that we can maintain a high localization accuracy, while substantially reducing the deployment overhead, which is an important concern for device-free localization methods. To achieve this goal, we have improved our training and testing procedures to reduce the overhead, studied the radio device placement to optimize the device cost, devised algorithms to extend the lifetime of the training data, and designed a set of auxiliary sensors and incorporate them into the system to achieve automatic re-calibration.

Today, people have the opportunity to opt-in to usage-based automotive insurances for reduced premiums by allowing companies to monitor their driving behavior. Several companies claim to measure only speed data to preserve privacy. With our elastic pathing algorithm, we show that drivers can be tracked by merely collecting their speed data and knowing their home location, which insurance companies do, with an accuracy that constitutes privacy intrusion. To demonstrate the algorithm's real-world applicability, we evaluated its performance with datasets from central New Jersey and Seattle, Washington, representing suburban and urban areas. Our algorithm predicted destinations with error within 250 meters for 14% traces and within 500 meters for 24% traces in the New Jersey dataset (254 traces). For the Seattle dataset (691 traces), we similarly predicted destinations with error within 250 and 500 meters for 13% and 26% of the traces respectively. Our work shows that these insurance schemes enable a substantial breach of privacy.

Smartphones are excellent mobile sensing platforms, with the microphone in particular being exercised in several audio inference applications. We take smartphone audio inference a step further and demonstrate for the first time that it's possible to accurately estimate the number of people talking in a certain place -- with an average error distance of 1.5 speakers -- through unsupervised machine learning analysis on audio segments captured by the smartphones. Inference occurs transparently to the user and no human intervention is needed to derive the classification model. Our results are based on the design, implementation, and evaluation of a system called Crowd++, involving 120 participants in 10 very different environments. We show that no dedicated external hardware or cumbersome supervised learning approaches are needed but only off-the-shelf smartphones used in a transparent manner. We believe our findings have profound implications in many research fields, including social sensing and personal wellbeing assessment.

Radio frequency based device-free passive (DfP) localization techniques have shown great potentials in localizing individual human subjects, without requiring them to carry any radio devices. In this study, we extend the DfP technique to count and localize multiple subjects in indoor environments. To address the impact of multipath on indoor radio signals, we adopt a fingerprinting based approach to infer subject locations from observed signal strengths through profiling the environment. When multiple subjects are present, our objective is to use the profiling data collected by a single subject to count and localize multiple subjects without any extra effort. In order to address the non-linearity of the impact of multiple subjects, we propose a successive cancellation based algorithm to iteratively determine the number of subjects. We model indoor human trajectories as a state transition process, exploit indoor human mobility constraints and integrate all information into a conditional random field (CRF) to simultaneously localize multiple subjects. As a result, we call the proposed algorithm SCPL -- sequential counting, parallel localizing. We test SCPL with two different indoor settings, one with size 150 m2 and the other 400 m2. In each setting, we have four different subjects, walking around in the deployed areas, sometimes with overlapping trajectories. Through extensive experimental results, we show that SCPL can count the present subjects with 86% counting percentage when their trajectories are not completely overlapping. Our localization algorithms are also highly accurate, with an average localization error distance of 1.3 m.

Device-free passive (DfP) localization is proposed to localize human subjects indoors by observing how the subject disturbs the pattern of the radio signals without having the subject wear a tag. In our previous work, we have proposed a probabilistic classification based DfP technique, which we call PC-DfP in short, and demonstrated that PC-DfP can classify which cell (32 cells in total) is occupied by the stationary subject with an accuracy as high as 97.2% in a one-bedroom apartment. In this poster, we focus on extending PC-DfP to track a mobile subject in indoor environments by taking into consideration that a human subject's locations should form a continuous trajectory. Through experiments in a 10 × 15 meters open plan office, we show that we can achieve better accuracies by exploiting the property of continuous mobility trajectories.

Radio frequency based device-free passive localization has been proposed as an alternative to indoor localization because it does not require subjects to wear a radio device. This technique observes how people disturb the pattern of radio waves in an indoor space and derives their positions accordingly. The well-known multipath effect makes this problem very challenging, because in a complex environment it is impractical to have enough knowledge to be able to accurately model the effects of a subject on the surrounding radio links. In addition, even minor changes in the environment over time change radio propagation sufficiently to invalidate the datasets needed by simple fingerprint-based methods. In this paper, we develop a fingerprinting-based method using probabilistic classification approaches based on discriminant analysis. We also devise ways to mitigate the error caused by multipath effect in data collection, further boosting the classification likelihood.

Device-free passive localization (DfP) techniques can localize human subjects without wearing a radio tag. Being convenient and private, DfP can find many applications in ubiquitous/pervasive computing. Unfortunately, DfP techniques need frequent manual recalibration of the radio signal values, which can be cumbersome and costly. We present SenCam, a sensor-camera collaboration solution that conducts automatic recalibration by leveraging existing surveillance camera(s). When the camera detects a subject, it can periodically trigger recalibration and update the radio signal data accordingly. This technique requires camera access occasionally each month, minimizing computational costs and reducing privacy concerns when compared to localization techniques solely based on cameras. Through experiments in an open indoor space, we show that this scheme can retain good localization results while avoiding manual recalibration.

In contrast to traditional sensor networks, the "Internet of Things" focuses on interactions between humans and physical objects rather than on sensing and reporting low level information. While several middle-ware systems have been created to simplify the task of managing and aggregating data from multiple sensor networks that use different hardware and software, management of the data is not sufficient to build an Internet of Things.

In this paper, we present the design, implementation and evaluation of a RF-based device-free passive localization strategy using active RFID nodes. Patterns of the measured power on multiple radio links are used to determine the location of a person in a room in a home environment. We develop an adaptive algorithm and training technique to minimize multi-path effects. With experimental deployment in a 5 x 8 meters room, we demonstrate that our system can successfully localize an individual to a 30-inch grid square with an 97.2% accuracy and 0.36 meters average error distance.

A typical wireless sensor network consists of many small sensors that collect instrument data around their locations and forward it to a central location for data processing. These networks can be deployed to monitor livestock and agricultural assets, products in a store, patients in a hospital, and so on. In many cases sensors have to be densely deployed, and collisions or overhead due to collision avoidance will considerably degrade the system performance below an application's required levels. With the decreasing cost of radio devices the obvious solution to this problem is the use of multiple receivers on different radio channels. However, we show that if receivers can be placed in different locations then increasing the number of receivers on a single channel will increase the rate of the capture effect and decrease collision losses, while also increasing the fairness of the transmitters' radio links. Not only can this single channel approach be more effective than using multiple channels, it is also required for some techniques, such as localization, where each receiver must be able to detect a transmission from any transmitter. We also show that the optimal choice between these two solutions is influenced by the radio attenuation rate and the number of receivers in the system.

We explore the problem of detecting whether a device has moved within a room. Our approach relies on comparing summaries of received signal strength measurements over time, which we call descriptors. We consider descriptors based on the differences in the mean, standard deviation, and histogram comparison. In close to 1000 mobility events we conducted, our approach delivers perfect recall and near perfect precision for detecting mobility at a granularity of a few seconds. It is robust to the movement of dummy objects near the transmitter as well as people moving within the room. The detection is successful because true mobility causes fast fading, while environmental mobility causes shadow fading, which exhibit considerable difference in signal distributions. The ability to produce good detection accuracy throughout the experiments also demonstrates that our approach can be applied to varying room environments and radio technologies, thus enabling novel security, health care, and inventory control applications.

Asset tracking is an important application domain for wireless sensor networks. However, continuous tracking of a large number of items at the individual item level over a significant period of time is still not feasible. There are two main obstacles. The first is the need for efficient, low-power communication protocols. Many current protocols employ energy-expensive methods to achieve reliable communication for arbitrary traffic situations. Such protocols are not suitable for continuous asset tracking applications. The second challenge is the lack of a robust presence detection algorithm that can differentiate packet losses caused by a missing item from packet losses caused by the ambient radio environment. In this paper, we designed a simple communication protocol, Uni-HB, and demonstrated it can lead to longer system lifetime and higher communication reliability than several popular protocols. We also devised two robust detection algorithms that can yield low false alarm rates while achieving timely loss notification. We took an experimental approach, and evaluated protocols on a generic embedded hardware platform that has an similar architecture to motes. We also derived analytical models to validate our experimental measurements.