Federated Learning (FL) enables collaborative machine learning across decentralized devices and data sources, but resource constraints on pervasive devices necessitate efficient model compression. Existing approaches, such as quantization for on-device training, often degrade accuracy, especially for classes that are difficult to learn due to imbalance, poor-quality samples, or inherent complexity. This results in persistent accuracy gaps across classes. We propose Fed-CASQ's a novel framework that couples class-aware strategies into the quantization process to jointly improve efficiency and accuracy in pervasive FL. Unlike prior works that address quantization and imbalance separately, Fed-CASQ adaptively selects quantization levels based on device resources and leverages Layer-wise Relevance Propagation (LRP) to assess class-relevant convolutional neural network (CNN) filters on the client side. An adaptive weight scaling mechanism is then applied to amplify critical information for low-accuracy classes before aggregation. At the server, a complementary novel aggregation strategy mitigates global imbalance across clients, ensuring that underperforming classes receive proportional attention during model updates. We theoretically establish that Fed-CASQ achieves a convergence rate of ${\mathcal{O}}\left({\frac{{\kappa *\hat \sigma *\hat \delta }}{{\sqrt T }}}\right)$ under non-convex settings. We empirically establish that quantization directly influences the performance of under sampled (minority) classes. Experimental results further show that Fed-CASQ substantially narrows the performance gap for low-accuracy classes, improving their accuracy by ≈30%, while reducing training latency by over 56% on resource-constrained pervasive devices.

The proliferation of camera-equipped systems in domestic environments, such as Augmented Reality (AR) applications, household robots, smart glasses, and indoor security cameras, raises critical privacy concerns. Beyond capturing users and visitors, these systems record home interiors that may contain objects and contextual cues inadvertently disclosing sensitive information, such as socioeconomic status cues, social or family structure, cultural or religious affiliation, and daily routines or habits. Yet, a systematic understanding of object-level privacy risks in this context remains limited. To address this gap, we conducted a user study with 210 participants, each annotating 18 images from a dataset of 279 home interior images, yielding 541 unique image–object sensitivity assessments. Based on these annotations, we introduce a multidimensional categorization scheme that integrates sensitivity scales, object categories (e.g., windows, bystanders), and information types (e.g., personal data, social circles). We further evaluate the capability of Vision–Language Models (VLMs) to infer sensitive information from images, including content that appears inconspicuous at first glance. Our analysis shows that VLMs correctly identify the objects’ sensitivity degree within images with up to 70%, and few-shot learning techniques further enhance the object sensitivity inference.

Heterogeneous sensors (e.g., 2D cameras and LiDAR) provide a novel opportunity to leverage multiple modalities in collaborative artificial intelligence (AI)-based video analytics pipelines. Such applications use sensors that are frequently attached to resource-limited edge devices which can hinder the execution of multimodal and deep DNN models. While powerful edge devices can still benefit from multimodal fusion to enhance robustness, joint training of such models for generalizable applications is often infeasible due to the lack of large-scale multimodal datasets and the prohibitive cost involved in annotating those datasets. To address this, we introduce FusionBridge: a lightweight fusion framework that combines the capabilities of independently trained 2D (image-based) and 3D (LiDAR-based) perception models to improve object detection at the edge. FusionBridge extracts mid-level features from single modality 3D models and performs cross-modal fusion via a lightweight transformer-based adapter. This enables hints to be exchanged without requiring joint end-to-end training. By bridging modality-specific experts, our approach maintains modularity, supports model reuse, and allows scalable deployment across heterogeneous sensor configurations with zero calibration or sensor alignment effort. Evaluations on simulated and real world deployments demonstrate that FusionBridge achieves up to a 57% F1-score improvement over any single-modality baseline, while only incurring a 15% latency overhead and 0.4KB/frame transmission overhead compared to the baseline.

Due to their limited ability to reason about the social context in which they are used, smart speakers pose significant privacy risks by responding in ways that may violate people's implicit social boundaries. We conducted a cross-cultural vignette study (N = 944) in Germany and Singapore to investigate how situational factors—specifically social context (bystander relationships and closeness), physical context (location), and interaction context (topic and deceptive intent)—regulate user preferences for smart speaker responses. Our results demonstrate that these factors are superior predictors of response preferences than dispositional user traits (i.e., intrinsic personal traits). We identify two distinct social dynamics: a structural influence for professional relationships (e.g., boss) that persists regardless of social closeness, and a closeness-based influence for personal relationships. We further demonstrate that deceptive intent drives privacy-seeking behaviour, acting as a tool for social impression management. In parallel, we evaluate how LLMs respond to the same scenarios, revealing mismatches between model behaviour and human expectations, particularly in culturally contingent situations. These mismatches expose new privacy risks arising from socially miscalibrated AI reasoning. We conclude with design strategies to better align device behaviour with these social nuances.

3-Dimensional Embodied Reference Understanding (3D-ERU) combines a language description and an accompanying pointing gesture to identify the most relevant target object in a 3D scene. Although prior work has explored pure language-based 3D grounding, there has been limited exploration of 3D-ERU, which also incorporates human pointing gestures. To address this gap, we introduce a data augmentation framework– Imputer, and use it to curate a new benchmark dataset– ImputeRefer for 3D-ERU, by incorporating human pointing gestures into existing 3D scene datasets that only contain language instructions. We also propose Ges3ViG, a novel model for 3D-ERU that achieves ~30% improvement in accuracy as compared to other 3D-ERU models and ~9% compared to other purely language-based 3D grounding models. Our code and dataset are available at https://github.com/AtharvMane/Ges3ViG.

Spatio-Temporal Video Grounding (STVG) - the task of identifying the target object in the field-of-view, that the language instruction refers to - is a fundamental vision-language task. Current STVG approaches typically utilize feeds from an RGB camera that is assumed to be always-on and process the video frames using complex neural network pipelines. As a result, they often impose prohibitive system overheads (energy, latency) on pervasive devices. To address this, we propose NeuroViG with two key innovations: (a) leveraging on event streams from a low-power neuromorphic event camera sensor to perform selective triggering of the more energy-hungry RGB camera for STVG, and (b) augmenting the STVG model with a lightweight Adaptive Frame Selector (AFS) that bypasses complex transformer-based operations for a majority of video frames, thereby enabling its execution on a pervasive Jetson AGX device. We have also introduced modifications to the neural network processing pipeline such that the system can offer tunable tradeoffs between accuracy and energy/latency. Our proposed NeuroViG system allows us to reduce the STVG energy overhead and latency by ~ 4x and ~ 3.8x, respectively, for less than 1% loss in accuracy.

While DNN models have dramatically improved the accuracy of machine vision tasks, pervasive deployments of vision sensors for surveillance tasks continue to suffer from high energy consumption and network traffic overhead. To tackle these problems, we introduce CollabCam , an edge-based machine vision system designed for multi-camera deployments that leverages the naturally-occurring overlaps in the field-of-view (FoV) among neighboring cameras. CollabCam synergistically combines two innovative ideas: (a) having each individual camera compose and transmit mixed-resolution frames (MRF) via lightweight down-sampling, where the transmitted images have significantly lower resolution in the shared, overlapping portions , and (b) performing inference, for an exemplar object detection task, for each camera stream using a new collaborative mechanism which utilizes suitably-translated object bounding boxes from a peer “collaborating” camera as an additional input channel. We demonstrate how this collaborative mechanism is generalizable and can be realized by simply retraining off-the-shelf object detector DNNs, such as YOLOv3 and SSD, without modifying their model structures. By emulating the performance of CollabCam using two benchmark outdoor-campus multi-camera datasets, we show that Collab-DNNs can accommodate a 50–60 fold reduction in image size (therefore reducing network transmission overhead), for both high-resolution (1056x1056) and low-resolution (512x512) images, with a modest ≤ 2 - 5% drop in object detection accuracy, compared to a non-collaborative approach that suffers a ∼ 45–60% drop in accuracy. Subsequently, by deploying a Raspberry-Pi based CollabCam prototype on a campus-based test-bed, we demonstrate that CollabCam can reduce the overall energy/image frame overhead by ∼25–35%, with even higher energy savings (∼35–45%) likely with hardware optimization. Finally, additional experiments help demonstrate that CollabCam can prove beneficial for varied (including multi-class) object detection tasks and that CollabCam’s performance benefits may be best realized by ensuring that the number of deployed, collaborating cameras is not excessively high.

Sustaining real-time, high-fidelity AI-based vision perception on edge devices is challenging due to both the high computational overhead of increasingly “deeper” Deep Neural Networks (DNNs) and the increasing resolution/quality of camera sensors. Such high-throughput vision perception is even more challenging in multi-tenancy systems, where video streams from multiple such high-quality cameras need to share the same GPU resource on a single edge device. Criticality-aware canvas-based processing is a promising paradigm that decomposes multiple concurrent video streams into Regions of Interest (RoI) and spatially channels the limited computational resources to selected RoI with higher “resolution,” thereby moderating the tradeoff between computational load, task fidelity, and processing throughput. RA-MOSAIC (Resource Adaptive MOSAIC) employs such canvas-based processing, while further tuning the incoming video streams and available resources on-demand to allow the system to adapt to dynamic changes in workload (often arising from variations in the number or size of relevant objects observed by individual cameras). RA-MOSAIC utilizes two distinct and synergistic concepts. First, at the camera sensor, a bandwidth-adaptive and lightweight Bandwidth-Adaptive Camera Transmission (BACT) method applies differential downsampling to create mixed-resolution individual frames that preferentially preserve resolution for critical RoIs, before being transmitted to the edge node. Second, at the edge, BACT video streams received from multiple cameras are decomposed into multi-scale RoI tiles and spatially packed using a novel workload-adaptive bin-packing strategy into a single “canvas frame.” Notably, the canvas frame itself is dynamically sized such that the edge device can opportunistically provide higher processing throughput for selected high-priority tiles during periods of lower aggregate workloads. To demonstrate RA-MOSAIC’s gains in processing throughput and perception fidelity, we evaluate RA-MOSAIC on a single NVIDIA Jetson TX2 edge device for two benchmark tasks—Drone-based Pedestrian Detection and Automatic License Plate Recognition. In a bandwidth-constrained wireless environment, RA-MOSAIC employs a batch size of 1 to pack up to 6 concurrent video streams on a dynamically sized canvas frame to provide (i) 14.3% gain in object detection accuracy and (ii) 11.11% gain in throughput on average (up to 20 FPS per camera, cumulatively 120 FPS), over our previous work MOSAIC, a naive canvas-based baseline. Compared to prior state-of-the-art baselines such as batched inference over extracted RoI, RA-MOSAIC provides a very significant, 29.6% gain in accuracy for a comparable throughput. Similarly, RA-MOSAIC dramatically outperforms bandwidth-adaptive baselines, such as First Come First Serve (FCFS) ( \(\leq 1\%\) accuracy gain but 5.6× or 566.67% throughput gain) and uniform grid packing (17% accuracy improvement and 5% throughput gain).

While video sensing, performed by resource-constrained pervasive devices, is a key enabler of many machine intelligence applications, the high energy and bandwidth overheads of streaming video transmission continue to present formidable deployment challenges. Motivated by the recent advancements in deep learning models, this article proposes the usage of a Generative Network-based technique for resource-efficient streaming video compression and transmission. However, we empirically show that while such generative network-based models offer superior compression gains compared to H.265, additional DNN optimization mechanisms are needed to substantially reduce their encoder complexity. Our proposed optimized system, dubbed Pr-Ge-Ne , adopts a carefully pruned encoder-decoder DNN, on the pervasive device, to efficiently encode a latent vector representation of intra-frame relative motion, and then uses a generator network at the decoder to reconstruct the frames by overlaying such motion information to ”animate” an initial reference frame. By evaluating three representative streaming video datasets, we show that Pr-Ge-Ne achieves around \(6\) – \(10\) -fold reduction in video transmission rates (with negligible impact on the accuracy of machine perception tasks) compared to H.265, while simultaneously reducing latency and energy overheads on a pervasive device by \(\sim\) 90% and 15–50%, respectively.

We present JIGSAW, a novel system that performs edge-based streaming perception over multiple video streams, while additionally factoring in the redundancy offered by the spatial overlap often exhibited in urban, multi-camera deployments. To assure high streaming throughput, JIGSAW extracts and spatially multiplexes multiple regions-of-interest from different camera frames into a smaller canvas frame. Moreover, to ensure that perception stays abreast of evolving object kinematics, JIGSAW includes a utility-based weighted scheduler to preferentially prioritize and even skip object-specific tiles extracted from an incoming stream of camera frames. Using the CityflowV2 traffic surveillance dataset, we show that JIGSAW can simultaneously process 25 cameras on a single Jetson TX2 with a 66.6% increase in accuracy and a simultaneous 18x (1800%) gain in cumulative throughput (475 FPS), far outperforming competitive baselines.

We address the challenge of multi-LiDAR interference, an issue of growing importance as LiDAR sensors are embedded in a growing set of pervasive devices. We introduce a novel approach named D2SR, enabling decentralized interference detection, mitigation, and recovery without explicit coordination among nearby LiDAR devices. D2SR comprises three stages: (a) Detection, which identifies interfered frames, (b) Mitigation, which performs time-shifting of a LiDAR’s active period to reduce interference, and (c) Recovery, which corrects or reconstructs the depth values in interfered regions of a depth frame. Key contributions include a lightweight interference detection algorithm achieving an F1-score of 92%, a simple yet effective decentralized de-synchronization mechanism, and a lightweight depth recovery pipeline that preserves high throughput processing on edge devices. Evaluation on Nvidia Jetson devices demonstrates D2SR’s efficacy: under static settings, D2SR accurately detects interference in 93% of cases (recall=82%) and reduces the depth estimation error by 27% (RMSE= 38.7 cm, compared to RMSE= 60.6 cm for a baseline without D2SR). Furthermore, D2SR is able to reduce the fraction of interfered frames by 75.1% and reduce the depth estimation error (for interfered frames) by 24.9% even for a moving robot scenario.

As RGB camera resolutions and frame-rates improve, their increased energy requirements make it challenging to deploy fast, efficient, and low-power applications on edge devices. Newer classes of sensors, such as the biologically inspired neuromorphic event-based camera, capture only changes in light intensity per-pixel to achieve operational superiority in sensing latency (O(μs)), energy consumption (O(mW)), high dynamic range (140dB), and task accuracy such as in object tracking, over traditional RGB camera streams. However, highly dynamic scenes can yield an event rate of up to 12MEvents/second, the processing of which could overwhelm resource-constrained edge devices. Efficient processing of high volumes of event data is crucial for ultra-fast machine vision on edge devices. In this poster, we present a profiler that processes simulated event streams from RGB videos into 6 variants of framed representations for DNN inference on an NVIDIA Jetson Orin AGX, a representative edge device. The profiler evaluates the trade-offs between the volume of events evaluated, the quality of the processed event representation, and processing time to present the design choices available to an edge-scale event camera-based application observing the same RGB scenes. We believe that this analysis opens up the exploration of novel system designs for real-time low-power event vision on edge devices.

As the use of pervasive devices expands into complex collaborative tasks such as cognitive assistants and interactive AR/VR companions, they are equipped with a myriad of sensors facilitating natural interactions, such as voice commands. Spatio-Temporal Video Grounding (STVG), the task of identifying the target object in the field-of-view referred to in a language instruction, is a key capability needed for such systems. However, current STVG models tend to be resource-intensive, relying on multiple cross-attentional transformers applied to each video frame. This results in runtime complexity that increases linearly with video length. Furthermore, deploying these models on mobile devices while maintaining a low-latency poses additional challenges. Hence, this paper explores the latency and energy requirements for implementing STVG models on a pervasive device.

Continuous tracking of eye movement dynamics plays a significant role in developing a broad spectrum of human-centered applications, such as cognitive skills (visual attention and working memory) modeling, human-machine interaction, biometric user authentication, and foveated rendering. Recently neuromorphic cameras have garnered significant interest in the eye-tracking research community, owing to their sub-microsecond latency in capturing intensity changes resulting from eye movements. Nevertheless, the existing approaches for event-based eye tracking suffer from several limitations: dependence on RGB frames, label sparsity, and training on datasets collected in controlled lab environments that do not adequately reflect real-world scenarios. To address these limitations, in this paper, we propose a dynamic graph-based approach that uses a neuromorphic event stream captured by Dynamic Vision Sensors (DVS) for high-fidelity tracking of pupillary movement. More specifically, first, we present EyeGraph, a large-scale multi-modal near-eye tracking dataset collected using a wearable event camera attached to a head-mounted device from 40 participants -- the dataset was curated while mimicking in-the-wild settings, accounting for varying mobility and ambient lighting conditions. Subsequently, to address the issue of label sparsity, we adopt an unsupervised topology-aware approach as a benchmark. To be specific, (a) we first construct a dynamic graph using Gaussian Mixture Models (GMM), resulting in a uniform and detailed representation of eye morphology features, facilitating accurate modeling of pupil and iris. Then (b) apply a novel topologically guided modularity-aware graph clustering approach to precisely track the movement of the pupil and address the label sparsity in event-based eye tracking. We show that our unsupervised approach has comparable performance against the supervised approaches while consistently outperforming the conventional clustering approaches.

We present the hardware design and software pipeline for an ultra-low power device, in the form factor of a wearable badge, that supports energy efficient sensing, processing and wireless transfer of human voice commands and interactions. The proposed system, called PA2BLO, is envisioned to support both: (a) real-time, scalable, authorized voice based interaction and control of devices and appliances, and (b) longitudinal, low-power logging of natural voice interactions. PA2BLO in-troduces two key novel capabilities. First, it includes a low power, low-complexity voice authentication module that is able to reliably authenticate an authorized user only using low sampling rate (500 Hz) audio data. Second, to reduce concerns around inadvertent leakage of voice biometrics to less secure voice-driven services, PA2BLO uses a power-efficient, randomized pitch shifting technique that dramatically lowers the ability to perform speaker recognition while preserving instruction/speech comprehensibility. We describe PA2BLO's Cortex M4F-based micro-controller based hardware implementation, which is care-fully designed to eliminate redundant processing and consumes less than 50J of energy per hour of active voice capture and processing. Through both controlled and naturalistic studies, we show that the PA2BLO prototype is capable of authenticating user voice segments reliably (accuracy> 89.8%) and can operate for well over a day (using a supercapacitor charged within just one minute) while capturing 2+ hours of active speaker data.

Canvas-based attention scheduling was recently pro-posed to improve the efficiency of real-time machine perception systems. This framework introduces a notion of focus locales, referring to those areas where the attention of the inference system should “allocate its attention”. Data from these locales (e.g., parts of the input video frames containing objects of interest) are packed together into a smaller canvas frame which is processed by the downstream machine learning algorithm. Compared with processing the entire input data frame, this practice saves resources while maintaining inference quality. Previous work was limited to a simplified solution where the focus locales are quantized to a small set of allowed sizes for the ease of packing into the canvas in a best-effort manner. In this paper, we remove this limiting constraint thus obviating quantization, and derive the first spatiotemporal schedulability bound for objects of arbitrary sizes in a canvas-based attention scheduling framework. We further allow object resizing and design a set of scheduling algorithms to adapt to varying workloads dynamically. Experiments on a representative AI-powered embedded platform with a real-world video dataset demonstrate the improvements in performance and inform the design and capacity planning of modern real-time machine perception pipelines.

We present LiLoc , a system for precise 3D localization and tracking of mobile IoT devices (e.g., robots) in indoor environments using multi-perspective LiDAR sensing. LiLoc stands out with two key differentiators. First, unlike traditional localization approaches, our method remains robust in dynamically changing environments, adeptly handling varying crowd levels and object layout changes. Second, LiLoc is independent of pre-built static maps, employing dynamically updated point clouds from infrastructural-mounted LiDARs and LiDARs on individual IoT devices. For fine-grained, near real-time tracking, LiLoc intermittently utilizes complex 3D “global” registration between point clouds for robust spot location estimates. It further complements this with simpler “local” registrations, continuously updating IoT device trajectories. We demonstrate that LiLoc can (a) support accurate location tracking with location and pose estimation error being ≦7.4 cm and ≦3.2°, respectively, for 84% of the time and the median error increasing only marginally (8%), for correctly estimated trajectories, when the ambient environment is dynamic; (b) achieve a 36% reduction in median location estimation error compared to an approach that uses only quasi-static global point cloud; and (c) obtain spot location estimates with a latency of only 973 msec. We also demonstrate how LiLoc efficiently integrates low-power inertial sensing, using a novel integration of inertial-based displacement to accelerate the local registration process, to enhance localization energy efficiency and latency.

Recent work suggests that computing resources, such as GPUs in real-time edge-based perception systems, need not have sufficient capacity to keep up with the input frame rates of all input devices (e.g., cameras) at their full-frame resolution. Rather, they can be under-provisioned because only parts of any given frame need to be inspected (i.e., paid attention to). This paper derives an attention allocation policy, called canvas-based attention scheduling that decides which parts of each frame of each device to inspect, and a corresponding schedulability condition that relates the spatiotemporal properties of surrounding objects to the ability of the edge-based perception subsystem to keep up with the state of the environment in real-time. It provides a quantitative estimate of adequate computing capacity for the expected perception workload. We implement a canvas-based attention scheduler for an object detection application and perform an empirical comparative study based on actual GPU hardware and surveillance videos. Results show that canvas-based attention scheduling keeps up with the environment while using a much smaller GPU capacity, compared with prior approaches.

Sustaining high fidelity and high throughput of perception tasks over vision sensor streams on edge devices remains a formidable challenge, especially given the continuing increase in image sizes (e.g., generated by 4K cameras) and complexity of DNN models. One promising approach involves criticality-aware processing, where the computation is directed selectively to "critical" portions of individual image frames. We introduce MOSAIC, a novel system for such criticality-aware concurrent processing of multiple vision sensing streams that provides a multiplicative increase in the achievable throughput with negligible loss in perception fidelity. MOSAIC determines critical regions from images received from multiple vision sensors and spatially bin-packs these regions using a novel multi-scale Mosaic Across Scales (MoS) tiling strategy into a single `canvas frame', sized such that the edge device can retain sufficiently high processing throughput. Experimental studies using benchmark datasets for two tasks, Automatic License Plate Recognition and Drone-based Pedestrian Detection, shows that MOSAIC, executing on a Jetson TX2 edge device, can provide dramatic gains in the throughput vs. fidelity tradeoff. For instance, for drone-based pedestrian detection, for a batch size of 4, MOSAIC can pack input frames from 6 cameras to achieve (a) 4.75X (475%) higher throughput (23 FPS per camera, cumulatively 138FPS) with ≤ 1% accuracy loss, compared to a First Come First Serve (FCFS) processing paradigm.

In real-time machine inference literature, canvas-based attention scheduling was recently introduced as an effective scheduling algorithm for real-time perception pipelines. In this framework, a notion of focus locales is maintained, referring to those locales on which the perception subsystem “focuses its attention”. Data from these locales (e.g., parts of input video frames corresponding to objects of interest) are packed into smaller bins called canvas frames that are then processed by the AI pipeline. The practice saves resources compared to processing the entirety of the original full frames. While prior work on canvas-based scheduling derived a schedulability bound, their bound applies only if focus locales are quantized into a small set of allowable container sizes for ease of packing into the canvas. In this work, we explore the possibility of removing this limiting assumption thus obviating quantization for a new bound, and generalizing the scheduling policy to allow for object resizing. Experiments on a representative AI-powered embedded platform with a real-world video dataset demonstrate improvements in efficiency in the presence and empirically validate the new bound. The result informs the design and capacity planning of modern real-time machine perception pipelines.

This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global coordinate system, and b) locate target objects referred by natural language and pointing gestures.

This paper presents a real-time multi-view scheduling framework for DNN-based live video analytics at the edge to minimize frame processing latency. The work is motivated by applications where a higher frame rate is important, not to miss actions of interest. Examples include defense, border security, and intruder detection applications where sensors (in this paper, cameras) are deployed to monitor key roads, chokepoints, or passageways to identify events of interest (and intervene in real-time). Supporting a higher frame rate entails lowering frame processing latency. We assume that multiple cameras are deployed with partially overlapping views. Each camera has access to limited onboard computing capacity. Many targets cross the field of view of these cameras (but the great majority do not require action). We take advantage of the spatial-temporal correlations among multi-camera video streams to perform target-to-camera assignment such that the maximum frame processing time across cameras is minimized. Specifically, we use a data-driven approach to identify objects seen by multiple cameras, and propose a batch-aware latency-balanced (BALB) scheduling algorithm to drive the object-to-camera assignment. We empirically evaluate the proposed system with a real-world surveillance dataset on a testbed consisting of multiple NVIDIA Jetson boards. The results show that our system substantially improves the video processing speed, attaining multiplicative speedups of 2.45× to 6.85×, and consistently outperforms the competitive static region partitioning strategy.

While Deep Neural Network (DNN) models have transformed machine vision capabilities, their extremely high computational complexity and model sizes present a formidable deployment roadblock for AIoT applications. We show that the complexity-vs-accuracy-vs-communication tradeoffs for such DNN models can be significantly addressed via a novel, lightweight form of "collaborative machine intelligence" that requires only runtime changes to the inference process. In our proposed approach, called ComAI, the DNN pipelines of different vision sensors share intermediate processing state with one another, effectively providing hints about objects located within their mutually-overlapping Field-of-Views (FoVs). CoMAI uses two novel techniques: (a) a secondary shallow ML model that uses features from early layers of a peer DNN to predict object confidence values in the image, and (b) a pipelined sharing of such confidence values, by collaborators, that is then used to bias a reference DNN’s outputs. We demonstrate that CoMAI (a) can boost accuracy (recall) of DNN inference by 20-50%, (b) works across heterogeneous DNN models and deployments, and (c) incurs negligible processing, bandwidth and processing overheads compared to non-collaborative baselines.

Supporting real-time referring expression comprehension (REC) on pervasive devices is an important capability for human-AI collaborative tasks. Model pruning techniques, applied to DNN models, can enable real-time execution even on resource-constrained devices. However, existing pruning strategies are designed principally for uni-modal applications, and suffer a significant loss of accuracy when applied to REC tasks that require fusion of textual and visual inputs. We thus present a multi-modal pruning model, LGMDP, which uses language as a pivot to dynamically and judiciously select the relevant computational blocks that need to be executed. LGMDP also introduces a new SoftSkip mechanism, whereby 'skipped' visual scales are not completely eliminated but approximated with minimal additional computation. Experimental evaluation, using 3 benchmark REC datasets and an embedded device implementation, shows that LGMDP can achieve 33% latency savings, with an accuracy loss 0.5% - 2%.

While many pervasive computing applications increasingly utilize real-time context extracted from a vision sensing infrastructure, the high energy overhead of DNN-based vision sensing pipelines remains a challenge for sustainable in-the-wild deployment. One common approach to reducing such energy overheads is the capture and transmission of lower-resolution images to an edge node (where the DNN inferencing task is executed), but this results in an accuracy-vs-energy tradeoff, as the DNN inference accuracy typically degrades with a drop in resolution. In this work, we introduce MRIM, a simple but effective framework to tackle this tradeoff. Under MRIM, the vision sensor platform first executes a lightweight preprocessing step to determine the saliency of different sub-regions within a single captured image frame, and then performs a saliency-aware non-uniform downscaling of individual sub-regions to produce a “mixed-resolution” image. We describe two novel low-complexity algorithms that the sensor platform can use to quickly compute suitable resolution choices for different regions under different energy/accuracy constraints. Experimental studies, involving object detection tasks evaluated traces from two benchmark urban monitoring datasets as well as a prototype Raspberry Pi-based MRIM implementation, demonstrate MRIM’s efficacy: even with unoptimized embedded platform, MRIM can provide system energy savings of 35+% or increase task accuracy by 8+%, over conventional baselines of uniform resolution downscaling or image encoding, while supporting high throughput.

We explore the effect of auxiliary labels in improving the classification accuracy of wearable sensor-based human activity recognition (HAR) systems, which are primarily trained with the supervision of the activity labels (e.g. running, walking, jumping). Supplemental meta-data are often available during the data collection process such as body positions of the wearable sensors, subjects' demographic information (e.g. gender, age), and the type of wearable used (e.g. smartphone, smart-watch). This information, while not directly related to the activity classification task, can nonetheless provide auxiliary supervision and has the potential to significantly improve the HAR accuracy by providing extra guidance on how to handle the introduced sample heterogeneity from the change in domains (i.e positions, persons, or sensors), especially in the presence of limited activity labels. However, integrating such meta-data information in the classification pipeline is non-trivial - (i) the complex interaction between the activity and domain label space is hard to capture with a simple multi-task and/or adversarial learning setup, (ii) meta-data and activity labels might not be simultaneously available for all collected samples. To address these issues, we propose a novel framework Conditional Domain Embeddings (CoDEm). From the available unlabeled raw samples and their domain meta-data, we first learn a set of domain embeddings using a contrastive learning methodology to handle inter-domain variability and inter-domain similarity. To classify the activities, CoDEm then learns the label embeddings in a contrastive fashion, conditioned on domain embeddings with a novel attention mechanism, enforcing the model to learn the complex domain-activity relationships. We extensively evaluate CoDEm in three benchmark datasets against a number of multi-task and adversarial learning baselines and achieve state-of-the-art nerformance in each avenue.

The primary contribution of this paper is designing and prototyping a real-time edge computing system, RhythmEdge, that is capable of detecting changes in blood volume from facial videos (Remote Photoplethysmography; rPPG), enabling cardio-vascular health assessment instantly. The benefits of RhythmEdge include non-invasive measurement of cardiovascular activity, real-time system operation, inexpensive sensing components, and computing. RhythmEdge captures a short video of the skin using a camera and extracts rPPG features to estimate the Photoplethysmography (PPG) signal using a multi-task learning framework while offloading the edge computation. In addition, we intelligently apply a transfer learning approach to the multi-task learning framework to mitigate sensor heterogeneities to scale the RhythmEdge prototype to work with a range of commercially available sensing and computing devices. Besides, to further adapt the software stack for resource-constrained devices, we postulate novel pruning and quantization techniques (Quantization: FP32, FP16; Pruned-Quantized: FP32, FP16) that efficiently optimize the deep feature learning while minimizing the runtime, latency, memory, and power usage. We benchmark RhythmEdge prototype for three different cameras and edge computing platforms while evaluating it on three publicly available datasets and an in-house dataset collected under challenging environmental circumstances. Our analysis indicates that RhythmEdge performs on par with the existing contactless heart rate monitoring systems while utilizing only half of its available resources. Furthermore, we perform an ablation study with and without pruning and quantization to report the model size (87%) vs. inference time (70%) reduction. We attested the efficacy of RhythmEdge prototype with a maximum power of 8W and a memory usage of 290MB, with a minimal latency of 0.0625 seconds and a runtime of 0.64 seconds per 30 frames.

In this demo paper, we design and prototype RhythmEdge [1], a low-cost, deep-learning-based contact-less system for regular HR monitoring applications. RhythmEdge benefits over existing approaches by facilitating contact-less nature, real-time/offline operation, inexpensive and available sensing components, and computing devices. Our RhythmEdge system is portable and easily deployable for reliable HR estimation in moderately controlled indoor or outdoor environments. RhythmEdge measures HR via detecting changes in blood volume from facial videos (Remote Photoplethysmography; rPPG) and provides instant assessment using off-the-shelf commercially available resource-constrained edge platforms and video cameras. We demonstrate the scalability, flexibility, and compatibility of the RhythmEdge by deploying it on three resource-constrained platforms of differing architectures (NVIDIA Jetson Nano, Google Coral Development Board, Raspberry Pi) and three heterogeneous cameras of differing sensitivity, resolution, properties (web camera, action camera, and DSLR). RhythmEdge further stores longitudinal cardiovascular information and provides instant notification to the users. We thoroughly test the prototype stability, latency, and feasibility for three edge computing platforms by profiling their runtime, memory, and power usage.

Ultra-low power communication is critical for supporting the next generation of battery-operated or energy harvesting battery-less Internet of Things (IoT) devices. Duty cycling protocols and wake-up receiver (WuRx) technologies, and their combinations, have been investigated as energy-efficient mechanisms to support selective, event-driven activation of devices. In this paper, we go one step further and show how WuRx can be used for an efficient and multi-purpose low power downlink (LPD) communication channel. We demonstrate how to (a) extend the wake-up signal to support low-power flexible and extensible unicast, multicast, and broadcast downlink communication and (b) utilize the WuRx-based LPD to also improve the energy efficiency of uplink data transfer. In addition, we show how the non-negligible energy overhead of conventional microcontroller based decoding of LPD communication can be substantially reduced by using the low-power universal asynchronous receiver/transmitter (LPUART) module of modern microcontrollers. Via experimental studies, involving both a functioning prototype and larger-scale simulations, we show that our proposed approach is compatible with conventional WLAN and offers a two-orders-of-magnitude improvement in uplink throughput and energy overheads over a competitive, IEEE 802.11 PSM-based baseline. This new LPD capability can also be used to improve the RF-based energy harvesting efficiency of battery-less IoT devices.

The paper introduces a novel, holistic approach for robust Screen-Camera Communication (SCC), where video content on a screen is visually encoded in a human-imperceptible fashion and decoded by a camera capturing images of such screen content. We first show that state-of-the-art SCC techniques have two key limitations for in-the-wild deployment: (a) the decoding accuracy drops rapidly under even modest screen extraction errors from the captured images, and (b) they generate perceptible flickers on common refresh rate screens even with minimal modulation of pixel intensity. To overcome these challenges, we introduce DeepLight, a system that incorporates machine learning (ML) models in the decoding pipeline to achieve humanly-imperceptible, moderately high SCC rates under diverse real-world conditions. DeepLight's key innovation is the design of a Deep Neural Network (DNN) based decoder that collectively decodes all the bits spatially encoded in a display frame, without attempting to precisely isolate the pixels associated with each encoded bit. In addition, DeepLight supports imperceptible encoding by selectively modulating the intensity of only the Blue channel, and provides reasonably accurate screen extraction (IoU values ≥ 83%) by using state-of-the-art object detection DNN pipelines. We show that a fully functional DeepLight system is able to robustly achieve high decoding accuracy (frame error rate < 0.2) and moderately-high data goodput (≥0.95 Kbps) using a human-held smartphone camera, even over larger screen-camera distances (ã 2m).

Driven by the ubiquitous proliferation of low-cost LED luminaires, visible light communication (VLC) has been established as a high-speed communications technology based on the high-frequency modulation of an optical source. In parallel, Visible Light Sensing (VLS) has recently demonstrated how vision-based at-a-distance sensing of mechanical vibrations (e.g., of factory equipment) can be performed using high frequency optical strobing. However, to date, exemplars of VLC and VLS have been explored in isolation, without consideration of their mutual dependencies. In this work, we explore whether and how high-throughput VLC and high-coverage VLS can be simultaneously supported. We first demonstrate the existence of a fundamental VLC-vs.-VLS tradeoff, driven by the duty cycle of the strobing light source: a larger duty cycle results in higher VLC throughput but reduced VLS coverage, and vice versa. To overcome this limitation, we evaluate two approaches: (a) time-multiplexed VLC and VLS on a single strobe, and (b) harmonic multi-strobing, where multiple light sources are strobed synchronously to effectively create low-duty cycle harmonics of the base strobe frequency. Finally, we present VibranSee, an approach that improves harmonic multi-strobing by adaptively tuning both (a) the strobe duty cycle and (b) the number of strobing harmonics used. Using both analytical studies and prototype-based experiments, we show VibranSee's benefits: it simultaneously achieves VLC data goodput that is ideally only 18.6% lower (and 23.9% lower for an actual working prototype) than the maximum communication rate and infers over 96.6% (100% for the prototype) of possible vibration frequencies.

We develop CACE (Constraints And Correlations mining Engine), a framework that significantly improves the recognition accuracy of complex daily activities in multi-inhabitant smarthomes. CACE views the implicit relationships between the activities of multiple people as an asset, and exploits such constraints and correlations in a hierarchical fashion, taking advantage of both personspecific sensor data (generated by wearable devices) and person-independent ambient sensor data (generated by ambient sensors). To effectively utilize such couplings, CACE first uses a multi-target particle filtering approach over ambient sensors captured movement data, to identify the number of distinct users and infer individual-specific movement trajectories. We then utilize a Hierarchical Dynamic Bayesian Network (HDBN)-based model for activity recognition. This model utilizes the inter-and-intra individual correlations and constraints, at both micro-activity and macro-activity levels, to recognize individual activities accurately. These constraints are learnt automatically using data-mining techniques, and help to dramatically reduce the computational complexity of HDBN-based inferencing. Empirical studies using a real-world testbed of five multi-inhabitant smarthomes shows that CACE is able to achieve an activity recognition accuracy of 95%, with a 16-fold reduction in computational overhead compared to traditional hybrid classification approaches.

In mobile crowd-sourcing systems, simply relying on people to opportunistically select and perform tasks typically leads to drawbacks such as low task acceptance/completion rates and undesirable spatial skews. In this paper, we utilize data from TASKer, a campus-based mobile crowd-sourcing platform, to empirically study and discover whether and how various context-aware notification strategies can help overcome such drawbacks. We first study worker interactions, in the absence of any notifications, to discover some spatio-temporal properties of task acceptance and completion. Based on these insights, we then experimentally demonstrate the effectiveness of two novel, non-personal, context-driven notification strategies, comparing the outcomes to two different baselines (no-notification and random-notification). Finally, using the data from the random-notification mechanism, we derive a classification model, incorporating several novel contextual features, that can predict a worker's responsiveness to notifications with high accuracy. Our work extends the crowd-sourcing literature by emphasizing the power of smart notifications for greater worker engagement.

This work demonstrates the feasibility and benefits of using pointing gestures, a naturally-generated additional input modality, to improve the multi-modal comprehension accuracy of human instructions to robotic agents for collaborative tasks.We present M2Gestic, a system that combines neural-based text parsing with a novel knowledge-graph traversal mechanism, over a multi-modal input of vision, natural language text and pointing. Via multiple studies related to a benchmark table top manipulation task, we show that (a) M2Gestic can achieve close-to-human performance in reasoning over unambiguous verbal instructions, and (b) incorporating pointing input (even with its inherent location uncertainty) in M2Gestic results in a significant (30%) accuracy improvement when verbal instructions are ambiguous.

Natural human interactions for Mixed Reality Applications are overwhelmingly multimodal: humans communicate intent and instructions via a combination of visual, aural and gestural cues. However, supporting low-latency and accurate comprehension of such multimodal instructions (MMI), on resource-constrained wearable devices, remains an open challenge, especially as the state-of-the-art comprehension techniques for each individual modality increasingly utilize complex Deep Neural Network models. We demonstrate the possibility of overcoming the core limitation of latency-vs.-accuracy tradeoff by exploiting cross-modal dependencies-i.e., by compensating for the inferior performance of one model with an increased accuracy of more complex model of a different modality. We present a sensor fusion architecture that performs MMI comprehension in a quasi-synchronous fashion, by fusing visual, speech and gestural input. The architecture is reconfigurable and supports dynamic modification of the complexity of the data processing pipeline for each individual modality in response to contextual changes. Using a representative “classroom” context and a set of four common interaction primitives, we then demonstrate how the choices between low and high complexity models for each individual modality are coupled. In particular, we show that (a) a judicious combination of low and high complexity models across modalities can offer a dramatic 3-fold decrease in comprehension latency together with an increase ~10-15% in accuracy, and (b) the right collective choice of models is context dependent, with the performance of some model combinations being significantly more sensitive to changes in scene context or choice of interaction.

We describe the development of a robust, accurate and practically-validated technique for estimating the occupancy count in indoor spaces, based on a combination of WiFi & video sensing. While fusing these two sensing-based inputs is conceptually straightforward, the paper demonstrates and tackles the complexity that arises from several practical artefacts, such as (i) over-counting when a single individual uses multiple WiFi devices and under-counting when the individual has no such device; (ii) corresponding errors in image analysis due to real-world artefacts, such as occlusion, and (iii) the variable errors in mapping image bounding boxes (which can include multiple possible types of human views: {head, torso, full-body}) to location coordinates. We develop statistical techniques to overcome these practical challenges, and finally propose a novel fusion algorithm, based on inexact bipartite matching of these two streams of independent estimates, to estimate the occupancy in complex, multi-inhabitant indoor spaces (such as university labs). We experimentally demonstrate that this estimation technique is robust and accurate, achieving less than 20% error, in an approx. 85m2lab space (with the error staying below 30% in a smaller 25m2area), across a wide variety of occupancy conditions.

Fine-grained, unobtrusive monitoring of gym exercises can help users track their own exercise routines and also provide corrective feedback. We propose W8-Scope, a system that uses a simple magnetic-cum-accelerometer sensor, mounted on the weight stack of gym exercise machines, to infer various attributes of gym exercise behavior. More specifically, using multiple machine learning models, W8-Scope helps identify who is exercising, what exercise she is doing, how much weight she is lifting, and whether she is committing any common mistakes. Real world studies, conducted with 50 subjects performing 14 different exercises over 103 distinct sessions in two gyms, show that W8-Scope can achieve high accuracy-e.g., identify the weight used with an accuracy of 97.5%, detect commonplace mistakes with 96.7% accuracy and identify the user with 98.7% accuracy. Moreover, by adopting incremental learning techniques, W8- Scope can also accurately track these various facets of exercise over longitudinal periods, in spite of the inherent natural changes in a user's exercising behavior.

We present ERICA, a digital personal trainer for users performing free weights exercises, with two key differentiators: (a) First, unlike prior approaches that either require multiple on-body wearables or specialized infrastructural sensing, ERICA uses a single in-ear "earable" device (piggybacking on a form factor routinely used by millions of gym-goers) and a simple inertial sensor mounted on each weight equipment; (b) Second, unlike prior work that focuses primarily on quantifying a workout, ERICA additionally identifies a variety of fine-grained exercising mistakes and delivers real-time, in-situ corrective instructions. To achieve this, we (a) design a robust approach for user-equipment association that can handle multiple (even 15) concurrently exercising users; (b) develop a suite of statistical models to detect several commonplace repetition-level mistakes; and (c) experimentally study the efficacy of multiple in-situ corrective feedback strategies. Via an end-to-end evaluation of ERICA with 33 participants naturally performing 3 dumbbell-based exercises, we show that (a) ERICA identifies over 94% of mistakes during the first 5 repetitions of a set, (b) the resulting feedback is viewed favorably by 78% of users, and (c) the feedback is effective, reducing mistakes by 10+% during subsequent repetitions.

This paper tackles the problem of detecting a comprehensive set of sensor faults that can occur in IoT-instrumented smart homes customized to infer Activities of Daily Living (ADL) from the activation of sensor sets. Specifically, sensors can suffer faults that (a) span durations that vary between several seconds to hours, (b) can result in both missing or false-alarm sensor-events. Previous fault detection approaches are geared primarily to identify missing faults (absence of sensor readings) of a permanent (very long-lived) nature, or sporadic false-alarm events. We propose DetectIF, a fault-detection framework that detects faults of varying time duration, and identifies both missing and false-alarm sensor events. DetectIF's key novelties include developing rules capturing spatiotemporal correlations among sensors and augmenting those rules with statistical properties of such sensor-specific behavior. To test DetectIF under a variety of fault behavior, we develop a unified fault framework where the tuning of a couple of parameters allows us to generate and inject faults of desired type and duration into an underlying sensor stream. Experiments with such comprehensive fault data shows that DetectIF achieves 82-95% fault-detection accuracy, improving precision by a huge amount (33-66%) over competitive, state-of-the-art baselines. Moreover, we demonstrate the benefits of applying DetectIF on unmodified, benchmark smart home datasets: it is able to detect additional likely faults that prior fault detection approaches miss, and thus consequently achieve an average of 30% higher ADL recognition accuracy compared to prior state-of-the-art fault detection techniques.

The proliferation of connected embedded devices, or the Internet of Things (IoT), together with recent advances in machine intelligence, will change the profile of future cloud services and introduce a variety of new research problems, both in cloud applications and infrastructure layers. These problems are centered around empowering individually resource-limited devices to exhibit intelligent behavior, both in sensing and control, thanks to a judicious utilization of cloud resources. Cloud services will enable learning from data, perform inference, and execute control, all with assurances on outcomes. This article discusses such emerging services and outlines five resulting new research directions towards enabling and optimizing intelligent, cloud-assisted sensing and control in the age of the Internet of Things.

The paper discusses an emerging suite of machine intelligence services that are of increasing importance in the highly instrumented world of the Internet of Things (IoT). The suite, called Eugene, would offer a form of intelligent behavior (based on deep neural networks) to otherwise simple embedded devices; the clients of the service. These devices would benefit from service resources to learn from data and to perform intelligent inference, classification, prediction, and estimation tasks that they are too limited to carry out on their own. The paper discusses the taxonomy of such services and the state of implementation, as well as the various challenges entailed, including scheduling, caching (of intelligent functions), and cooperative learning.

While analysis of urban commuting data has a long and demonstrated history of providing useful insights into human mobility behavior, such analysis has been performed largely in offline fashion and to aid medium-to-long term urban planning. In this work, we demonstrate the power of applying predictive analytics on real-time mobility data, specifically the smart-card generated trip data of millions of public bus commuters in Singapore, to create two novel and "live" smart city services. The key analytical novelty in our work lies in combining two aspects of urban mobility: (a) conformity: which reflects the predictability in the aggregated flow of commuters along bus routes, and (b) regularity: which captures the repeated trip patterns of each individual commuter. We demonstrate that the fusion of these two measures of behavior can be performed at city-scale using our BuScope platform, and can be used to create two innovative smart city applications. The Last-Mile Demand Generator provides O(mins) lookahead into the number of disembarking passengers at neighborhood bus stops; it achieves over 85% accuracy in predicting such disembarkations by an ingenious combination of individual-level regularity with aggregate-level conformity. By moving driverless vehicles proactively to match this predicted demand, we can reduce wait times for disembarking passengers by over 75%. Independently, the Neighborhood Event Detector uses outlier measures of currently operating buses to detect and spatiotemporally localize dynamic urban events, as much as 1.5 hours in advance, with a localization error of ~450 meters.

Energy harvesting, from a diverse set of modes such as light or motion, has been viewed as the key to developing batteryless sensing devices. In this paper, we develop the nascent idea of harvesting RF energy from WiFi transmissions, applying it to power a prototype wearable device that captures and transmits accelerometer sensor data. Our solution, WiWear, has two key innovations: 1) beamforming WiFi transmissions to significantly boost the energy that a receiver can harvest ~23 meters away, and 2) smart zero-energy, triggering of inertial sensing, that allows intelligent duty-cycled operation of devices whose transient power consumption far exceeds what can be instantaneously harvested. We provide experimental validation, using both careful measurement studies as well as a controlled study with human participants, to show the viability of a custom-built WiWear-based wearable device, at least in office environments.

We present SmrtFridge, a consumer-grade smart fridge prototype that demonstrates two key capabilities: (a) identify the individual food items that users place in or remove from a fridge, and (b) estimate the residual quantity of food items inside a refrigerated container (opaque or transparent). Notably, both of these inferences are performed unobtrusively, without requiring any explicit user action or tagging of food objects. To achieve these capabilities, SmrtFridge uses a novel interaction-driven, multi-modal sensing pipeline, where Infrared (IR) and RGB video sensing, triggered whenever a user interacts naturally with the fridge, is used to extract a foreground visual image of the food item, which is then processed by a state-of-the-art DNN classifier. Concurrently, the residual food quantity is estimated by exploiting slight thermal differences, between the empty and filled portions of the container. Experimental studies, involving 12 users interacting naturally with 19 common food items and a commodity fridge, show that SmrtFridge is able to (a) extract at least 75% of a food item's image in over 97% of interaction episodes, and consequently identify the individual food items with precision/recall values of ~ 85%, and (b) perform robust coarse-grained (3 level) classification of the residual food quantity with an accuracy of ~ 75%.

In this paper, we present I4S, a system that identifies item interactions of customers in a retail store through sensor data fusion from smartwatches, smartphones and distributed BLE beacons. To identify these interactions, I4S builds a gesture-triggered pipeline that (a) detects the occurrence of "item picks", and (b) performs fine-grained localization of such pickup gestures. By analyzing data collected from 31 shoppers visiting a midsized stationary store, we show that we can identify person-independent picking gestures with a precision of over 88%, and identify the rack from where the pick occurred with 91%+ precision (for popular racks).

This paper analyzes a human-centric framework, called SmartABLE, for easy retrieval of the sensor values from pervasively deployed smart objects in a campus-like environment. In this framework, smartphones carried by campus occupants act as data mules, opportunistically retrieving data from nearby BLE (Bluetooth Low Energy) equipped smart object sensors and relaying them to a backend repository. We focus specifically on dynamically varying the transmission power of the deployed BLE beacons, so as to extend their operational lifetime without sacrificing the frequency of sensor data retrieval. We propose a memetic algorithm-based power adaptation strategy that can handle deployments of thousands of beacons and tackles two distinct objectives: (1) maximizing BLE beacon lifetime, and (2) reducing the BLE scanning energy of the mules. Using real-world movement traces on the Singapore Management University campus, we show that the benefit of such mule movement-aware power adaptation: it provides reliably frequent retrieval of BLE sensor data, while achieving a significant (5-fold) increase in the sensor lifetime, compared to a traditional fixed-power approach.

Real-world deployments of WiFi-based indoor localization in large public venues are few and far between as most state-of-the-art solutions require either client or infrastructure-side changes. Hence, even though high location accuracy is possible with these solutions, they are not practical due to cost and/or client adoption reasons. Majority of the public venues use commercial controller-managed WLAN solutions, that neither allow client changes nor infrastructure changes. In fact, for such venues we have observed highly heterogeneous devices with very low adoption rates for client-side apps.

With app-based interaction increasingly permeating all aspects of daily living, it is essential to ensure that apps are designed to be inclusive and are usable by a wider audience such as the elderly, with various impairments (e.g., visual, audio and motor). We propose Empath-D, a system that fosters empathetic design, by allowing app designers, in-situ, to rapidly evaluate the usability of their apps, from the perspective of impaired users. To provide a truly authentic experience, Empath-D carefully orchestrates the interaction between a smartphone and a VR device, allowing the user to experience simulated impairments in a virtual world while interacting naturally with the app, using a real smartphone. By carefully orchestrating the VR-smartphone interaction, Empath-D tackles challenges such as preserving low-latency app interaction, accurate visualization of hand movement and low-overhead perturbation of I/O streams. Experimental results show that user interaction with Empath-D is comparable (both in accuracy and user perception) to real-world app usage, and that it can simulate impairment effects as effectively as a custom hardware simulator.

No abstract available.

We investigate the problem of making human activity recognition (AR) scalable-i.e., allowing AR classifiers trained in one context to be readily adapted to a different contextual domain. This is important because AR technologies can achieve high accuracy if the classifiers are trained for a specific individual or device, but show significant degradation when the same classifier is applied context-e.g., to a different device located at a different on-body position. To allow such adaptation without requiring the onerous step of collecting large volumes of labeled training data in the target domain, we proposed a transductive transfer learning model that is specifically tuned to the properties of convolutional neural networks (CNNs). Our model, called HDCNN, assumes that the relative distribution of weights in the different CNN layers will remain invariant, as long as the set of activities being monitored does not change. Evaluation on real-world data shows that HDCNN is able to achieve high accuracy even without any labeled training data in the target domain, and offers even higher accuracy (significantly outperforming competitive shallow and deep classifiers) when even a modest amount of labeled training data is available.

We describe the design and implementation of a smartwatch-based, completely unobtrusive, food journaling system, where the smartwatch helps to intelligently capture useful images of food that an individual consumes throughout the day. The overall system, called Annapurna, is based on three key components: (a) a smartwatch-based gesture recognizer to identify eating gestures, (b) a smartwatch-based image capturer that obtains a small set of relevant and useful images with a low energy overhead, and (c) a server-based image filtering engine that removes irrelevant uploaded images, and then catalogs them through a portal. Our primary challenge is to make the system robust to the huge diversity in natural eating habits and food choices. We show how we address this by an appropriate coupling between a smartwatch's camera sensor and inertial sensor-based tracking of eating gestures, thereby helping to capture multiple likely-to-be-useful images with low energy overhead. Through a series of real-world, in-the-wild studies, we demonstrate the end-to-end working of Annapurna, which captures useful images in over 95% of all natural eating episodes.

In this article, we investigate effective ways of utilizing crowdworkers in providing various urban services. The task recommendation platform that we design can match tasks to crowdworkers based on workers’ historical trajectories and time budget limits, thus making recommendations personal and efficient. One major challenge we manage to address is the handling of crowdworker’s trajectory uncertainties. In this article, we explicitly allow multiple routine routes to be probabilistically associated with each worker. We formulate this problem as an integer linear program whose goal is to maximize the expected total utility achieved by all workers. We further exploit the separable structures of the formulation and apply the Lagrangian relaxation technique to scale up computation. Numerical experiments have been performed over the instances generated using the realistic public transit dataset in Singapore. The results show that we can find significantly better solutions than the deterministic formulation, and in most cases we can find solutions that are very close to the theoretical performance limit. To demonstrate the practicality of our approach, we deployed our recommendation engine to a campus-scale field trial, and we demonstrate that workers receiving our recommendations incur fewer detours and complete more tasks, and are more efficient against workers relying on their own planning (25% more for top workers who receive recommendations). This is achieved despite having highly uncertain worker trajectories. We also demonstrate how to further improve the robustness of the system by using a simple multi-coverage mechanism.

We advocate for and introduce TRANSense, a framework for urban transportation service analytics that combines participatory smartphone sensing data with city-scale transportation-related transactional data (taxis, trains, etc.). Our work is driven by the observed limitations of using each data type in isolation: (a) commonly-used anonymous city-scale datasets (such as taxi bookings and GPS trajectories) provide insights into the aggregate behavior of transport infrastructure, but fail to reveal individual-specific transport experiences (e.g., wait times in taxi queues); while (b) mobile sensing data can capture individual-specific commuting-related activities, but suffers from accuracy and energy overhead challenges due to usage artefacts and lack of appropriate sensing triggers. TRANSense demonstrates how a judicious fusion of such disparate data sources can overcome these challenges and offer novel insights. We detail two examples: (a) Taxi Service Analyzer that provides accurate detection of commuter queuing for taxis and estimates their wait time, by using taxi trip records to identify potential taxi locations with high demand and subsequently selectively triggering mobile sensing-based queuing analytics on nearby commuters; and (b) Subway Boarding Analyzer that identifies instances when passengers fail to board arriving trains, by first estimating train arrivals from temporal patterns of passenger egress at station gantries, and then using mobile sensing-based analysis of commuter movement behavior on platforms. Experiments with real-world datasets (from over 20,000 taxis and 1.7 million commuters in Singapore) show the power of this approach: the taxi service analyzer detects commuter queuing with over 90 percent accuracy with negligible energy overhead and estimates wait times with error margins below 15 percent, whereas the subway boarding analyzer can detect failed boarding events with a precision of over 90 percent (more than thrice what is achievable through purely mobile sensing).

We present Follow-My-Lead, an alternative indoor navigation technique that uses visual information recorded on an actual navigation path as a navigational guide. Its design revealed a trade-off between the fidelity of information provided to users and their effort to acquire it. Our first experiment revealed that scrolling through a continuous image stream of the navigation path is highly informative, but it becomes tedious with constant use. Discrete image checkpoints require less effort, but can be confusing. A balance may be struck by adding fast video transitions between image checkpoints, but precise control is required to handle difficult situations. Authoring still image checkpoints is also difficult, and this inspired us to invent a new technique using video checkpoints. We conducted a second experiment on authoring and navigation performance and found video checkpoints plus fast video transitions to be better than both image checkpoints plus fast video transitions and traditional written instructions.

This paper establishes the power of dynamic collaborative task completion among workers for urban mobile crowd-sourcing. Collaboration is defined via the notion of peer referrals, whereby a worker who has accepted a location-specific task, but is unlikely to visit that location, offloads the task to a willing friend. Such a collaborative framework might be particularly useful for task bundles, especially for bundles that have higher geographic dispersion. The challenge, however, comes from the high similarity observed in the spatio-temporal pattern of task completion among friends. Using extensive real-world crowd-sourcing studies conducted over 7 weeks and 1000+ workers on a campus-based crowd-sourcing platform, we quantify the effect of such "task completion homophily", and show that incorporating such peer-preferences can improve worker-specific models of task preferences by over 30%. We then show that such collaborative offloading works in spite of such spatio-temporal similarity, primarily because workers refer tasks to their close friends, who in turn perform such peer-requested tasks (with over 95% completion rate) even if they experience detours that are significantly larger (often more than twice) than what they normally tolerate for platform-recommended tasks.

We propose BreathPrint, a new behavioural biometric signature based on audio features derived from an individual's commonplace breathing gestures. Specifically, BreathPrint uses the audio signatures associated with the three individual gestures: sniff, normal, and deep breathing, which are sufficiently different across individuals. Using these three breathing gestures, we develop the processing pipeline that identifies users via the microphone sensor on smartphones and wearable devices. In BreathPrint, a user performs breathing gestures while holding the device very close to their nose. Using off-the-shelf hardware, we experimentally evaluate the BreathPrint prototype with 10 users, observed over seven days. We show that users can be authenticated reliably with an accuracy of over 94% for all the three breathing gestures in intra-sessions and deep breathing gesture provides the best overall balance between true positives (successful authentication) and false positives (resiliency to directed impersonation and replay attacks). Moreover, we show that this breathing sound based biometric is also robust to some typical changes in both physiological and environmental context, and that it can be applied on multiple smartphone platforms. Early results suggest that breathing based biometrics show promise as either to be used as a secondary authentication modality in a multimodal biometric authentication system or as a user disambiguation technique for some daily lifestyle scenarios.

Mobile crowd-tasking markets are growing at an unprecedented rate with increasing number of smartphone users. Such platforms differ from their online counterparts in that they demand physical mobility and can benefit from smartphone processors and sensors for verification purposes. Despite the importance of such mobile crowd-tasking markets, little is known about the labor supply dynamics and mobility patterns of the users.

While mobile crowd-sourcing has become a game-changer for many urban operations, such as last mile logistics and municipal monitoring, we believe that the design of such crowd-sourcing strategies must better accommodate the real-world behavioral preferences and characteristics of users. To provide a real-world testbed to study the impact of novel mobile crowd-sourcing strategies, we have designed, developed and experimented with a real-world mobile crowd-tasking platform on the SMU campus, called TA&Sslash;Ker. We enhanced the TA$Ker platform to support several new features (e.g., task bundling, differential pricing and cheating analytics) and experimentally investigated these features via a two-month deployment of TA$Ker, involving 900 real users on the SMU campus who performed over 30,000 tasks. Our studies (i) show the benefits of bundling tasks as a combined package, (ii) reveal the effectiveness of differential pricing strategies and (iii) illustrate key aspects of cheating (false reporting) behavior observed among workers.

We propose CACE (Constraints And Correlations mining Engine) which investigates the challenges of improving the recognition of complex daily activities in multi-inhabitant smart homes, by better exploiting the spatiotemporal relationships across the activities of different individuals. We first propose and develop a loosely-coupled Hierarchical Dynamic Bayesian Network (HDBN), which both (a) captures the hierarchical inference of complex (macro-activity) contexts from lower-layer microactivity context (postural and improved oral gestural context), and (b) embeds the various types of behavioral correlations and constraints (at both micro-and macro-activity contexts) across the individuals. While this model is rich in terms of accuracy, it is computationally prohibitive, due to the explosive increase in the number of jointly-defined states. To tackle this challenge, we employ data mining to learn behaviorally-driven context correlations in the form of association rules, we then use such rules to prune the state space dramatically. To evaluate our framework, we build a customized smart home system and collected naturalistic multi-inhabitant smart home activities data. The system performance is illustrated with results from real-time system deployment experiences in a smart home environment reveals a radical (max 16 fold) reduction in the computational overhead compared to traditional hybrid classification approaches, as well as an improved activity recognition accuracy of max 95%.

The increasing popularity of mobile devices in recent years has spurred interest in querying and tracking their locations for a wide variety of applications. Many naive approaches to localizing mobile devices, however, suffer from excessive energy use, poor accuracy in indoor environments, or excessive complexity presented to the application developer. We ask: does processing and evaluating location queries using a server help reduce complexity and inaccuracy for these queries without overusing the limited energy available on mobile devices? Also, what techniques can be combined with this approach to further reduce energy use? In this work, we try to answer these questions by developing a dynamic query optimization framework. We implement a simple prototype using Android smartphones and evaluate its performance against common naive approaches and Google's geofences location query system. Our results show energy-use reductions of up to 75% and dramatically improved precision of 45-60% compared to less sophisticated approaches, which compare favorably to Google's geofences while allowing more fine-grained selection of regions of interest.

In this paper, we present LiveLabs, a first-of-its-kind testbed that is deployed across a university campus, convention centre, and resort island and collects real-time attributes such as location, group context etc., from hundreds of opt-in participants. These venues, data, and participants are then made available for running rich human-centric behavioural experiments that could test new mobile sensing infrastructure, applications, analytics, or more social-science type hypotheses that influence and then observe actual user behaviour. We share case studies of how researchers from around the world have and are using LiveLabs, and our experiences and lessons learned from building, maintaining, and expanding Live-Labs over the last three years.

We investigate the possibility of using a combination of a smartphone and a smartwatch, carried by a shopper, to get insights into the shopper's behavior inside a retail store. The proposed IRIS framework uses standard locomotive and gestural micro-activities as building blocks to define novel composite features that help classify different facets of a shopper's interaction/experience with individual items, as well as attributes of the overall shopping episode or the store. Besides defining such novel features, IRIS builds a novel segmentation algorithm, which partitions the duration of an entire shopping episode into atomic item-level interactions, by using a combination of feature-based landmarking, change point detection and variable-order HMM-based sequence prediction. Experiments with 50 real-life grocery shopping episodes, collected from 25 shoppers, we show that IRIS can demarcate item-level interactions with an accuracy of approx. 91%, and subsequently characterize item-and-episode level shopper behavior with accuracies of over 90%.

In pervasive computing environments, understanding the context of an entity is essential for adapting the application behavior to changing situations. In our view, context is a high-level representation of a user or entity's state and can capture location, activities, social relationships, capabilities, etc. Inherently, however, these high-level context metrics are difficult to capture using uni-modal sensors only and must therefore be inferred using multi-modal sensors. A key challenge in supporting context-aware pervasive computing is how to determine multiple high-level context metrics simultaneously and energy-efficiently using low-level sensor data streams collected from the environment and the entities present therein. A key challenge is addressing the fact that the algorithms that determine different high-level context metrics may compete for access to low-level sensors. In this paper, we first highlight the complexities of determining multiple context metrics as compared to a single context and then develop a novel framework and practical implementation for this problem. The proposed framework captures the tradeoff between the accuracy of estimating multiple context metrics and the overhead incurred in acquiring the necessary sensor data streams. In particular, we develop two variants of a heuristic algorithm for multi-context search that compute the optimal set of sensors contributing to the multi-context determination as well as the associated parameters of the sensing tasks (e.g., the frequency of data acquisition). Our goal is to satisfy the application requirements for a specified accuracy at a minimum cost. We compare the performance of our heuristics with a brute-force based approach for multi-context determination. Experimental results with SunSPOT, Shimmer and Smartphone sensors in smart home environments demonstrate the potential impact of the proposed framework.

This paper investigates how digital traces of people's movements and activities in the physical world (e.g., at college campuses and commutes) may be used to detect local, short-lived events in various urban spaces. Past work that use occupancy-related features can only identify high-intensity events (those that cause large-scale disruption in visit patterns). In this paper, we first show how longitudinal traces of the coordinated and group-based movement episodes obtained from individual-level movement data can be used to create a socio-physical network (with edges representing tie strengths among individuals based on their physical world movement & collocation behavior). We then investigate how two additional families of socio-physical features: (i) group-level interactions observed over shorter timescales and (ii) socio-physical network tie-strengths derived over longer timescales, can be used by state-of-the-art anomaly detection methods to detect a much wider set of both high & low intensity events. We utilize two distinct datasets--one capturing coarse-grained SMU campus-wide indoor location data from hundreds of students, and the other capturing commuting behavior by millions of users on Singapore's public transport network--to demonstrate the promise of our approaches: the addition of group and socio-physical tie-strength based features increases recall (the percentage of events detected) more than 2-folds (to 0.77 on the SMU campus and to 0.73 at sample MRT stations), compared to pure occupancy-based approaches.

In this paper, we show that characterizing user behaviour from location or smartphone usage traces, without accounting for the interaction of individuals in physical-world groups, can lead to erroneous results. We conducted one of the largest studies in the UbiComp domain thus far, involving indoor location traces of more than 6,000 users, collected over a 4-month period at our university campus, and further studied fine-grained App usage of a subset of 156 Android users. We apply a state-of-the-art group detection algorithm to annotate such location traces with group vs. individual context, and then show that individuals vs. groups exhibit significant differences along three behavioural traits: (1) the mobility pattern, (2) the responsiveness to calls / SMSs and (3) application usage. We show that these significant differences are robust to underlying errors in the group detection technique and that the use of such group context leads to behavioural results that differ from those reported in prior popular work.

We present QueueVadis, a system that addresses the problem of estimating, in real-time, the properties of queues at commonplace urban locations, such as coffee shops, taxi stands and movie theaters. Abjuring the use of any queuing-specific infrastructure sensors, QueueVadis uses participatory mobile sensing to detect both (i) the individual-level queuing episodes for any arbitrarily-shaped queue (by a characteristic locomotive signature of short bursts of "shuffling forward" between periods of "standing") and (ii) the aggregate-level queue properties (such as expected wait or service times) via appropriate statistical aggregation of multi-person data. Moreover, for venues where multiple queues are too close to be separated via location estimates, QueueVadis also uses a novel disambiguation technique to separate users into multiple distinct queues. User studies, performed with 138 cumulative total users observed at 23 different real-world queues across Singapore and Japan, show that QueueVadis is able to (a) identify all individual queuing episodes, (b) predict service and wait times fairly accurately (with median estimation errors in the 10%--20% range), independent of the queue's shape, (c) separate users in multiple proximate queues with close to 80% accuracy and (d) provide reasonable estimates when the participation rate (the fraction of QueueVadis-equipped people in the queue) is modest.

Inertial sensing based indoor localization currently requires fairly precise layout maps, to help provide constraints and landmarks that bound the error drift. In this paper, we seek to improve the accuracy of one component of inertial-based tracking, namely the estimation of an individual's stride-length, so as to reduce the cumulative drift. We show that an individual's stride-length is affected by both his/her movement speed and heading-changes in the trajectory, and present an adaptive, online stride-length estimation algorithm that learns appropriate stride-length distributions for different (speed, heading) combinations. Initial experiments conducted using our proposed approach in combination with state-of-the-art step counting and heading estimation techniques, reduce the 95th percentile of average localization error by ≈ 30%. We thus envisage that inertial tracking may become practical even with coarse-grained map information.

The US Army Research Laboratory (ARL) and the Air Force Research Laboratory (AFRL) have established a collaborative research enterprise referred to as the Situational Understanding Research Institute (SURI). The goal is to develop an information processing framework to help the military obtain real-time situational awareness of physical events by harnessing the combined power of multiple sensing sources to obtain insights about events and their evolution. It is envisioned that one could use such information to predict behaviors of groups, be they local transient groups (e.g., Protests) or widespread, networked groups, and thus enable proactive prevention of nefarious activities. This paper presents a vision of how social media sources can be exploited in the above context to obtain insights about events, groups, and their evolution.

Driven by the rapid market growth of sensors and beacons that offer Bluetooth Low Energy (BLE) based connectivity, this paper empirically investigates the performance characteristics of the BLE interface on multiple Android smartphones, and the consequent impact on a proposed BLE-based service: continuous indoor location. We first use extensive measurement studies with multiple Android devices to establish that the BLE interface on current smartphones is not as "low-energy" as nominally expected, and establish that continuous use of such a BLE interface is not feasible unless we choose a moderately large scan interval and a low duty cycle. We then explore the implications of such constraints, on the parameters of a smart phone's BLE stack, on the accuracy of a BLE-based indoor localization techniques. We show that while RF-based indoor location can be highly accurate (80% of estimates have errors less than or equal to 4 meters) for stationary users only if the density of beacons is high, the combination of (large scan interval, low duty cycle) causes the location error to degrade significantly for moving users. These results provide practical insights into the use cases and limitations for future BLE-based mobile services.

To promote energy-efficient operations in residential and office buildings, non-intrusive load monitoring (NILM) techniques have been proposed to infer the fine-grained power consumption and usage patterns of appliances from power-line measurement data. Fine-grained monitoring of everyday appliances (such as toasters and coffee makers) can not only promote energy-efficient building operations, but also provide unique insights into the context and activities of individuals. Current building-level NILM techniques are unable to identify the consumption characteristics of relatively low-load appliances, whereas smart-plug based solutions incur significant deployment and maintenance costs. In this paper, we investigate an intermediate architecture, where smart circuit breakers provide measurements of aggregate power consumption at room (or section) level granularity. We then investigate techniques to identify the usage and energy consumption of individual appliances from such measurements. We first develop a novel correlation-based approach called CBPA to identify individual appliances based on both their unique transient and steady-state power signatures. While promising, CBPA fails when the set of candidate appliances is too large. To further improve the accuracy of appliance level usage estimation, we then propose a hybrid system called AARPA, which uses mobile sensing to first infer high-level activities of daily living (ADLs), and then uses knowledge of such ADLs to effectively reduce the set of candidate appliances that potentially contribute to the aggregate readings at any point. We evaluate two variants of this algorithm, and show, using real-life data traces gathered from 10 domestic users, that our fusion of mobile and power-line sensing is very promising: it identified all devices that were used in each data trace, and it identified the usage duration and energy consumption of low-load consumer appliances with 87% accuracy.

While mobile and wearable sensing can capture unique insights into fine-grained activities (such as gestures and limb-based actions) at an individual level, their energy overheads are still prohibitive enough to prevent them from being executed continuously. In this paper, we explore practical alternatives to addressing this challenge-by exploring how cheap infrastructure sensors or information sources (e.g., BLE beacons) can be harnessed with such mobile/wearable sensors to provide an effective solution that reduces energy consumption without sacrificing accuracy. The key idea is that many fine-grained activities that we desire to capture are specific to certain location, movement or background context: infrastructure sensors and information sources (e.g., BLE beacons) offer practical and cheap ways to identify such context. In this paper, we first explore how various infrastructure, mobile & wearable sensors can be used to identify fine-grained location/movement context (e.g., transiting through a door). We then show, using a couple of illustrative examples (specifically, the detection of `switch pressing' before exiting a room and the identification of `water drinking' after approaching a water cooler) to show that such background context can be predicted, with sufficient accuracy, with sufficient lead time to enable a `triggered' model for mobile/wearable sensing of such microscopic, transient gestures and activities. Moreover, such `triggered' sensing also helps to improve the accuracy of such microscopic gesture recognition, by reducing the set of candidate activity labels. Empirical experiments show that we are able to identify 82.2% of switch-pressing and 91.73% of water-drinking activities in a campus lab setting, with a significant reduction in active sensing time (up to 92.9% compared to continuous sensing).

This paper presents a cross-layer approach for enabling high-throughput reliable multicast in multi-hop wireless mesh networks. The building block of our approach is a multicast routing metric, called the expected multicast transmission count (EMTX). EMTX is designed to capture the combined effects of MAC-layer retransmission-based reliability, wireless broadcast advantage, and link quality awareness. The EMTX of single-hop transmission of a multicast packet from a sender is the expected number of multicast transmissions (including retransmissions) required for its next-hop recipients to receive the packet successfully. We formulate the EMTX-based multicast problem with the objective of minimizing the sum of EMTX over all forwarding nodes in the multicast tree, aiming to reduce network bandwidth consumption while ensure high end-to-end packet delivery ratio for the multicast traffic. We provide rigorous mathematical formulations and methods to find near-optimal solutions of the problem computationally efficiently. We present centralized and distributed algorithms, and demonstrate their effectiveness in tackling the EMTX-based multicast problem with a combination of theoretical and numerical results. Simulation experiments show that, in comparison with two baseline approaches, EMTX-based multicast routing reduces the number of hop-by-hop transmissions per packet by up to 40 percent and yet improves the multicast throughput by up to 24 percent.

We investigate the problem of large-scale mobile crowd-tasking, where a large pool of citizen crowd-workers are used to perform a variety of location-specific urban logistics tasks. Current approaches to such mobile crowd-tasking are very decentralized: a crowd-tasking platform usually provides each worker a set of available tasks close to the worker's current location; each worker then independently chooses which tasks she wants to accept and perform. In contrast, we propose TRACCS, a more coordinated task assignment approach, where the crowd-tasking platform assigns a sequence of tasks to each worker, taking into account their expected location trajectory over a wider time horizon, as opposed to just instantaneous location. We formulate such task assignment as an optimization problem, that seeks to maximize the total payoff from all assigned tasks, subject to a maximum bound on the detour (from the expected path) that a worker will experience to complete her assigned tasks. We develop credible computationally-efficient heuristics to address this optimization problem (whose exact solution requires solving a complex integer linear program), and show, via simulations with realistic topologies and commuting patterns, that a specific heuristic (called Greedy-ILS) increases the fraction of assigned tasks by more than 20%, and reduces the average detour overhead by more than 60%, compared to the current decentralized approach.

In this paper, we reduce the energy overheads of continuous mobile sensing for context-aware applications that are interested in collective context or events. We propose a cloud-based query management and optimization framework, called CloQue, which can support concurrent queries, executing over thousands of individual smartphones. CloQue exploits correlation across context of different users to reduce energy overheads via two key innovations: i) Dynamically reordering the order of predicate processing to preferentially select predicates with not just lower sensing cost and higher selectivity, but that maximally reduce the uncertainty about other context predicates, and ii) intelligently propagating the query evaluation results to dynamically update the uncertainty of other correlated, but yet-to-be evaluated, context predicates. An evaluation, using real cell phone traces from a real world dataset shows significant energy savings (between 30 to 50% compared with traditional short-circuit systems) with little loss in accuracy (5% at most).

Detecting the group context of an individual (i.e., whether an individual is alone or part of a group) in crowded public spaces, such as shopping malls, is an important goal with many practical applications. However, in crowded indoor spaces, understanding the group-dependent movement behavior is a non-trivial problem as: (1) detecting groups is hard as the density ensures that at any location, a large number of people are moving together, (2) location tracking in many real-world venues is either absent or not very accurate, and (3) indoor mobility models that take into account group attributes (such as group size) are rare. In this paper, we first introduce GruMon, a platform for near real-time group monitoring in dense, public spaces, and then demonstrate how the movement & residency properties of individuals are significantly affected when they are in groups.

Real-time monitoring of groups and their rich contexts will be a key building block for futuristic, group-aware mobile services. In this paper, we propose GruMon, a fast and accurate group monitoring system for dense and complex urban spaces. GruMon meets the performance criteria of precise group detection at low latencies by overcoming two critical challenges of practical urban spaces, namely (a) the high density of crowds, and (b) the imprecise location information available indoors. Using a host of novel features extracted from commodity smartphone sensors, GruMon can detect over 80% of the groups, with 97% precision, using 10 minutes latency windows, even in venues with limited or no location information. Moreover, in venues where location information is available, GruMon improves the detection latency by up to 20% using semantic information and additional sensors to complement traditional spatio-temporal clustering approaches. We evaluated GruMon on data collected from 258 shopping episodes from 154 real participants, in two large shopping complexes in Korea and Singapore. We also tested GruMon on a large-scale dataset from an international airport (containing ≈37K+ unlabelled location traces per day) and a live deployment at our university, and showed both GruMon's potential performance at scale and various scalability challenges for real-world dense environment deployments.

Existing algorithms for trajectory-based clustering usually rely on simplex representation and a single proximity-related distance (or similarity) measure. Consequently, additional information markers (e.g., social interactions or the semantics of the spatial layout) are usually ignored, leading to the inability to fully discover the communities in the trajectory database. This is especially true for human-generated trajectories, where additional fine-grained markers (e.g., movement velocity at certain locations, or the sequence of semantic spaces visited) can help capture latent relationships between cluster members. To address this limitation, we propose TODMIS: a general framework for Trajectory cOmmunity Discovery using Multiple Information Sources. TODMIS combines additional information with raw trajectory data and creates multiple similarity metrics. In our proposed approach, we first develop a novel approach for computing semantic level similarity by constructing a Markov Random Walk model from the semantically-labeled trajectory data, and then measuring similarity at the distribution level. In addition, we also extract and compute pair-wise similarity measures related to three additional markers, namely trajectory level spatial alignment (proximity), temporal patterns and multi-scale velocity statistics. Finally, after creating a single similarity metric from the weighted combination of these multiple measures, we apply dense sub-graph detection to discover the set of distinct communities. We evaluated TODMIS extensively using traces of (i) student movement data in a campus, (ii) customer trajectories in a shopping mall, and (iii) city-scale taxi movement data. Experimental results demonstrate that TODMIS correctly and efficiently discovers the real grouping behaviors in these diverse settings.

In this paper, we present the design and implementation of Focus, a system for effectively and efficiently reducing power consumption of OLED displays on smartphones. These displays, while becoming exceedingly common still consume significant power. The key idea of Focus is that we use the notion of saliency to save display power by dimming portions of the applications that are less important to the user. We envision Focus being especially useful during low battery situations when usability is less important than power savings. We tested Focus using 15 applications running on a Samsung Galaxy S III and show that it saves, on average, between 23 to 34% of the OLED display power with little impact on task completion times. Finally, we present the results of a user study, involving 30 participants that shows that Focus, even with its dimming behaviour, is still quite usable.

This paper describes the topic and participants of the MDM 2013 panel.

Many emerging context-aware mobile applications involve the execution of continuous queries over sensor data streams generated by a variety of on-board sensors on multiple personal mobile devices (aka smartphones). To reduce the energy-overheads of such large-scale, continuous mobile sensing and query processing, this paper introduces CQP, a collaborative query processing framework that exploits the overlap (in both the sensor sources and the query predicates) across multiple smartphones. The framework automatically identifies the shareable parts of multiple executing queries, and then reduces the overheads of repetitive execution and data transmissions, by having a set of `leader' mobile nodes execute and disseminate these shareable partial results. To further reduce energy, CQP utilizes lower-energy short-range wireless links (such as Bluetooth) to disseminate such results directly among proximate smartphones. We describe algorithms to support our server-assisted distributed query sharing and optimization strategy. Simulation experiments indicate that this approach can result in 60% reduction in the energy overhead of continuous query processing; when `leader' selection is dynamically rotated to equitably share the burden, we observe an increase of up to 65% in operational lifetime.

Advertisements are the de-facto currency of the Internet with many popular applications (e.g. Angry Birds) and online services (e.g., YouTube) relying on advertisement generated revenue. However, the current economic models and mechanisms for mobile advertising are fundamentally not sustainable and far from ideal. In particular, as we show, applications which use mobile advertising are capable of using significant amounts of a mobile users' critical resources without being controlled or held accountable. This paper seeks to redress this situation by enabling advertisement supported applications to become significantly more ``user-friendly''. To this end, we present the design and implementation of CAMEO, a new framework for mobile advertising that 1) employs intelligent and proactive retrieval of advertisements, using context prediction, to significantly reduce the bandwidth and energy overheads of advertising, and 2) provides a negotiation protocol and framework that empowers applications to subsidize their data traffic costs by ``bartering'' their advertisement rights for access bandwidth from mobile ISPs. Our evaluation, that uses real mobile advertising data collected from around the globe, demonstrates that CAMEO effectively reduces the resource consumption caused by mobile advertising.

We propose a hybrid approach for recognizing complex Activities of Daily Living that lie between the two extremes of intensive use of body-worn sensors and the use of infrastructural sensors. Our approach harnesses the power of infrastructural sensors (e.g., motion sensors) to provide additional `hidden' context (e.g., room-level location) of an individual and combines this context with smartphone-based sensing of micro-level postural/locomotive states. The major novelty is our focus on multi-inhabitant environments, where we show how spatiotemporal constraints can be used to significantly improve the accuracy and computational overhead of traditional coupled-HMM based approaches. Experimental results on a smart home dataset demonstrate that this approach improves the accuracy of complex ADL classification by over 30% compared to pure smartphone-based solutions.

This paper describes our experiences and observations with a localization system that continuously tracks the indoor location of a large number of consumer mobile devices. Unlike past work that focuses principally on the accuracy of the location tracking algorithm, we study the performance of the localization system in terms of key additional metrics: scalability and energy-efficiency, which can sometimes conflict with the desire for high accuracy. To ensure that our solution can handle both Android and iOS-based mobile devices (& other closed mobile platforms), we adapt the conventional client-side fingerprinting-based localization approaches to develop a novel and practical infrastructure-based location tracking strategy. We study the relative accuracy to the two approaches in two different types of indoor buildings. Our studies establish how the building and its occupancy characteristics affect the accuracy achievable by different algorithms, and provide insights into why scalable, energy efficient and accurate indoor location tracking remains a challenge in practice.

No abstract available.

In-network Processing, involving operations such as filtering, compression, and fusion is a technique widely used in wireless sensor and ad hoc networks for reducing the communication overhead. In many tactical stream-oriented applications, especially in military scenarios, both link bandwidth and node energy are critically constrained resources. For such applications, in-network processing itself imposes nonnegligible computing cost. In this work, we have developed a unified, utility-based closed-loop control framework that permits distributed convergence to both 1) the optimal level of compression performed by a forwarding node on streams, and 2) the best set of nodes where the operators of the stream processing graph should be deployed. We also show how the generalized model can be adapted to more realistic cases, where the in-network operator may be varied only in discrete steps, and where a fusion operation cannot be fractionally distributed across multiple nodes. Finally, we provide a real-time implementation of the protocol on an 802.11b network with a video application and show that the performance of the network is improved significantly in terms of the packet loss, node lifetime, and quality of video received.

This paper studies video multicasting in large-scale areas using wireless mesh networks. The focus is on the use of Internet access gateways that allow a choice of alternative routes to avoid potentially lengthy and low-capacity multihop wireless paths. A set of heuristic-based algorithms is described that together aim to maximize reliable network capacity: the two-tier integrated architecture algorithm, the weighted gateway uploading algorithm, the link-controlled routing tree algorithm, and the dynamic group management algorithm. These algorithms use different approaches to arrange nodes involved in video multicasting into a clustered and two-tier integrated architecture in which network protocols can make use of multiple gateways to improve system throughput. Simulation results are presented, showing that our multicasting algorithms can achieve up to 40 percent more throughput than other related published approaches.

Both bandwidth and energy become important resource constraints when multihop wireless networks are used to transport high-data-rate traffic for a moderately long duration. In such networks, it is important to control the traffic rates to not only conform to the link capacity bounds, but also to ensure that the energy of battery-powered forwarding nodes is utilized judiciously to avoid premature exhaustion (i.e., the network lasts as long as the applications require data from the sources) without being unnecessarily conservative (i.e., ensuring that the applications derive the maximum utility possible). Unlike prior work that focuses on the instantaneous distributed optimization of such networks, we consider the more challenging question of how such optimal usage of both link capacity and node energy may be achieved over a time horizon. Our key contributions are twofold. We first show how the formalism of optimal control may be used to derive optimal resource usage strategies over a time horizon, under a variety of both deterministic and statistically uncertain variations in various parameters, such as the duration for which individual applications are active or the time-varying recharge characteristics of renewable energy sources (e.g., solar cell batteries). In parallel, we also demonstrate that these optimal adaptations can be embedded, with acceptably low signaling overhead, into a distributed, utility-based rate adaptation protocol. Simulation studies, based on a combination of synthetic and real data traces, validate the close-to-optimal performance characteristics of these practically realizable protocols.

This article develops a utility-based optimization framework for resource sharing by multiple competing missions in a mission-oriented wireless sensor network (WSN) environment. Prior work on network utility maximization (NUM) based optimization has focused on unicast flows with sender-based utilities in either wireline or wireless networks. In this work, we develop a generalized NUM model to consider three key new features observed in mission-centric WSN environments: i) the definition of the utility of an individual mission (receiver) as a joint function of data from multiple sensor sources; ii) the consumption of each sender's (sensor) data by multiple missions; and iii) the multicast-tree-based dissemination of each sensor's data flow, using link-layer broadcasts to exploit the “wireless broadcast advantage” in data forwarding. We show how a price-based, distributed protocol (WSN-NUM) can ensure optimal and proportionally fair rate allocation across multiple missions, without requiring any coordination among missions or sensors. We also discuss techniques to improve the speed of convergence of the protocol, which is essential in an environment as dynamic as the WSN. Further, we analyze the impact of various network and protocol parameters on the bandwidth utilization of the network, using a discrete-event simulation of a stationary wireless network. Finally, we corroborate our simulation-based performance results of the WSN-NUM protocol with an implementation of an 802.11b network.

We propose a routing metric for enabling high-throughput reliable multicast in multi-rate wireless mesh networks. This new multicast routing metric, called expected multicast transmission time (EMTT), captures the combined effects of 1) MAC-layer retransmission-based reliability, 2) transmission rate diversity, 3) wireless broadcast advantage, and 4) link quality awareness. The EMTT of one-hop transmission of a multicast packet minimizes the amount of expected transmission time (including that required for retransmissions). This is achieved by allowing the sender to adapt its bit-rate for each ongoing transmission/retransmission, optimized exclusively for its next-hop receivers that have not yet received the multicast packet. We model the rate adaptation process as a Markov decision process (MDP) and derive an efficient procedure for computing EMTT from the theory of MDP. We present receiver-initiated algorithms and describe protocol implementation for the EMTT-based multicast routing problem. Numerical results are presented to demonstrate the accuracy of the proposed algorithms against optimal solutions to the multicast routing problem. Simulation experiments confirm that, in comparison with single-rate multicast, multi-rate multicast using the EMTT metric effectively reduces the overall multicast transmission time while yielding higher packet delivery ratio and lower end-to-end latency.

Many pervasive applications, such as activity recognition or remote wellness monitoring, utilize a personal mobile device (aka smart phone) to perform continuous processing of data streams acquired from locally-connected, wearable, sensors. To ensure the continuous operation of such applications on a battery-limited mobile device, it is essential to dramatically reduce the energy overhead associated with the process of sensor data acquisition and processing. To achieve this goal, this paper introduces a technique of 'acquisition-cost' aware continuous query processing, as part of the Acquisition Cost-Aware Query Adaptation (ACQUA) framework. ACQUA replaces the current paradigm, where the data is typically streamed (pushed) from the sensors to the smart phone, with a pull-based asynchronous model, where the phone retrieves appropriate blocks of sensor data from individual sensors, only when the stream elements are judged to be relevant to the query being processed. We describe algorithms that dynamically optimize the sequence (for complex stream queries with conjunctive and disjunctive predicates) in which such sensor data streams are retrieved by the phone, based on a combination of the communication cost and selectivity properties of individual sensor streams. Simulation experiments indicate that this approach can result in 70% reduction in the energy overhead of continuous query processing, without affecting the fidelity of the processing logic.

In pervasive computing environments, understanding the context of an entity is essential for adapting the application behavior to changing situations. In our view, context is a high-level representation of a user or entity's state and can capture location, activities, social relationships, capabilities, etc. Inherently, however, these high-level context metrics are difficult to capture using uni-modal sensors only, and must therefore be inferred with the help of multi-modal sensors. However a key challenge in supporting context-aware pervasive computing environments, is how to determine in an energy-efficient manner multiple (potentially competing) high-level context metrics simultaneously using low-level sensor data streams about the environment and the entities present therein. In this paper, we first highlight the intricacies of determining multiple context metrics as compared to a single context, and then develop a novel framework and practical implementation for this problem. The proposed framework captures the tradeoff between the accuracy of estimating multiple context metrics and the overhead incurred in acquiring the necessary sensor data stream. In particular, we develop a multi-context search heuristic algorithm that computes the optimal set of sensors contributing to the multi-context determination as well as the associated parameters of the sensing tasks. Our goal is to satisfy the application requirements for a specified accuracy at a minimum cost. We compare the performance of our heuristic based framework with a brute-forced approach for multi-context determination. Experimental results with SunSPOT sensors demonstrate the potential impact of the proposed framework.

This paper presents MediAlly, a middleware for supporting energy-efficient, long-term remote health monitoring. Data is collected using physiological sensors and transported back to the middleware using a smart phone. The key to MediAlly's energy efficient operations lies in the adoption of an Activity Triggered Deep Monitoring (ATDM) paradigm, where data collection episodes are triggered only when the subject is determined to possess a specified context. MediAlly supports the on-demand collection of contextual provenance using a novel low-overhead provenance collection sub-system. The behaviour of this sub-system is configured using an application-defined context composition graph. The resulting provenance stream provides valuable insight while interpreting the ‘episodic’ sensor data streams. The paper also describes our prototype implementation of MediAlly using commercially available devices.

Presence, broadly defined as an event publish-notification infrastructure for converged applications, has emerged as a key mechanism for collecting and disseminating context attributes for next-generation services in both enterprise and provider domains. Current presence-based solutions and products lack in the ability to a) support flexible user-defined queries over dynamic presence data and b) derive composite presence from multiple provider domains. Accordingly, current uses of context are limited to individual domains/organizations and do not provide a programmable mechanism for rapid creation of context-aware services. This paper describes a presence virtualization architecture, where a Virtualized Presence Server receives customizable queries from multiple presence clients, retrieves the necessary data from the base presence servers, applies the required virtualization logic and notifies the presence clients. To support both query expressiveness and computational efficiency, virtualization queries are structured to separately identify both the XSLT-based transformation primitives and the presence sources over which the transformation occurs. For improved scalability, the proposed architecture offloads the XSLT-related processing to a high-performance XML processing engine. We describe our current implementation and present performance results that attest to the promise of this virtualization approach.

Wireless sensor networks are usually equipped with single radio interfaces where a sensor node can participate in only one transmission at a time. As a result, a node's airtime is ¿shared¿ by multiple flows that pass through the node. Although it is practically important, how to effectively allocate airtime among flows has not received sufficient research attention in the existing literature. In this paper, after showing the potential gain from adaptive airtime allocation in alleviating network congestions, we formulate a new congestion control problem using network utility maximization with respect to airtime fractions, transmission power and flow rate. We prove that the new congestion control is a concave problem and develop a local algorithm for adaptively tuning airtime fractions in parallel with power and rate. Simulation results show the convergence of the optimal airtime allocation as well as its effectiveness in boosting flow rate and conserving power.

Both bandwidth and energy become important resource constraints when multi-hop wireless networks are used to transport relatively high data rate sensor flows. A particularly challenging problem involves the selection of flow data rates that maximize application (or mission) utilities over a time horizon, especially when different missions are active over different time intervals. Prior works on utility driven adaptation of flow data rates typically focus only on instantaneous utility maximization and are unable to address this temporal variation in mission durations. In this work, we derive an optimal control-based Network Utility Maximization (NUM) framework that is able to maximize the system utility over a lifetime that is known either deterministically or statistically. We first consider a static setup in which all the missions are continuously active for a deterministic duration, and show how the rates can be optimally adapted, via a distributed protocol, to maximize the total utility. Next, we develop adaptive protocols for the dynamic cases when we have (i) complete knowledge about the mission utilities and their arrivals and departures, and (ii) a varying amount of statistical information about the missions. Our simulation results indicate that our protocols are robust, efficient and close to the optimal.

This paper addresses the problem of "efficientrdquo broadcast in a multiradio, multichannel, multirate wireless mesh network (MR2-MC WMN). In such an MR2-MC WMN, nodes are equipped with multiple radio interfaces, tuned to orthogonal channels, that can dynamically adjust their transmission rate by choosing a modulation scheme appropriate for the channel conditions. We choose "broadcast latency,rdquo defined as the maximum delay between a packet's network-wide broadcast at the source and its eventual reception at all network nodes, as the ldquoefficiencyrdquo metric of broadcast performance. We study in this paper how the availability of multirate transmission capability and multiple radio interfaces tuned to orthogonal channels in MR2-MC WMN nodes can be exploited, in addition to the medium's ldquowireless broadcast advantagerdquo (WBA), to improve the ldquobroadcast latencyrdquo performance. In this paper, we present four heuristic solutions to our considered problem. We present detailed simulation results for these algorithms for an idealized scheduler, as well as for a practical 802.11-based scheduler. We also study the effect of channel assignment on broadcast performance and show that channel assignment can affect the broadcast performance substantially. More importantly, we show that a channel assignment that performs well for unicast does not necessarily perform well for broadcast/multicast.

This paper studies video multicasting in large scale areas using wireless mesh networks. The focus is on the use of Internet access gateways that allow a choice of alternative routes to avoid potentially lengthy multi-hop wireless paths with low capacity. A set of heuristic-based algorithms are described that together aim to maximize network capacity: the two-tier integrated architecture algorithm, the weighted gateway uploading algorithm, the link-controlled routing tree algorithm, and the alternative channel assignment algorithm. These algorithms use different approaches to arrange multicast group members into a clustered and two-tier integrated architecture in which network protocols can make use of multiple gateways to improve system throughput. Simulation results are used to determine the performance of the different approaches.

A promising architecture for remote healthcare monitoring involves the use of a pervasive device (such as a cellular phone), which aggregates data from multiple body-worn medical sensors and transmits the data to the backend. Unfortunately, the volume of data generated by increasingly sophisticated continuously- active sensors can overwhelm the resources on the mobile device. We propose imbuing the mobile device with the intelligence to perform context-aware filtering of sensor data streams in order to reduce transmissions in cases where the observed data corresponds to the norm expected by the system in a given context. To investigate the efficacy of this technique, we implemented the HARMONI middleware on a mobile device, and used it to collect real sensor data from users. Our experiments demonstrate that context-aware filtering can reduce the uplink bandwidth requirements of the system by up to 72%.

We extend the existing network utility maximization (NUM) framework for wired networks to wireless sensor networks by formulating it in order to take into account interference among radio links. We study the conditions under which the formulated problem is a feasible convex optimization problem. Under such conditions, a distributed algorithm is proposed to solve the problem optimally. Finally, we provide numerical results, based on computer simulations, to show the performance of the proposed algorithm and the rate of convergence of its solution.

This paper extends the distributed network utility maximization (NUM) framework to consider the case of resource sharing by multiple competing missions in a military-centric wireless sensor network (WSN) environment. Prior work on NUM-based optimization has considered unicast flows with sender-based utilities in either wireline or wireless networks. We extend the NUM framework to consider three key new features observed in mission-centric WSN environments: i) the definition of an individual mission's utility as a joint function of data from multiple sensor sources ii) the consumption of each senders (sensor) data by multiple receivers (missions) and iii) the multicast-tree based dissemination of each sensors data flow, using link-layer broadcasts to exploit the "wireless broadcast advantage" in data forwarding. We show how a receiver-centric, pricing-based, decentralized algorithm can ensure optimal and proportionally-fair rate allocation across the multiple missions, without requiring any coordination among independent missions (or sensors). We also discuss techniques to improve the speed of convergence of the protocol, which is essential in an environment as dynamic as the WSN.

A particularly compelling vision of long-term remote health monitoring advocates the use of a personal pervasive device (such as a cellphone) as an intermediate relay, which transports data streams from multiple body-worn sensors to a backend analytics infrastructure. Unfortunately, a pure relay-based functionality on the cellphone is inadequate in the longer term, as increasingly sophisticated medical sensors impose unnacceptably high uplink traffic and energy consumption costs on the mobile device. To address this challenge, we are building an event-processing middleware, called HARMONI, which enables the pervasive device to perform context-aware processing and event filtering on the sensor data streams and locally extract higher-level features of interest, thereby reducing the volume of transmitted data. This paper presents the design and architectural components of HARMONI, with special emphasis on its implementation of context-aware event processing. This paper then demonstrates that the mobile device can extract localized context from the incoming sensor stream with sufficient accuracy to achieve satisfactory context-aware filtering. Our results also establish the need for personalizing such context extraction, as they show that similar sensor data patterns obtained from different individuals can imply significantly different activity contexts.

We address the problem of minimizing the worst-case broadcast delay in ldquomulti-radio multi-channel multi-rate wireless mesh networksrdquo (MR2-MC WMN) in a distributed and localized fashion. Efficient broadcasting in such networks is especially challenging due to the desirability of exploiting the ldquowireless broadcast advantagerdquo (WBA), the interface-diversity, the channel-diversity and the rate-diversity offered by these networks. We propose a framework that calculates a set of forwarding nodes and transmission rate at these forwarding nodes irrespective of the broadcast source. Thereafter, a forwarding tree is constructed taking into consideration the source of broadcast. Our broadcasting algorithms are distributed and utilize locally available information. We present a detailed performance evaluation of our distributed and localized algorithm and demonstrate that our algorithm can greatly improve broadcast performance by exploiting the rate, interface and channel diversity of MR2-MC WMNs and match the performance of centralized algorithms proposed in literature while utilizing only limited two-hop neighborhood information.

This paper studies probabilistically reliable multicast in wireless mesh networks (WMNs), utilizing MAC layer re-transmission and wireless broadcast advantage to improve both the multicast throughput and the delivery rate. We first present a new multicast routing metric which we call the expected multicast transmissions (EMT). EMT captures the effect of link packet delivery ratio, MAC layer retransmission and wireless broadcast advantage at the same time. The EMT of a MAC layer multicast transmission is the expected number of data transmissions (including retransmissions) required for a packet to reach all the recipients. The EMT of a multicast tree is the sum over the EMT of each forwarding node. Then, we propose a probabilistically reliable on-demand (PROD) multicast protocol with the objective of minimizing the EMT of the multicast tree. Simulation results show that, in comparison with existing approaches, PROD reduces the end-to-end packet loss ratio by up to 30% and improves the multicast throughput by up to 25%. In addition, it reduces the number of transmissions per packet by up to 40% and thus significantly reduces the network overhead of the multicast session.

Remote health monitoring affords the possibility of improving the quality of health care by enabling relatively inexpensive out-patient care. However, remote health monitoring raises new a problem: the potential for data explosion in health care systems. To address this problem, the remote health monitoring systems must be integrated with analysis tools that provide automated trend analysis and event detection in real time. In this paper, we propose an overview of Century, an extensible framework for analysis of large numbers of remote sensor-based medical data streams.

This paper studies some of the fundamental challenges and opportunities associated with the network-layer broadcast and multicast in a multihop multirate wireless mesh network (WMN). In particular, we focus on exploiting the ability of nodes to perform link-layer broadcasts at different rates (with correspondingly different coverage areas). We first show how, in the broadcast wireless medium, the available capacity at a mesh node for a multicast transmission is not just a function of the aggregate pre-existing traffic load of other interfering nodes, but intricately coupled to the actual (sender, receiver) set and the link-layer rate of each individual transmission. We then present and study four alternative heuristic strategies for computing a broadcast tree that not only factors in a flow's traffic rate but also exploits the wireless broadcast advantage (WBA). Finally, we demonstrate how our insights can be extended to multicast routing in a WMN, and present results that show how a tree-formation algorithm that combines contention awareness with transmission rate diversity can significantly increase the total amount of admissible multicast traffic load in a WMN.

We address the problem of minimizing the worst-case broadcast delay in multi-rate wireless mesh networks (WMN) in a distributed and localized fashion. Efficient broadcasting in such networks is especially challenging due to the multi-rate transmission capability and the interference between wireless transmissions of WMN nodes. We propose connecting dominating set (CDS) based broadcast routing approach which calculates the set of forwarding nodes and the transmission rate at each forwarding node independent of the broadcast source. Thereafter, a forwarding tree is constructed taking into consideration the source of the broadcast. In this paper, we propose three distributed and localized rate-aware broadcast algorithms. We compare the performance of our distributed and localized algorithms with previously proposed centralized algorithms and observe that the performance gap is not large. We show that our algorithms greatly improve performance of rate-unaware broadcasting algorithms by incorporating rate-awareness into the broadcast tree construction algorithm process.

In this paper, we propose two information-theoretic techniques for efficiently trading off the location update and paging costs associated with mobility management in wireless cellular networks. Previous approaches always attempt to accurately convey a mobile's movement sequence and hence cannot reduce the signaling cost below the entropy bound. Our proposed techniques, however, exploit the rate distortion theory to arbitrarily reduce the update cost at the expense of an increase in the corresponding paging overhead. To this end, we describe two location tracking algorithms based on spatial quantization and temporal quantization, which first quantize the movement sequence into a smaller set of codewords and then report a compressed representation of the codeword sequence. Although the spatial quantization algorithm clusters individual cells into registration areas, the more powerful temporal quantization algorithm groups sets of consecutive movement patterns. The quantizers themselves are adaptive and periodically reconfigure to accommodate changes in the mobile's movement pattern. Simulation study with synthetic and real movement traces for both single-system and multisystem cellular networks demonstrate that the proposed algorithms can reduce the mobile's update frequency to 3-4 updates/day with reasonable paging cost, low computational complexity, storage overhead, and codebook updates.

A multi-rate capable IEEE 802.11a/b/g node can utilize different link-layer transmission rates. Interestingly, multi-rate capability is defined by IEEE802.11 standards only for unicast transmissions. In this paper, we consider a novel type of multi-radio multi-channel wireless mesh network (WMN) where a radio can multicast at different link-layer transmission rates to its neighbors. Such link-layer multi-rate multicast capability will enable low-latency network-layer broadcast/multicast for multimedia. In our previous work, we assumed a "fully multi-rate multicast" (EMM) framework in which nodes can adjust link-layer multicast transmission rate for each link-layer frame. We propose a new framework called "single best-rate multicast" (SBM) that exploits the link-layer rate diversity by enabling each WMN to decide, depending on its topological properties, a single transmission rate for all its link-layer data multicasts. Although, EMM improves performance significantly, employing SBM is attractive since it can eliminate some undesirable features of practical multi-rate media access control (MAC) protocols. We propose methods to determine the "best" link-layer transmission rate for the SBM framework. We also propose two heuristic broadcast solutions, using SBM framework that can realize low-latency broadcast by exploiting inherent rate and interface diversity in multi-radio multi-channel WMN. Simulation results indicate that SBM broadcast heuristics give comparable performance to EMM broadcast heuristics, especially in dense networks

Next-generation converged networks shall deliver many innovative services over the standardized SIPbased IMS signaling infrastructure. Several such services exploit the joint presence information of a consumer, i.e. SIP entity requesting a service, and a vendor, i.e. SIP resource providing a service. Presence information is a collection of contextual attributes (e.g. location, availability, reputation), some of which change dynamically. Moreover, this collective presence information is distributed across multiple presence servers. While performing query matching based on joint presence information, a server usually routes each query to a locally available resource. However, skews in the spatio-temporal distribution of queries and resources may require queries to be routed to alternate servers with available resources. We propose a novel Resource-Aware Query Routing scheme, called RAQR, where each server proactively establishes gradients to suitable servers via a diffusion-based algorithm. Gradients are set up whenever a server anticipates scarcity of resources and withdrawn when the resource crunch is mitigated. We compare RAQR with alternative resource matching schemes and show that it adapts to spatio-temporal variations in resource availability, thereby leading to effective query matching with minimal control overhead.

In this paper, we design and evaluate an energy efficient data retrieval architecture for continuous aggregate queries in wireless sensor networks. We show how the modification of precision in one sensor affects the sample-reporting frequency of other sensors, and how the precisions of a group of sensors may be collectively modified to achieve the target quality of information (QoI) with higher energy-efficiency. The proposed collective adaptive precision setting (CAPS) architecture is then extended to exploit the observed temporal correlation among successive sensor samples for even greater energy efficiency. Detailed simulations with synthetic and real data traces demonstrate how the combination of weak consistency semantics and temporal correlation can dramatically lower the energy consumption in practical sensor environments

We address the problem of minimizing the worst-case broadcast delay in multi-radio multi-channel multi-rate (MR2-MC) wireless mesh networks (WMN). The problem of 'efficient' broadcast in such networks is especially challenging due to the numerous interrelated decisions that have to be made. The multi-rate transmission capability of WMN nodes, interference between wireless transmissions, and the hardness of optimal channel assignment adds complexity to our considered problem. We present four heuristic algorithms to solve the minimum latency broadcast problem for such settings and show that the 'best' performing algorithms usually adapt themselves to the available radio interfaces and channels. We also study the effect of channel assignment on broadcast performance and show that channel assignment can affect the broadcast performance substantially. More importantly, we show that a channel assignment that performs well for unicast does not necessarily perform well for broadcast/multicast. To the best of our knowledge, this work constitutes the first contribution in the area of broadcast routing for MR2-MC WMN

This paper addresses the problem of efficient packet forwarding in a multihop, wireless "mesh" network. We present an efficient interface contained forwarding (ICF) architecture for a "wireless router," i.e., a forwarding node with a single wireless network interface card (NIC) in a multihop wireless network that allows a packet to be forwarded entirely within the NIC of the forwarding node without requiring per-packet intervention by the node's CPU. To effectively forward packets in a pipelined fashion without incurring the 802.11-related overheads of multiple independent channel accesses, we specify a slightly modified version of the 802.11 MAC, called data driven cut-through multiple access (DCMA) that uses multiprotocol label switching (MPLS)-like labels in the control packets, in conjunction with a combined ACK/RTS packet, to reduce 802.11 channel access latencies. Our proposed technique can be used in combination with "frame bursting" as specified by the IEEE 802.11e standard to provide an end-to-end cut-through channel access. Using extensive simulations, we compare the performance of DCMA with 802.11 DCF MAC with respect to throughput and latency and suggest a suitable operating region to get maximum benefits using our mechanism as compared to 802.11

In a multirate wireless network, a node can dynamically adjust its link transmission rate by switching between different modulation schemes. In the current IEEE802.11a/b/g standards, this rate adjustment is defined for unicast traffic only. In this paper, we consider a wireless mesh network (WMN), where a node can dynamically adjust its link-layer multicast rates to its neighbors, and address the problem of realizing low-latency network-wide broadcast in such a mesh. We first show that the multirate broadcast problem is significantly different from the single-rate case. We will then present an algorithm for achieving low-latency broadcast in a multirate mesh which exploits both the wireless multicast advantage and the multirate nature of the network. Simulations based on current IEEE802.11 parameters show that multirate multicast can reduce broadcast latency by 3-5 times compared with using the lowest rate alone. In addition, we show the significance of the product of transmission rate and transmission coverage area in designing multirate WMNs for broadcast

This paper presents an architecture to facilitate efficient evaluation and selection of 3rd party Web services for service providers. Most service provider architectures have primarily focused on providing Web service front ends to legacy systems, aggregating and delivering services via workflows. These architectures primarily considered static business contracts between the service provider and its (Web-service enabled) business partners. This approach makes these architectures inflexible to variations in business requirement, partners' performance and customer requirements. Our architecture provides a flexible means for service providers to optimize business performance. Based on the historical performance, extant context, and optimising business rules, the appropriate service is selected and invoked to serve a customer request. We have developed a prototype system ODSS-in-ENDS, that demonstrates this capability.

In the absence of generic programming abstractions for dynamic data in most enterprise programming environments, individual applications treat data streams as a special case requiring custom programming. With the growing number of live data sources such as RSS feeds, messaging and presence servers, multimedia streams, and sensor data. a general-purpose client-server programming model is needed to easily incorporate live data into applications. In this paper, we present Live Data Views, a programming abstraction that represents live data as a time-windowed view over a set of data streams. Live Data Views allow applications to create and retrieve stateful abstractions of dynamic data sources in a uniform manner, via the application of intra- and inter- stream operators. We provide details of our model and evaluate a proof-of-concept Live Data Views implementation to monitor traffic conditions on a highway. We also provide the preliminary design of a J2EE-based implementation, and outline some of the research challenges raised by this abstraction in a distributed computing environment.

We address the problem of energy-efficient reliable wireless communication in the presence of unreliable or lossy wireless link layers in multi-hop wireless networks. Prior work [1] has provided an optimal energy efficient solution to this problem for the case where link layers implement perfect reliability. However, a more common scenario --- a link layer that is not perfectly reliable, was left as an open problem. In this paper we first present two centralized algorithms, BAMER and GAMER, that optimally solve the minimum energy reliable communication problem in presence of unreliable links. Subsequently we present a distributed algorithm, DAMER, that approximates the performance of the centralized algorithm and leads to significant performance improvement over existing single-path or multi-path based techniques.

This paper presents the detailed design and performance analysis of MACA-P, a RTS/CTS based MAC protocol, that enables simultaneous transmissions in wireless mesh networks. The IEEE 802.11 DCF MAC prohibits any parallel transmission in the neighborhood of either a sender or a receiver (of an ongoing transmission). MACA-P is a set of enhancements to the 802.11 MAC that allows parallel transmissions in situations when two neighboring nodes are either both receivers or transmitters, but a receiver and a transmitter are not neighbors. The performance of MACA-P in terms of system throughput is obtained through a simulation of the protocol using ns and is compared with the 802,11 RTS/CTS MAC. Experiments with the base MACA-P protocol reveal the need for certain enhancements, especially to avoid the drawbacks associated with attempts at parallel transmissions in scenarios where such parallelism is not feasible. Studies with the enhanced MACA-P protocol also demonstrate how significant performance gains in wireless mesh network performance may be realized if the radio transceiver behavior is modified in tandem with the MAC protocol.

A practical information theoretic framework is developed for studying the optimal tradeoff between location update and paging costs in cellular networks. The framework envisions the quantization of location information into a registration area (RA) level granularity, followed by the use of an entropy-coding technique to decrease the location update rate. The rate distortion theory of the lossy quantization is identified as an appropriate measure for capturing the optimal tradeoff between a mobile's update rate and its location uncertainty. Based on LZ-78 compression, two different RA-level location update algorithms (RA-LeZi and LeZi-RA) have been developed, both of which asymptotically approach this rate-distortion bound. By allowing for quantization loss in the mobile node's movement pattern, this framework can reduce the overall update cost below the entropy bound associated with the original loss-less LeZi-update mobility management algorithm. Simulation results demonstrate a sharp decrease (/spl sim/ 50%) in the update cost, at the expense of a minor (/spl sim/ 25%) increase in the overall location management costs. The key essence of this framework lies in its practical applicability, because today's wireless networks already track the mobile user at an RA-level granularity.

Distributed hash table (DHT) overlay networks offer an efficient and robust technique for wire-area data storage and queries. Workload from real applications that use DHT networks will likely exhibit significant skews that can result in bottlenecks and failures that limit the overall scalability of the DHT approach. We present the content and load-aware scalable hashing (CLASH) protocol that can enhance the load distribution behavior of a DHT. CLASH relies on a variable-length identifier key scheme, where the length of any individual key is a function of load. CLASH uses variable-length keys to cluster content-related objects on single nodes to achieve processing efficiencies, and minimally disperse objects across multiple servers when hotspots occur. We demonstrate the performance benefits of CLASH through analysis and simulation.

An information-theoretic framework is developed for optimal location management in multisystem, fourth generation (4G) wireless networks. The framework envisions that each individual subsystem operates fairly independently, and does not require public knowledge of individual subnetwork topologies. To capture the variation in paging and location update costs in this heterogeneous environment, the location management problem is formulated in terms of a new concept of weighted entropy. The update process is based on the Lempel-Ziv compression algorithms, which are applied to a vector-valued sequence consisting of both the mobile's movement pattern and its session activity state. Three different tracking strategies which differ in their degrees of centralized control and provide trade off between the location update and paging costs, are proposed and evaluated. While both the proposed centralized and distributed location management strategies are endowed with optimal update capability, the proposed selective location management heuristic also offers a practical trade off between update and paging costs. Simulation experiments demonstrate that our proposed schemes can result in more than 50% savings in both update and paging costs, in comparison with the basic movement-based, multisystem location management strategy. These update strategies can be realized with only modest amounts of memory (12-15 Kbytes) on the mobile.

The intra-domain mobility management protocol (IDMP) has recently been proposed as a protocol for managing IP mobility within a cellular access network. The paper investigates the scalability performance of IDMP's quality of service (QoS) framework, which uses a modified form of the differentiated services (DiffServ) architecture, with a centralized bandwidth broker (BB) performing admission control and resource provisioning for different traffic classes. The theoretical analysis shows that requests for bandwidth reservation due to intra-domain mobility should be controlled to alleviate the processing burden at the BB. Accordingly, we propose a scalable bandwidth reservation scheme. By reserving the bandwidth in the trunk instead of on a per-host basis, QoS-related signaling load and handoff latency can be reduced significantly.

Access to data services via wireless LANs at private (e.g., corporations or home) and public "hot-spot" (e.g., hotels and airports) setting is becoming a common place daily. Data access via (for profit) public wireless LAN (PWLAN) installations is typically based on user subscription and preconfigured services profiles pertaining primarily access to the global Internet. The goal of the ts-PWLAN project is to define an architecture and a prototype implementation that enables the provision of premium and non-premium tiers of services to transient and non-transient users. ts-PWLAN provides for dynamic renegotiations of tier of services and enables various billing modes, e.g., based on connectivity time and usage, thus enabling service providers to increase their revenue opportunities via multiple service offerings.

This paper presents the initial design and performance study of MACA-P, a RTS/CTS based MAC protocol that enables simultaneous transmissions in multihop ad-hoc wireless networks. Providing such low-cost multihop and high performance wireless access networks is an important enabler of pervasive computing. MACA-P is a set of enhancements to the 802.11 DCF that allows parallel transmissions in many situations when two neighboring nodes are either both receivers or both transmitters, but a receiver and a transmitter are not neighbors. Like 802.11, MACA-P contains a contention-based reservation phase prior to data transmission. However, the data transmission is delayed by a control phase interval, which allows multiple sender-receiver pairs to synchronize their data transfers, thereby avoiding collisions and improving system throughput.

We define energy-efficient broadband and multicast schemes for reliable communication in multi-hop wireless networks. Unlike previous techniques, the choice of neighbors in the broadband and multicast trees in these schemes, are based not only on the link distance, but also on the error rates associated with the link. Our schemes can be implemented using both positive and negative acknowledgement based reliable broadcast techniques in the link layer. Through simulations, we show that our scheme achieves up to 45% improvement over previous schemes on realistic 100-node network topologies. A positive acknowledgment based implementation is preferred. Our simulations show that the additional benefits of a positive acknowledgement based implementation is marginal (1-2%). Therefore a negative acknowledgement based implementation of our schemes is equally applicable in constructing energy-efficient reliable and multicast data delivery paths.

We use simulations to study the comparative routing overheads of three ad-hoc routing protocols, namely AODV, DSDV and DSR. In contrast to earlier studies, we focus exclusively on the energy consumption and not on other metrics such as the number of routing packets. In particular, we study the 'range effects' of the three protocols, i.e., how changes to the transmission power and transmission radius affect the overall energy consumed by routing-related packets. Due to the broadcast nature of the wireless medium, the energy spent in packet receptions is almost as important as the transmission power; using the number of transmissions as an indicator of the routing overhead can thus be fairly misleading. Our studies show that the energy overhead of the three protocols varies with the transmission power in distinct and non-obvious ways.

We study the dependence of the capacity of multi-hop wireless networks on the transmission range of nodes in the network with TCP regulated traffic. Specifically, we examine the sensitivity of the capacity to the speed of the nodes and the number of TCP connections in an ad hoc network. By incorporating the notion of a minimal acceptable QoS metric (loss) for an individual session, we argue that the QoS-aware capacity is a more accurate model of the TCP-centric capacity of an ad-hoc network. We study the dependence of capacity on the source application (Telnet or FTP) and on the choice of the ad-hoc routing protocol (ad-hoc on-demand distance vector - AODV, dynamic source routing - DSR or destination-sequenced distance vector - DSDV). We conclude that persistent and non-persistent traffic behave quite differently in an ad-hoc network.

The intra-domain mobility management protocol (IDMP) has been proposed as a protocol for managing IP mobility within a cellular access network. This paper evaluates the performance of IDMP's quality of service (QoS) framework, which uses a modified form of the differentiated services architecture, with a centralized bandwidth broker performing admission control and resource provisioning for different traffic classes. While experimental results demonstrate that our IDMP-based QoS mechanism is able to support differential performance guarantees for mobile users with fairly low latency overheads, analytical derivations show that this QoS support can be achieved with only a marginal increase in the signaling cost.

We study the performance metrics associated with TCP-regulated traffic in multi-hop wireless networks that use a common physical channel (e.g., IEEE 802.11). In contrast to earlier analyses, we focus simultaneously on two key operating metrics - the energy efficiency and the session throughput. Using analysis and simulations, we show how these metrics are strongly influenced by the radio transmission range of individual nodes. Due to tradeoffs between the individual packet transmission energy and the likelihood of retransmissions, the total energy consumption is a convex function of the number of hops (and hence, of the transmission range). On the other hand, the TCP session throughput decreases supra-linearly with a decrease in the transmission range. In certain scenarios, the overall network capacity can then be a concave function of the transmission range. Based on our analysis of the performance of an individual TCP session, we finally study how parameters such as the node density and the radio transmission range affect the overall network capacity under different operating conditions. Our analysis shows that capacity metrics at the TCP layer behave quite differently than corresponding idealized link-layer metrics.

Current algorithms for minimum-energy routing in wireless networks typically select minimum-cost multi-hop paths. In scenarios where the transmission power is fixed, each link has the same cost and the minimum-hop path is selected. In situations where the transmission power can be varied with the distance of the link, the link cost is higher for longer hops; the energy-aware routing algorithms select a path with a large number of small-distance hops. In this paper, we argue that such a formulation based solely on the energy spent in a single transmission is misleading --- the proper metric should include the total energy (including that expended for any retransmissions necessary) spent in reliably delivering the packet to its final destination.We first study how link error rates affect this retransmission-aware metric, and how it leads to an efficient choice between a path with a large number of short-distance hops and another with a smaller number of large-distance hops. Such studies motivate the definition of a link cost that is a function of both the energy required for a single transmission attempt across the link and the link error rate. This cost function captures the cumulative energy expended in reliable data transfer, for both reliable and unreliable link layers. Finally, through detailed simulations, we show that our schemes can lead to upto 30-70% energy savings over best known current schemes, under realistic environments.

We propose MRPC, a new power-aware routing algorithm for energy-efficient routing that increases the operational lifetime of multi-hop wireless networks. In contrast to conventional power-aware algorithms, MRPC identifies the capacity of a node not just by its residual battery energy, but also by the expected energy spent in reliably forwarding a packet over a specific link. Such a formulation better captures scenarios where link transmission costs also depend on physical distances between nodes and the link error rates. Using a max-min formulation, MRPC selects the path that has the largest packet capacity at the 'critical' node (the one with the smallest residual packet transmission capacity). We also present CMRPC, a conditional variant of MRPC that switches from minimum energy routing to MRPC only when the packet forwarding capacity of nodes falls below a threshold. Simulation based studies have been used to quantify the performance gains of our algorithms.

We present our implementation of TeleMIP, a two-level architecture for IP-based mobility management. TeleMIP essentially uses an intra-domain mobility management protocol (IDMP) for managing mobility within a domain, and mobile IP for supporting inter-domain (global) mobility. Unlike other proposed schemes for intra-domain mobility management, IDMP uses two care-of addresses for mobility management. The global care-of address is relatively stable and identifies the mobile node's current domain, while the local care-of address changes every time the mobile changes subnets and identifies the mobile's current point of attachment. The paper describes our TeleMIP implementation based on enhancements to the Stanford University mobile IP Linux code and presents performance results obtained through experiments on our test-bed. Finally, we use analysis to accurately quantify the savings in signaling overhead obtained when TeleMIP is used in environments where mobiles change subnets relatively rapidly.

The paper analyzes how using a longer memory of the past queue occupancy in computing the average queue occupancy affects the stability and variability of a RED queue. Extensive simulation studies with both persistent and Web TCP sources are used to study the variance of the RED queue as a function of the memory of the averaging process. Our results show that there is very little performance improvement (and in fact, possibly significant performance degradation) if the length of memory is increased beyond a very small value. Contrary to current practice, our results show that a longer memory reduces the negative correlation typically observed among the windows of the constituent TCP flows, and hence, suggest the use of the instantaneous queue occupancy in practical RED queues.

Abstract This paper considers three applications—VoIP, mobile Web access and mobile server‐based data transfers—and evaluates the applicability of various IP‐based mobility management mechanisms. We first survey the features and characteristics of various IP mobility protocols, such as MIPv4, MIPv6, MIP‐RO, SIP, CIP, HAWAII, MIP‐RR and IDMP, and then evaluate their utility on an application‐specific basis. The diversity in the mobility‐related requirements ensures that no single mobility solution is universally applicable. We recommend a hierarchical mobility architecture. The framework uses our Dynamic Mobility Agent (DMA) architecture for managing intra‐domain mobility and multiple application‐based binding protocols for supporting inter‐domain mobility. Thus, we recommend SIP as the global binding protocol for VoIP applications and MIPv4/MIPv6 as the global binding mechanism for the mobile server scenario. Copyright © 2001 John Wiley & Sons, Ltd.

To model possible suggested changes in TCP window adaptation in response to randomized feedback, such as ECN (explicit congestion notification), we formulate a generalized version of the TCP congestion avoidance algorithm. We first consider multiple such generalized TCP flows sharing a bottleneck buffer under the assured service model and use a fixed point technique to obtain the mean window sizes and throughputs for the TCP flows, To further study how changes in the adaptation algorithm affect the variability in the throughput, we use an analytical-cum-numerical technique to derive the window distribution (and related statistics) of a single generalized flow under state-dependent randomized congestion feedback.

We compare various existing approaches for locating roaming users and providing continuous connectivity in quasi-dynamic networks in which the movement is much more restricted than in ad-hoc networks. However, there are frequent host movements combined with occasional major topology changes (e.g., whole network is changing their point of attachment or getting disconnected). The higher dynamics causes many proposed mobility solutions including basic Mobile IP (which work well in environments limited to host mobility), perform poorly. Based on some qualitative comparisons, we suggest which approaches are the best to handle mobility in quasi-dynamic networks.

This paper analyzes the stationary behavior of the TCP congestion window performing ideal congestion avoidance when the packet loss probability is not constant, but varies as a function of the window size. By neglecting the detailed window behavior during fast recovery, we are able to derive a Markov process that is then approximated by a continuous-time, continuous state space process. The stationary distribution of this process is analyzed and derived numerically and then extrapolated to obtain the stationary distribution of the TCP window. This numerical analysis enables us to predict the behavior of the TCP congestion window when interacting with a router port performing early random drop (or random early detection) where the loss probability varies with the queue occupancy.