Radars are an ideal complement to cameras: both are inexpensive, solid-state sensors, with cameras offering fine angular resolution, while radars provide metric depth and robustness under adverse weather. However, radar data is more difficult to interpret than camera images and varies significantly between sensors, necessitating increased reliance on simulation for prototyping sensors and processing pipelines. Recent work treating radar reconstruction as a novel view synthesis problem has shown great promise in reconstructing radar-relevant geometry and simulating lowlevel radar data. However, such methods are constrained by the low spatial resolution of the underlying radar. To address this, we propose a unified differentiable renderer, RadarSim, which leverages the high angular resolution of RGB cameras to generate Doppler radar range images from a camera-initialized neural field. Using a novel data set of calibrated radar camera recordings from a custom handheld rig, we demonstrate that RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.

This paper introduces GenAssist, a system for generating interactive Extended Reality (XR) programs from natural language prompts. Given plain text descriptions of desired programs, our system uses Retrieval-Augmented Generation (RAG) to retrieve related documentation and example code, which is then used to prompt Large Language Models (LLMs) to generate and execute hot-pluggable XR programs in real time. To ensure that the programs are written correctly to the user’s specifications, we add a closed-loop feedback mechanism using virtual cameras in the scene that iteratively refines the system’s output, mimicking the development cycle of human developers that compile and then interactively test programs. GenAssist generates scripts that can not only place multiple primitives and 3D models in various locations in a virtual scene, but it can also animate and enable user interactions with those objects. We show that across a benchmark of 50 diverse XR program prompts, our system achieves high output accuracy and program generation quality. Furthermore, we conduct a user study with 18 participants that demonstrates GenAssist’s effectiveness and usability (NASA TLX = 36.6) for XR program generation. We compare GenAssist to prior systems and show that it is significantly faster (<10 seconds per run) and requires fewer LLM calls.

The rapid adoption of volumetric capture technologies has created a pressing need for efficient storage and streaming of dynamic 3D content. Unfortunately, current compression standards often treat dynamic sequences as independent frames or rely on computationally expensive non-rigid registration, making them unsuitable for real-time applications or large-scale environments. In this paper, we present a novel end-to-end compression framework for dynamic Truncated Signed Distance Field volumes derived from captured 3D content, leveraging a representation that is temporally stable, easily parallelizable, and already widely used in scene reconstruction and volumetric fusion pipelines. We then adapt classic 2D video coding paradigms such as spatial coding via Discrete Cosine Transform and temporal coding using a real-time motion compensation pipeline to provide robust, real-time, and training-free encoding and decoding for 3D content. Extensive evaluations on human performance captures demonstrate that our codec achieves ~35% bitrate savings at equal distortion while operating in real time at 30 FPS, while stronger temporal coherence in large-scale synthetic environments yields up to 12× bitrate reduction at equal distortion.

Volumetric video and 6-DoF scene capture are becoming central to immersive applications such as telepresence and mixed reality content delivery. However, existing volumetric datasets are often short in duration, restricted to studio-captured human subjects, and provide only limited geometric representations. Consequently, evaluating real-world immersive applications in full-scene contexts often necessitates custom capture and 3D reconstruction setups, creating high practical barriers and ultimately hindering reproducibility. To this end, we present SceneHub4D, a new dataset and evaluation framework. Our dataset captures long, dynamic sequences across diverse real-world indoor environments with synchronized multi-view RGB-D streams, calibrated camera poses, and high-resolution background geometry reconstructed via photogrammetry and LiDAR. We provide multiple 3D representations, including point clouds, textured meshes, and Gaussian splats, along with a software toolkit for format conversion, rendering, and metric evaluation. To support structured comparison and perceptual analysis, we provide supplementary metrics including Geometry Complexity Score and Volumetric Temporal Information, and evaluate rendering performance across desktop GPUs and VR headsets. By lowering the practical barriers to capture, reconstruction, and evaluation, SceneHub4D enables researchers to study immersive 3D streaming and rendering systems without requiring custom hardware setups or complex data collection pipelines. We expect it will serve as a useful foundation for advancing volumetric media research.

Wasm is gaining popularity outside the Web as a well-specified low-level binary format with ISA portability, low memory footprint and polyglot targetability, enabling efficient in-process sandboxing of untrusted code. Despite these advantages, Wasm adoption for new domains is often hindered by the lack of many standard system interfaces which precludes reusability of existing software and slows ecosystem growth.

The emergence of the Spatial Web -- the Web where content is tied to real-world locations has the potential to improve and enable many applications such as augmented reality, navigation, robotics, and more. The Spatial Web is missing a key ingredient that is impeding its growth -- a spatial naming system to resolve real-world locations to names. Today's spatial naming systems are digital maps such as Google and Apple maps. These maps and the location-based services provided on top of these maps are primarily controlled by a few large corporations and mostly cover outdoor public spaces. Emerging classes of applications, such as persistent world-scale augmented reality, require detailed maps of both outdoor and indoor spaces. Existing centralized mapping infrastructures are proving insufficient for such applications because of the scale of cartography efforts required and the privacy of indoor map data.

mmWave radars are compact, inexpensive, and durable sensors that are robust to occlusions and work regardless of environmental conditions, such as weather and darkness. However, this comes at the cost of poor angular resolution, especially for inexpensive single-chip radars, which are typically used in automotive and indoor sensing applications. Although many have proposed learning-based methods to mitigate this weakness, no standardized foundational models or large datasets for the mmWave radar have emerged, and practitioners have largely trained task-specific models from scratch using relatively small datasets. In this paper, we collect (to our knowledge) the largest available raw radar dataset with 1M samples (29 hours) and train a foundational model for 4D single-chip radar, which can predict 3D occupancy and semantic segmentation with quality that is typically only possible with much higher resolution sensors. We demonstrate that our Generalizable Radar Transformer (GRT) generalizes across diverse settings, can be fine-tuned for different tasks, and shows logarithmic data scaling of 20\% per $10\times$ data. We also run extensive ablations on common design decisions, and find that using raw radar data significantly outperforms widely-used lossy representations, equivalent to a $10\times$ increase in training data. Finally, we roughly estimate that $\approx$100M samples (3000 hours) of data are required to fully exploit the potential of GRT.

World-scale augmented reality (AR) applications need a ubiquitous 6DoF localization backend to anchor content to the real world consistently across devices. Large organizations such as Google and Niantic are 3D scanning outdoor public spaces in order to build their own Visual Positioning Systems (VPS). These centralized VPS solutions fail to meet the needs of many future AR applications-they do not cover private indoor spaces because of privacy concerns, regulations, and the labor bottleneck of updating and maintaining 3D scans. In this paper, we present OpenFLAME, a federated VPS backend that allows independent organizations to 3D scan and maintain a separate VPS service for their own spaces. This enables access control of indoor 3D scans, distributed maintenance of the VPS backend, and encourages larger coverage. Sharding of VPS services introduces several unique challenges-coherency of localization results across spaces, quality control of VPS services, selection of the right VPS service for a location, and many others. We introduce the concept of federated image-based localization and provide reference solutions for managing and merging data across maps without sharing private data.

We introduce Silverline, a novel framework for lightweight virtualization and orchestration of distributed real-time systems. Leveraging WebAssembly (Wasm) for robust sandboxing and multi-language (polyglot) capabilities, Silverline decouples applications from their platforms through distinct manifests, enabling a centralized orchestrator to optimize resource allocation and deploy Wasm modules seamlessly across the edge-cloud continuum. It features a split data and control plane with orchestration sidecars, allowing applications to use native communication protocols and respond autonomously to network changes. We evaluate our framework in two real application contexts: an industrial automation use-case and an automotive body electronics demonstrator. Through micro-benchmarks and end-to-end testing, we demonstrate Silverline's potential for managing real-time workloads in diverse heterogeneous ecosystems.

As Artificial Intelligence (AI) and eXtended Reality (XR) evolve, integrating them effectively remains a challenge. Although multimodal large language models (MLLMs) offer powerful reasoning over text and images, they lack an inherent understanding of 3D space. Additionally, XR headsets are resource-constrained and cannot run these models locally. To address this gap, we introduce XaiR, a system that integrates MLLMs with XR to enable AI-driven spatial reasoning and interaction. XaiR employs a client-server architecture in which an XR headset (client) captures spatial data, generates 2D snapshots of the 3D environment, and renders augmented reality (AR) content, while a remote server runs multiple parallel MLLMs to generate contextually aware responses. Our demo showcases an XR cognitive assistant application that guides a user through a series of instructions. Deployed on a mobile AR headset, our system dynamically interprets user actions, tracks task progress in real time, and provides textual feedback and AR-guided assistance.

Heisenbugs , notorious for their ability to change behavior and elude reproducibility under observation, are among the toughest challenges in debugging programs. They often evade static detection tools, making them especially prevalent in cyber-physical edge systems characterized by complex dynamics and unpredictable interactions with physical environments. Although dynamic detection tools work much better, most still struggle to meet low enough jitter and overhead performance requirements, impeding their adoption. More importantly however, dynamic tools currently lack metrics to determine an observed bug’s “difficulty” or “heisen–ness” undermining their ability to make any claims regarding their effectiveness against heisenbugs. This paper proposes a methodology for detecting and identifying heisenbugs with low overheads at scale, actualized through the lens of dynamic data-race detection. In particular, we establish the critical impact of execution diversity across both instrumentation density and hardware platforms for detecting heisenbugs; the benefits of which outweigh any reduction in efficiency from limited instrumentation or weaker devices. We develop an experimental WebAssembly-backed dynamic data-race detection framework, Beanstalk, which exploits this diversity to show superior bug detection capability compared to any homogeneous instrumentation strategy on a fixed compute budget. Beanstalk’s approach also gains power with scale , making it suitable for low-overhead deployments across numerous compute nodes. Finally, based on a rigorous statistical treatment of bugs observed by Beanstalk, we propose a novel metric, the heisen factor , that similar detectors can utilize to categorize heisenbugs and measure effectiveness. We reflect on our analysis of Beanstalk to provide insight on effective debugging strategies for both in-house and in deployment settings.

As AR/VR systems evolve to demand increasingly powerful GPUs, physically separating compute from display hardware emerges as a natural approach to enable a lightweight, comfortable form factor. Unfortunately, splitting the system into a client-server architecture leads to challenges in transporting graphical data. Simply streaming rendered images over a network suffers in terms of latency and reliability, especially given variable bandwidth. Although image-based reprojection techniques can help, they often do not support full motion parallax or disocclusion events. Instead, scene geometry can be streamed to the client, allowing local rendering of novel views. Traditionally, this has required a prohibitively large amount of interconnect bandwidth, excluding the use of practical networks. This paper presents a new quad-based geometry streaming approach that is designed with compression and the ability to adjust Quality-of-Experience (QoE) in response to target network bandwidths. Our approach advances previous work by introducing a more compact data structure and a temporal compression technique that reduces data transfer overhead by up to 15×, reducing bandwidth usage to as low as 100 Mbps. We optimized our design for hardware video codec compatibility and support an adaptive data streaming strategy that prioritizes transmitting only the most relevant geometry updates. Our approach achieves image quality comparable to, and in many cases exceeds, state-of-the-art techniques while requiring only a fraction of the bandwidth, enabling real-time geometry streaming on commodity headsets over WiFi.

Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking. However, simulating realistic radar scans is a challenging task that requires an accurate model of the scene, radio frequency material properties, and a corresponding radar synthesis function. Rather than specifying these models explicitly, we propose DART - Doppler Aided Radar Tomography, a Neural Radiance Field-inspired method which uses radar-specific physics to create a reflectance and transmittance-based rendering pipeline for range-Doppler images. We then evaluate DART by constructing a custom data collection platform and collecting a novel radar dataset together with accurate position and instantaneous velocity measurements from lidar-based localization. In comparison to state-of-the-art baselines, DART synthesizes superior radar range-Doppler images from novel views across all datasets and additionally can be used to generate high quality tomographic images.11Our implementation, data collection platform, and collected datasets can be found via our project site: https://wiselabcmu.github.io/dart/.

This paper discusses the integration of Multimodal Large Language Models (MLLMs) with Extended Reality (XR) headsets, focusing on enhancing machine understanding of physical spaces. By combining the contextual capabilities of MLLMs with the sensory inputs from XR, there is potential for more intuitive spatial interactions. However, the integration faces challenges due to the inherent limitations of MLLMs in processing 3D inputs and their significant resource demands for XR headsets. We introduce XaiR, a platform that facilitates integrating MLLMs with XR applications. XaiR uses a split architecture that offloads complex MLLM operations to a server while handling 3D world processing on the headset. This setup manages multiple input modalities, parallel models, and links them with real-time pose data, improving AR content placement in physical scenes. We tested XaiR’s effectiveness with a “cognitive assistant” application that guides users through tasks like making coffee or assembling furniture. Results from a 15-participant study shows over 90% accuracy in task guidance and 85% accuracy in AR content anchoring. Additionally, we evaluate MLLMs against human operators for cognitive assistant tasks which provides insights into the quality of the captured data as well as the current gap in performance for cognitive assistant tasks.

3D video enables a remote viewer to observe a 3D scene from any angle or location. However, current 3D capture solutions incur high latency, consume significant bandwidth, and scale poorly with the number of depth sensors and size of scenes. These problems are largely caused by the current monolithic approach to 3D capture and the use of inefficient data representations for streaming. This paper introduces MeshReduce, a distributed scene capture, stream, and render system that advocates for the use of textured mesh data representation early in the 3D video capture and transmission process. Textured meshes are compact and can provide lower bitrates for the same quality compared to other 3D data representations. However, streaming textured meshes creates compute and memory challenges to achieve bandwidth efficiency. MeshReduce addresses these issues by using a pipeline that creates independent mesh reconstructions and incrementally merges them, rather than creating a single mesh directly from all sensor streams. While this enables a more efficient implementation, this approach requires optimal exchange of textured meshes across the network. MeshReduce also incorporates a novel approach for network rate control that divides bandwidth between texture and mesh for efficient, adaptive 3D video streaming. We demonstrate a real-time integrated embedded compute implementation of MeshReduce that can operate with commercial Azure Kinect depth cameras as well as a custom sensor front-end that uses LiDAR and 360° camera inputs to dramatically increase coverage.

Localizing mobile phone users precisely enough to provide AR content in theaters and concert venues is extremely challenging due to dynamic staging and variable lighting. Visual markers are often disruptive in terms of aesthetics, and static pre-defined feature maps are not robust to visual changes. In this paper, we study several techniques that leverage sparse fixed infrastructure to monitor and adapt to changes in the environment at runtime to enable robust AR quality pose tracking for large audiences. Our most basic technique uses one or more fixed cameras in the environment to prune away poor feature points due to motion and lighting from a static model. For more challenging environments, we propose transmitting dynamic 3D feature maps that adapt to changes in the scene in real-time. Users with a mobile phone camera can use these maps to accurately localize across highly dynamic environments without explicit markers. We show the performance trade-offs resulting from StageAR’s different reconstruction techniques, ranging from multiple stereo cameras to cameras paired with LiDAR. We evaluate each approach in our system across a wide variety of simulated and real environments at auditorium/theater scale and find that our most accurate technique can match the performance of large ($1.5 \times 1.5{\mathrm {m}}$) back-lit static markers without being visible to users.

The concept of Digital Twins (DTs) has been discussed intensively for the past couple of years. Today we have instances of digital twins that range from static descriptions of manufacturing data and material properties over live interfaces on operational data of cyber physical systems to the functions and services they provide. Currently, there are no standardized interfaces to aggregate atomic DTs (e.g., the twin of the lowest-level function of a machine) to higher-level DTs providing more complex services in the virtual world. Additionally, there is no existing infrastructure to reliably link the DTs in the virtual world to the integrated CPSs in the physical world, such as a car consisting of many ECUs with even more functions. The concept of the Metaverse is gaining increasing traction and has been explored from different angles, usually centered around a human user, true to its original definition. Beyond social interactions, the Metaverse offers possibilities to integrate layers of interconnected Digital Twins (DTs) representing parts of and interacting with the physical world in real-time, enabling not only analysis and representation of current state, but also feedback loops and control. This paper describes how the Metaverse can become the virtual world where DTs of humans and machines live, and how to reliably connect DTs to the physical world.

This paper explores a machine learning approach on data from a single-chip mmWave radar for generating high resolution point clouds – a key sensing primitive for robotic applications such as mapping, odometry and localization. Unlike lidar and vision-based systems, mmWave radar can operate in harsh environments and see through occlusions like smoke, fog, and dust. Unfortunately, current mmWave processing techniques offer poor spatial resolution compared to lidar point clouds. This paper presents RadarHD, an end-to-end neural network that constructs lidar-like point clouds from low resolution radar input. Enhancing radar images is challenging due to the presence of specular and spurious reflections. Radar data also doesn't map well to traditional image processing techniques due to the signal's sinc-like spreading pattern. We overcome these challenges by training RadarHD on a large volume of raw I/Q radar data paired with lidar point clouds across diverse indoor settings. Our experiments show the ability to generate rich point clouds even in scenes unobserved during training and in the presence of heavy smoke occlusion. Further, RadarHD's point clouds are high-quality enough to work with existing lidar odometry and mapping workflows.

We demonstrate Platypus, a sub-millimeter micro-displacement sensing system presented in [3]. Micro-displacement measurement is a crucial task in industrial systems such as structural health monitoring, where millimeter-level displacement of specific points on the structure or machinery parts can jeopardize the integrity of the structure and potentially leading to catastrophic damage or collapse. Platypus enables sub-millimeter level sensing accuracy by using mmWave backscatter tags and their reflection as phase carriers to shift the phase changes due to tiny displacements to clean frequency bins for precise tracking. It then reconstructs the tag phase changes with sub-millimeter level accuracy even from extended ranges (over 100m) or in non-line-of-sight (NLoS) situations where the tag is blocked by other objects. Here, we demonstrate Platypus’s performance by attaching a Platypus tag to a stepper motor-driven motion-stage and demonstrating the micro-displacement detection in real time, and the system robustness against multipath and occlusions.

Micro-displacement measurement is a crucial task in industrial systems such as structural health monitoring, where millimeter-level displacement of specific points on the structure or machinery displace can jeopardize the integrity of the structure and potentially leading to catastrophic damage or collapse. Traditionally, such displacements on large structures are measured using visual sensing platforms or advanced surveying equipment. However, they either fall short in varying weather and lighting conditions or require installation and maintenance of high-power sensing platforms that are expensive to deploy at scale, especially if continuous measurements are desired.

Many modern-day XR devices (e.g. mobile headsets, phones, etc.) lack the computing resources required to render complex 3D scenes in real-time. Typically, to render a high-resolution scene on a lightweight XR device, 3D designers arduously decimate and fine-tune the objects. As an alternative, remote rendering systems can utilize powerful nearby servers to stream rendering results to a client. While this is a promising solution, it can introduce a variety of latency and reliability issues, especially under variable network conditions. In this paper, we present a distributed rendering system that combines both remote rendering and on-device, “local” rendering to add robustness to network fluctuations and device workloads. To maximize user QoE, our approach dynamically swaps an object’s rendering medium, adjusting for client workload, low frame rates, and several perceptual characteristics. To model these characteristics, we perform a study under simulated conditions to measure how users perceive latency and complexity differences between objects in a scene. Using the results of the study, we then provide an algorithm for choosing the optimal object rendering medium, based on rendering complexity as well as network and latency models, ensuring that a target frame rate will be met. Finally, we evaluate this algorithm on a prototype implementation that can provide cross-platform split rendering using web technologies.

Millimeter wave radars can perceive through occlusions like dust, fog, smoke and clothes. But compared to cameras and lidars, their perception quality is orders of magnitude poorer. RadarHD [3] tackles this problem of poor quality by creating a machine learning super resolution pipeline trained against high quality lidar scans to mimic lidar. RadarHD ingests low resolution radar and generates high quality lidar-like point clouds even in occluded settings. RadarHD can also make use of the high quality output for typical robotics tasks like odometry, mapping and classification using conventional lidar workflows. Here, we demonstrate the effectiveness of RadarHD's point clouds against lidar in occluded settings.

Directional antennas are a promising solution for improving the range of client devices and capacity of wireless networks. Unfortunately, in LP-WAN systems directional antennas tend to be both large and expensive due to operating at subGHz frequencies. However, if a client device is willing to forgo improvements in antenna gain, it is possible to realize compact and low-cost antennas that provide spatial diversity control. In this paper, we show that by increasing spatial diversity in LPWAN clients with limited (or no) client gain, we can dramatically increase overall network capacity and improve client battery life by avoiding re-transmissions. This type of directional control can also be used for hot-spot management by more effectively load balancing clients across gateways.We performed a sensitivity analysis using a combination of real hardware and simulation to explore the impact of various switchable antenna geometries on network capacity under a variety of deployment configurations. We then designed and evaluated three prototype multi-sector array clients: (1) a switchable patch antenna configuration, (2) a digital phase-shift nulling configuration, and (3) a low-cost switched PCB element phase-shift system. Each design explores a different hardware cost vs antenna beam performance operating point. We experimentally see that our real antenna beam patterns, captured in an anechoic chamber, perform in a similar manner to our simulated prediction models in terms of beam pattern and in simulation improve network capacity by up to 28% from interference isolation alone and up to 95% when offloading hot spots between four gateways. We also perform a small measurement study of how often our final design changes its configuration when deployed over multiple days on a campus testbed.

Virtual Reality (VR) telepresence platforms are being challenged to support live performances, sporting events, and conferences with thousands of users across seamless virtual worlds. Current systems have struggled to meet these demands which has led to high-profile performance events with groups of users isolated in parallel sessions. The core difference in scaling VR environments compared to classic 2D video content delivery comes from the dynamic peer-to-peer spatial dependence on communication. Users have many pair-wise interactions that grow and shrink as they explore spaces. In this paper, we discuss the challenges of VR scaling and present an architecture that supports hundreds of users with spatial audio and video in a single virtual environment. We leverage the property of spatial locality with two key optimizations: (1) a Quality of Service (QoS) scheme to prioritize audio and video traffic based on users' locality, and (2) a resource manager that allocates client connections across multiple servers based on user proximity within the virtual world. Through real-world deployments and extensive evaluations under real and simulated environments, we demonstrate the scalability of our platform while showing improved QoS compared with existing approaches.

The rise of IoT devices brings a lot of security risks. To mitigate them, researchers have introduced various promising network-based anomaly detection algorithms, which oftentimes leverage machine learning. Unfortunately, though, their deployment and further improvement by network operators and the research community are hampered. We believe this is due to three key reasons. First, known ML-based anomaly detection algorithms are evaluated -in the best case- on a couple of publicly available datasets, making it hard to compare across algorithms. Second, each ML-based IoT anomaly-detection algorithm makes assumptions about attacker practices/classification granularity, which reduce their applicability. Finally, the implementation of those algorithms is often monolithic, prohibiting code reuse. To ease deployment and promote research in this area, we present Lumen. Lumen is a modular framework paired with a benchmarking suite that allows users to efficiently develop, evaluate, and compare IoT ML-based anomaly detection algorithms. We demonstrate the utility of Lumen by implementing state-of-the-art anomaly detection algorithms and faithfully evaluating them on various datasets. Among other interesting insights that could inform real-world deployments and future research, using Lumen, we were able to identify what algorithms are most suitable to detect particular types of attacks. Lumen can also be used to construct new algorithms with better performance by combining the building blocks of competing efforts and improving the training setup.

Current collaborative augmented reality (AR) systems establish a common localization coordinate frame among users by exchanging and comparing maps comprised of feature points. However, relative positioning through map sharing struggles in dynamic or feature-sparse environments. It also requires that users exchange identical regions of the map, which may not be possible if they are separated by walls or facing different directions. In this paper, we present Cappella11Like its musical inspiration, Cappella utilizes collaboration among agents to forgo the need for instrumentation, an infrastructure-free 6-degrees-of-freedom (6DOF) positioning system for multi-user AR applications that uses motion estimates and range measurements between users to establish an accurate relative coordinate system. Cappella uses visual-inertial odometry (VIO) in conjunction with ultra-wideband (UWB) ranging radios to estimate the relative position of each device in an ad hoc manner. The system leverages a collaborative particle filtering formulation that operates on sporadic messages exchanged between nearby users. Unlike visual landmark sharing approaches, this allows for collaborative AR sessions even if users do not share the same field of view, or if the environment is too dynamic for feature matching to be reliable. We show that not only is it possible to perform collaborative positioning without infrastructure or global coordinates, but that our approach provides nearly the same level of accuracy as fixed infrastructure approaches for AR teaming applications. Cappella consists of an open source UWB firmware and reference mobile phone application that can display the location of team members in real time using mobile AR. We evaluate Cappella across mul-tiple buildings under a wide variety of conditions, including a contiguous 30,000 square foot region spanning multiple floors, and find that it achieves median geometric error in 3D of less than 1 meter.

Robotic geo-fencing and surveillance systems require accurate monitoring of objects if/when they violate perimeter restrictions. In this paper, we seek a solution for depth imaging of such objects of interest at high accuracy (few tens of cm) over extended ranges (up to 300 meters) from a single vantage point, such as a pole mounted platform. Unfortunately, the rich literature in depth imaging using camera, lidar and radar in isolation struggles to meet these tight requirements in real-world conditions. This paper proposes Metamoran, a solution that explores long-range depth imaging of objects of interest by fusing the strengths of two complementary technologies: mmWave radar and camera. Unlike cameras, mmWave radars offer excellent cm-scale depth resolution even at very long ranges. However, their angular resolution is at least 10x worse than camera systems. Fusing these two modalities is natural, but in scenes with high clutter and at long ranges, radar reflections are weak and experience spurious artifacts. Metamoran's core contribution is to leverage image segmentation and monocular depth estimation on camera images to help declutter radar and discover true object reflections. We perform a detailed evaluation of Metamoran's depth imaging capabilities in 400 diverse scenarios. Our evaluation shows that Metamoran estimates the depth of static objects up to 90 m away and moving objects up to 305 m away and with a median error of 28 cm, an improvement of 13 x over a naive radar+camera baseline and 23 x compared to monocular depth estimation.

Collaborative AR applications are gaining popularity, but have heavy computing requirements for identifying and tracking AR devices and objects in the ecosystem. Prior AR frameworks typically rely on edge infrastructure to offload AR's compute-heavy tasks. However, such infrastructure may not always be available, and continuously running AR computations on user devices can rapidly drain battery and impact application longevity. In this work, we enable infrastructure-free mobile AR with a low energy footprint, by using collaborative time slicing to distribute compute-heavy AR tasks across user devices. Realizing this idea is challenging because distributed execution can result in inconsistent synchronization of the AR virtual overlays. Our framework, FreeAR, tackles this with novel lightweight techniques for tightly synchronized virtual overlay placements across user views, and low latency recovery upon disruptions. We prototype FreeAR on Android and show that it can improve the virtual overlay positioning accuracy (with respect to the IOU metric) by up to 78%, relative to state-of-the-art collaborative AR systems, while also reducing power by up to 60% relative to a direct application of those prior solutions.

It has long been a goal of immersive telepresence to capture and stream 3D spaces such that a remote viewer can watch from any location or angle within the scene. This demonstration presents Mosaic, a new distributed 3D scene capture system that uses textured mesh data representation for streaming a 3D volumetric video of a space to remote viewers. Compared to more common point cloud based methods, we show that textured mesh data requires less bandwidth and yields the same visual quality. However, textured mesh reconstruction is compute and memory intensive, mesh simplification is not easily parallelizable, and texture maps lacks spatial and temporal coherence. Mosaic tackles these challenges by examining each computational stage and determines how they can be efficiently distributed across multiple compute nodes to reduce overall latency, minimize bandwidth, and maintain quality. We then provide an end-to-end latency and bandwidth breakdown that can be used to target future acceleration work.

LoRa is a popular Low-Power Wide-Area Networking (LP-WAN) technology that allows devices powered by a ten year AA battery to connect to radio infrastructure miles away. One of the most promising features of LoRa is the ability to track the location of radios from a distance, enabling applications ranging from inventory tracking, smart infrastructure monitoring and structural health sensing. Yet, state-of-the-art LoRa localization systems experience errors of several tens or even hundreds of meters in location tracking, owing to the narrow bandwidth and limited battery life of LoRa devices.

Many have predicted the future of the Web to be the integration of Web content with the real-world through technologies such as Augmented Reality (AR). This has led to the rise of Extended Reality (XR) Web Browsers used to shorten the long AR application development and deployment cycle of native applications especially across different platforms. As XR Browsers mature, we face new challenges related to collaborative and multi-user applications that span users, devices, and machines. These collaborative XR applications require: (1) networking support for scaling to many users, (2) mechanisms for content access control and application isolation, and (3) the ability to host application logic near clients or data sources to reduce application latency. In this paper, we present the design and evaluation of the AR Edge Networking Architecture (ARENA) which is a platform that simplifies building and hosting collaborative XR applications on WebXR capable browsers. ARENA provides a number of critical components including: a hierarchical geospatial directory service that connects users to nearby servers and content, a token-based authentication system for controlling user access to content, and an application/service runtime supervisor that can dispatch programs across any network connected device. All of the content within ARENA exists as endpoints in a PubSub scene graph model that is synchronized across all users. We evaluate ARENA in terms of client performance as well as benchmark end-to-end response-time as load on the system scales. We show the ability to horizontally scale the system to Internet-scale with scenes containing hundreds of users and latencies on the order of tens of milliseconds. Finally, we highlight projects built using ARENA and showcase how our approach dramatically simplifies collaborative multi-user XR development compared to monolithic approaches.

Public spaces like concert stadiums and sporting arenas are ideal venues for AR content delivery to crowds of mobile phone users. Unfortunately, these environments tend to be some of the most challenging in terms of lighting and dynamic staging for vision-based relocalization. In this paper, we introduce FLASH1, a system for delivering AR content within challenging lighting environments that uses active tags (i.e., blinking) with detectable features from passive tags (quads) for marking regions of interest and determining pose. This combination allows the tags to be detectable from long distances with significantly less computational overhead per frame, making it possible to embed tags in existing video displays like large jumbotrons. To aid in pose acquisition, we implement a gravity-assisted pose solver that removes the ambiguous solutions that are often encountered when trying to localize using standard passive tags. We show that our technique outperforms similarly sized passive tags in terms of range by 20-30% and is fast enough to run at 30 FPS even within a mobile web browser on a smartphone.

In this paper, we demonstrate Adaptive Millimetro as an extension of Millimetro, an ultra-low power millimeter-wave (mmWave) retro-reflector presented in [1], for high mobility scenarios. Adaptive Millimetro makes use of automotive radars and enables communication with and accurate localization of roadside infrastructure overextended distances (i.e. >100m). Millimetro achieves this by designing ultra-low-power retro-reflective tags that operate in the mmWave frequency band and can be embedded in road signs, pavements, bi-cycles, or even the clothing of pedestrians. Millimetro addresses the severe path loss problem of mmWave signals by combining coding gain and retro-reflective antenna front-end to achieve long-range operation. However, highly mobile scenarios may still experience unreliable performance due to the Doppler effect changing the received signals. In this paper, we demonstrate a simple solution for robust localization in high mobility by implementing a Moving Target Indication (MTI) filter and an adaptive Kalman filter. We also present an augmented reality app, as an in-car AR platform, that uses Adaptive Millimetro’s algorithms to estimate the tag positions and overlay a virtual box at the estimated locations.

This paper presents Millimetro, an ultra-low-power tag that can be localized at high accuracy over extended distances. We develop Millimetro in the context of autonomous driving to efficiently localize roadside infrastructure such as lane markers and road signs, even if obscured from view, where visual sensing fails. While RF-based localization offers a natural solution, current ultra-low-power localization systems struggle to operate accurately at extended ranges under strict latency requirements. Millimetro addresses this challenge by re-using existing automotive radars that operate at mmWave frequency where plentiful bandwidth is available to ensure high accuracy and low latency. We address the crucial free space path loss problem experienced by signals from the tag at mmWave bands by building upon Van Atta Arrays that retro-reflect incident energy back towards the transmitting radar with minimal loss and low power consumption. Our experimental results indoors and outdoors demonstrate a scalable system that operates at a desirable range (over 100 m), accuracy (centimeter-level), and ultra-low-power (< 3 uW).

Low-Power Wide-Area Networks (LP-WANs) are seeing wide-spread deployments connecting millions of sensors, each powered by a ten-year AA battery to radio infrastructure, often miles away. By design, iteratively querying all sensors in an LP-WAN may take several hours or even days, given the stringent battery limits of client radios. This precludes obtaining even an approximate real-time view of sensed information across LP-WAN devices over a large area, say in the event of a disaster, fault or simply for diagnostics.This paper presents QuAiL1, a system that provides a coarse aggregate view of sensed data across LP-WAN devices over a wide- area within a time span of just one LP-WAN packet. QuAiL achieves this by coordinating multiple LP-WAN radios to transmit their information synchronously in time and frequency despite their power constraints. We design each client’s transmission so that the base station can retrieve an approximate heatmap of sensed data by exploiting the spatial correlation of this data across clients. We further show how our system can be optimized for statistical and machine learning queries, all while maintaining the security and privacy of sensed data from individual clients. Our deployment over a 3 sq. km. LP-WAN deployment around CMU campus in Pittsburgh demonstrates a 4x faster information retrieval versus the state-of- the-art statistical methods to retrieve the spatial sensor heatmap at a desired resolution.

One of the major challenges faced by Augmented Reality (AR) systems is linking virtual content accurately on physical objects and locations. This problem is amplified for applications like mobile payment, device control or secure pairing that requires authentication. In this paper, we present an active LED tag system called GLITTER that uses a combination of Bluetooth Low-Energy (BLE) and modulated LEDs to anchor AR content with no a priori training or labeling of an environment. Unlike traditional optical markers that encode data spatially, each active optical marker encodes a tag’s identifier by blinking over time, improving both the tag density and range compared to AR tags and QR codes.We show that with a low-power BLE-enabled micro-controller and a single 5 mm LED, we are able to accurately link AR content from potentially hundreds of tags simultaneously on a standard mobile phone from as far as 30 meters. Expanding upon this, using active optical markers as a primitive, we show how a constellation of active optical markers can be used for full 3D pose estimation, which is required for many AR applications, using either a single LED on a planar surface or two or more arbitrarily positioned LEDs. Our design supports 108 unique codes in a single field of view with a detection latency of less than 400 ms even when held by hand.

Conventional wireless communication systems are typically designed assuming a single transmitter-receiver pair for each link. In Low-Power Wide-Area Networks (LP-WANs), this one-to-one design paradigm is often overly pessimistic in terms of link budget because client packets are frequently detected by multiple gateways (i.e. one-to-many). Prior work has shown massive improvement in performance when specialized hardware is used to coherently combine signals at the physical layer.

Tire wear is a leading cause of automobile accidents globally. Beyond safety, tire wear affects performance and is an important metric that decides tire replacement, one of the biggest maintenance expense of the global trucking industry. We believe that it is important to measure and monitor tire wear in all automobiles. Current approach to measure tire wear is manual and extremely tedious. Embedding sensor electronics in tires to measure tire wear is challenging, given the inhospitable temperature, pressure and dynamics of the tire. Further, off-tire sensors placed in the well such as laser range-finders are vulnerable to road debris that may settle in tire grooves.

In this paper, we demonstrate Osprey, a tire wear sensor presented in [4]. Osprey makes use of commodity automotive, mmWave RADAR, places it in the tire well of automobiles to image the tire and then measures the tire wear. Osprey measures accurate tire wear continuously while being resilient to road debris and without embedding any electronics in tires. Osprey achieves this by building a super resolution algorithm based on Inverse Synthetic Aperture RADAR imaging and by embedding thin metallic strips along coded patterns in the grooves to combat debris. Here, we implement Osprey on a tire rotation rig and demonstrate the ability to measure tire wear (with and without debris) accurately and detect potentially harmful foreign objects.

Beacon-based time-of-flight indoor localization systems have shown great promise for applications ranging from indoor navigation to asset tracking. In large-scale deployments, a major practical challenge is determining the placement of a minimal number of beacons that ensures full coverage -- each point in the domain has line-of-sight paths to enough beacons to uniquely localize itself. Three beacons with line-of-sight paths are always enough, but two beacons within line of sight may also work, given a favorable geometry. In this paper, we propose two beacon placement algorithms that leverage the floor plan geometry with provable theoretical guarantees. First, we present a greedy algorithm using properties of sub-modular functions to place O(OPT · ln m) beacons, where m is the number of discrete location points in the region that need to be localized, and OPT is the size of the optimal solution. Second, we present a random sampling algorithm that places O (OPT · log(OPT)) beacons while localizing all targets. We evaluate our algorithms on both real-world and randomly generated floor plans. Our algorithms place on an average 6 ~ 23% and 12% fewer beacons in real-world topologies and randomly generated floor plans respectively, as compared to prior work. We also present a study where we ask users to attempt to place nodes manually and discover that even humans that are well versed on the coverage problem find it hard to balance the trade-off between the number of beacons and area localized.

In this paper, we show how beacon-based indoor localization and additional environmental fingerprints like magnetic field data can be used to both accelerate and increase the robustness of Augmented Reality (AR) relocalization. We show how the combination of Visual Inertial Odometry (VIO) and beacons can be used to construct a dense indoor magnetic field map that can act as a fine-grained calibration for compasses to quickly determine a mobile device's orientation. Unique to our approach is that we leverage accurate VIO trajectories to provide full vector magnetic field mapping that can be collected and used with devices placed in any orientation. We demonstrate a system running on an iPhone that can acquire location with 80th percentile 3D accuracy of 27cm in LOS and 46cm in NLOS, and our magnetic field mapping approach can instantly estimate orientation with 80th percentile accuracy of 11.7 degrees. We demonstrate an end-to-end system for generating a magnetic field map and subsequently localizing and tracking mobile users using beacons and VIO. This has the side effect of enabling multi-user (even cross-platform) AR applications, as all users can now be localized with respect to a common global reference without any sharing of visual feature maps.

Understanding the location of acoustically reflective surfaces in a room is a critical component in advanced sound processing. For example, intelligent speakers can use a room's acoustic geometry to improve playback quality, source separation accuracy, and speech recognition. In this paper, we present Synesthesia, a system for capturing the acoustic properties of a room using a single fixed speaker and a mobile phone that records audio at multiple locations. Using the arrival time of echoes, the system is able to reconstruct the position of reflective surfaces like walls and then estimate properties like surface absorption.

In this demonstration, we present OpenChirp, an open-source Low-Power Wide-Area Networking (LPWAN) infrastructure. OpenChirp is a management framework that provides data context, storage, visualization, and access control over the web. At the physical layer of the system, we present LPRAN, a low-cost high-performance software-defined radio hardware platform that can receive signals up to -30 dB below the noise floor. Using our LPRAN hardware, it is possible to operate on raw I/Q streams in the cloud to perform collaborative tasks across multiple gateways such as jointly decoding weak signals and localization.

Low-Power Wide-Area Networks (LPWANs) are an emerging wireless platform which can support battery-powered devices lasting 10-years while communicating at low data-rates to gateways several kilometers away. Not all such devices will experience the promised 10 year battery life despite the high density of LPWAN gateways expected in cities. Transmission from devices located deep within buildings or in remote neighborhoods will suffer severe attenuation forcing the use of slow data-rates to reach even the closest gateway, thus resulting in battery drain. This paper presents Charm, a system that enhances both the battery life of client devices and the coverage of LPWANs in large urban deployments. Charm allows multiple LoRaWAN gateways to pool their received signals in the cloud, coherently combining them to detect weak signals that are not decodable at any individual gateway. Through a novel hardware and software design at the gateway, Charm carefully detects which chunks of the received signal need to be sent to the cloud, thereby saving uplink bandwidth. We present a scalable solution to decoding weak transmissions at city-scale by identifying the set of gateways whose signals need to be coherently combined over time. In evaluations over a test network and from simulations using traces from a large LoRaWAN deployment in Pittsburgh, Pennsylvania, Charm demonstrates a gain of up to 3x in range and 4x in client battery-life.

We specify and evaluate a new software-defined clock network architecture, Stitch. We use Stitch to derive all subsystem clocks from a single local oscillator (LO) on an embedded platform, and enable efficient radio frequency synchronization (RFS) between two nodes' LOs. RFS uses the complex baseband samples from a low-power low-cost narrowband transceiver to drive the frequency difference between the two devices to less than 3 parts per billion (ppb). Recognizing that the use of a wideband channel to measure clock frequency offset for synchronization purposes is inefficient, we propose to use a separate narrowband radio to provide these measurements. However, existing platforms do not provide the ability to unify the local oscillator across multiple subsystems. We demonstrate Stitch with a reference hardware implementation on a research platform. We show that, with Stitch and RFS, we are able to achieve dramatic efficiency gains in ultra-wideband (UWB) time synchronization and ranging. We demonstrate the same UWB ranging accuracy in state-of-the-art systems but with 59% less utilization of the UWB channel.

Indoor localization systems typically determine a position using either ranging measurements, inertial sensors, environmental-specific signatures or some combination of all of these methods. Given a floor plan, inertial and signature-based systems can converge on accurate locations by slowly pruning away inconsistent states as a user walks through the space. In contrast, range-based systems are capable of instantly acquiring locations, but they rely on densely deployed beacons and suffer from inaccurate range measurements given non-line-of-sight (NLOS) signals. In order to get the best of both worlds, we present an approach that systematically exploits the geometry information derived from building floor plans to directly improve location acquisition in range-based systems. Our solving approach can disambiguate multiple feasible locations taking into account a mix of LOS and NLOS hypotheses to accurately localize with significantly fewer beacons. We demonstrate our geometry-aware solving approach using a new ultrasonic beacon platform that is able to perform direct time-of-flight ranges on commodity smartphones. The platform uses Bluetooth Low Energy (BLE) for time synchronization and ultrasound for measuring propagation distance. We evaluate our system's accuracy with multiple deployments in a university campus and show that our approach shifts the 80% accuracy point from 4-8m to 1m as compared to solvers that do not use the floor plan information. We are able to detect and remove NLOS signals with 91.5% accuracy.

We demonstrate multi-user persistent Augmented Reality (AR) on mobile devices with a novel technique that provides nearly instant acquisition of location and orientation. Visual Inertial Odometry (VIO) provides accurate position and orientation tracking relative to device start-up for AR applications. Unfortunately, the tracking is local to the AR session of a single user and is not anchored in a global coordinate system. In order to provide all devices an accurate location in a common frame of reference, we utilize UWB nodes that range to the devices. To avoid the long startup time required to compute the device's orientation, we propose a novel technique that utilizes previously recorded magnetic field information to rapidly calibrate the compass. In order to simplify setup, we demonstrate automatic mapping of beacon locations and surveying of magnetic field by a pedestrian walking around the test area with a mobile device.

Recent advances in Unmanned Aerial Vehicles (UAVs) have enabled countless new applications in the domain of aerial sensing. In scenarios such as intrusion detection, target tracking and facility monitoring it is important to reach a given area of interest (AOI), and create an online data streaming connection to a monitoring ground station (GS) for immediate delivery of content to the operator. In previous work, we showed that a multi-hop line network can increase the range of the mission by finding the optimal number of relay UAVs, and their optimal placement. In this demo, we show that CSMA (typical 802.11's medium access protocol) behaves poorly in this type of networks due to mutual interference, and that TDMA is a better alternative. We will also discuss how changing slot width online can overcome typical and less known TDMA in-efficiencies, and therefore reach maximum end-to-end throughput and low delay.

In this paper, we introduce Pulsar, a wireless time transfer platform that can achieve clock synchronization to better than five nanosecond between indoor or GPS-denied devices. Nanosecond-level clock synchronization is a missing capability for many real-time applications like next-generation wireless systems that leverage spatial multiplexing to improve channel capacity and provide services like time-of-flight localization. With fine-grained synchronization, both clock stability and propagation delays introduce significant sources of error. Pulsar leverages a stable clock source derived from a Chip-Scale Atomic Clock (CSAC) along with an Ultra-WideBand (UWB) radio able to perform sub-nanosecond packet timestamping to estimate and correct for clock offsets. We design and evaluate a proof-of-concept network-wide synchronization protocol for Pulsar that selects low-jitter links to both estimate the location of nodes and reduce cumulative synchronization error across multiple hops. The Pulsar platform and protocol together provide a phase synchronized one pulse per second (1PPS) signal and 10 MHz reference clock that can be easily integrated with typical enduser applications like software-defined radios and communication systems. We experimentally evaluate the Pulsar platform in terms of clock synchronization accuracy, Allan deviation between pairwise clocks and ranging accuracy to show a clock synchronization of better than five nanoseconds per hop with an average of 2.12 ns and a standard deviation of 0.84 ns. The platform is able to identify and avoid clock error in cases where there is heavy multi-path or non-Line-of-Sight signals.

Occupancy estimation is an important primitive for a wide range of applications including building energy efficiency, safety, and security. In this paper, we explore the potential of using depth sensors to detect, estimate, identify, and track occupants in buildings. While depth sensors have been widely used for human detection and gesture recognition, computer vision algorithms are typically run on a powerful computer like XBOX or Intel R CoreTM i7 processor. In this work, we develop a prototype system called FORK using off-the-shelf components that performs the entire depth data processing on a cheaper and low power ARM processor in real-time. As ARM processors are extremely weak in running computer vision algorithms, FORK is designed to detect humans and track them in a very efficient way by leveraging a novel lightweight model based approach instead of traditional approaches based on histogram of oriented gradients (HOG) features. Unlike other camera based approaches, FORK is much less privacy invasive (even if the sensor is compromised). Based on a complete implementation, real-world deployment, and extensive evaluation at realistic scenarios, we observe that FORK achieves over 99% accuracy in real-time (4-9 FPS) in occupancy estimation.

Unmanned Aerial Vehicles (UAVs) are rapidly becoming an important tool for applications like surveillance, target tracking and facility monitoring. In many of these contexts, one or more UAVs need to reach an area of interest (AOI) while streaming live video to a ground station (GS) where one or more operators inspect the AOI and carry out fine control of UAVs position. In remote areas, intermediate UAVs can act as relays and form a line network to extend range. Interactive control requires a live video stream where both throughput and delay are important. In this paper, we show that routing packets over CSMA/CA (native medium access protocol of WiFi, the most common wireless technology among UAVs) behaves poorly in this context due to link asymmetries. We propose a novel distributed, adaptive and self-synchronized TDMA protocol (DVSP) that both enhances delay and packet delivery while operating on commodity hardware and leveraging a standard UDP/IP protocol stack. We prove that DVSP converges to a global solution that minimizes delay using local information, only, thus in a fully distributed manner. Real world experiments with multiple UAVs show gains in delay up to 75%, and packet delivery up to 50%, without sacrificing goodput.

Unmanned aerial vehicles (UAVs) recently enabled a myriad of new applications spanning domains from personal entertainment to surveillance and monitoring. In this paper, we focus on using several small UAVs collaboratively to provide extended reach to an online video monitoring system for inspection of industrial installations. We make use of 802.11 radios on low-cost commercial-off-the-shelf UAVs, set up a time-division multiple access overlay protocol to avoid mutual interference, and enable high channel utilization in multihop networks. In particular, we provide a model for the quality of the UAV-to-UAV link, in terms of packet delivery ratio as a function of distance, packet size, and orientation, based on an extensive measurement campaign. We show that this platform is not omnidirectional in the horizontal plane and that UAV-to-UAV communication ceases around 75 m. Concerning the operation in a multihop mode to allow extending the network, the paper derives the optimal number of hops that maximize the end-to-end throughput, as well as the corresponding hop lengths. We validate our mathematical model with extensive experimental measurements transmitting payloads up to 200 m (over 802.11 g at 54 MBps).

In this paper, we address the problem of range-based beacon placement given a floor plan to support indoor localization systems. Existing approaches for trilateration require three or more beacons to determine a unique position solution. We show that with prior knowledge of the map and a model of beacon coverage, it is possible to uniquely localize with only two beacons. This not only reduces installation cost by requiring fewer nodes, but can also improve robustness. One of the main challenges with respect to beacon placement algorithms is defining a metric for estimating performance. We propose augmenting the commonly used Geometric Dilution of Precision (GDOP) metric to account for indoor spaces. We then use this enhanced GDOP metric as part of a toolchain to compare various beacon placement algorithms in terms of coverage and expected accuracy. When applied to a set of real floor plans, our approach is able to reduce the number of beacons between 22% and 60% (33% on an average) as compared to standard trilateration.

Having a shared and accurate sense of time is critical to distributed Cyber-Physical Systems (CPS) and the Internet of Things (IoT). Thanks to decades of research in clock technologies and synchronization protocols, it is now possible to measure and synchronize time across distributed systems with unprecedented accuracy. However, applications have not benefited to the same extent due to limitations of the system services that help manage time, and hardware-OS and OS-application interfaces through which timing information flows to the application. Due to the importance of time awareness in a broad range of emerging applications, running on commodity platforms and operating systems, it is imperative to rethink how time is handled across the system stack. We advocate the adoption of a holistic notion of Quality of Time (QoT) that captures metrics such as resolution, accuracy, and stability. Building on this notion we propose an architecture in which the local perception of time is a controllable operating system primitive with observable uncertainty, and where time synchronization balances applications' timing demands with system resources such as energy and bandwidth. Our architecture features an expressive application programming interface that is centered around the abstraction of a timeline - a virtual temporal coordinate frame that is defined by an application to provide its components with a shared sense of time, with a desired accuracy and resolution. The timeline abstraction enables developers to easily write applications whose activities are choreographed across time and space. Leveraging open source hardware and software components, we have implemented an initial Linux realization of the proposed timeline-driven QoT stack on a standard embedded computing platform. Results from its evaluation are also presented.

In this demonstration we present Pulsar, a speed-of-light propagation-aware time synchronization platform. Pulsar uses ultra-wideband (UWB) radios for time transfer with each node backed by a chip scale atomic clock (CSAC) that in combination are able to provide accuracy on the order of 10's of nanoseconds for indoor applications. The demonstration will show two Pulsar boards generating a tightly synchronized PPS output that is able to adjust for the distance between the nodes. Even without communication, the devices maintain synchronization over multiple seconds due to a stable CSAC clocking the system.

Recent advances in Unmanned Aerial Vehicles (UAVs) have enabled a myriad of new applications in many different domains from personal entertainment to process and infrastructure online monitoring in large industrial sites, among other. Our work focuses on how one can use several small UAVs collaboratively to provide extended reach to an online video monitoring system. We demonstrate how a TDMA overlay using 802.11 radios on low-cost commercial-off-the-shelf (COTS) UAVs can be used to enable high channel utilization in multi-hop networks, by avoiding mutual interference. This paper presents an extensive network characterisation and modelling of the quality of the UAV-to-UAV link, in terms of packet delivery ratio as a function of distance, packet size and orientation. We show that this platform is non-omnidirectional in the flight plane and that UAV-to-UAV communication ceases around 75m. Then, we solve the mathematical problem of finding the optimal link length and number of hops that maximize the end-to-end throughput, as we extend the network. We validate our mathematical model with extensive experimental campaigns transmitting payloads up to 200m (over 802.11g @ 54MBps).

We consider the class of projector-camera systems that adaptively image and illuminate a dynamic environment. Examples include adaptive front lighting in vehicles, dynamic stage performance lighting, adaptive dynamic range imaging and volumetric displays. A simulator is developed to explore the design space of such Reactive Visual Systems. Simulations are conducted to characterize system performance by analyzing the effects of end-to-end latency, jitter, and prediction algorithm complexity. Key operating points are identified where systems with simple prediction algorithms can outperform systems with more complex prediction algorithms. Based on the lessons learned from simulations, a low latency and low jitter, tight closed-loop reactive visual system is built. For the first time, we measure end-to-end latency, perform jitter analysis, investigate various prediction algorithms and their effect on system performance, compare our system's performance to previous work, and demonstrate dis-illumination of falling snow-like particles and photography of fast moving scenes.

In this paper, we present the design and evaluation of a platform that can be used for time synchronization and indoor positioning of mobile devices. The platform uses the Time-Difference-Of-Arrival (TDOA) of multiple ultrasonic chirps broadcast from a network of beacons placed throughout the environment to find an initial location as well as synchronize a receiver's clock with the infrastructure. These chirps encode identification data and ranging information that can be used to compute the receiver's location. Once the clocks have been synchronized, the system can continue performing localization directly using Time-of-Flight (TOF) ranging as opposed to TDOA. This provides similar position accuracy with fewer beacons (for tens of minutes) until the mobile device clock drifts enough that a TDOA signal is once again required. Our hardware platform uses RF-based time synchronization to distribute clock synchronization from a subset of infrastructure beacons connected to a GPS source. Mobile devices use a novel time synchronization technique leverages the continuously free-running audio sampling subsystem of a smartphone to synchronize with global time. Once synchronized, each device can determine an accurate proximity from as little as one beacon using TOF measurements. This significantly decreases the number of beacons required to cover an indoor space and improves performance in the face of obstructions. We show through experiments that this approach outperforms the Network Time Protocol (NTP) on smartphones by an order of magnitude, providing an average 720μs synchronization accuracy with clock drift rates as low as 2ppm.

The proliferation of Bluetooth Low-Energy (BLE) chipsets on mobile devices has lead to a wide variety of user-installable tags and beacons designed for location-aware applications. In this paper, we present the Acoustic Location Processing System (ALPS), a platform that augments BLE transmitters with ultrasound in a manner that improves ranging accuracy and can help users configure indoor localization systems with minimal effort. A user places three or more beacons in an environment and then walks through a calibration sequence with their mobile device where they touch key points in the environment like the floor and the corners of the room. This process automatically computes the room geometry as well as the precise beacon locations without needing auxiliary measurements. Once configured, the system can track a user's location referenced to a map.

We demonstrate the Acoustic Location Processing System (ALPS), a platform that augments BLE proximity beacons with ultrasonic transmitters in a manner that allows for precise and robust indoor localization. {\em ALPS} uses Time-Difference-Of-Arrival (TDOA) and Time-Of-Flight (TOF) ranging to accurately localize mobile devices such as off-the-shelf smartphones and tablets in 2D space. Users inside the demo area will be able to determine their location and can directly plot it relatively to a map of the area using our app on a smartphone. Once a receiving device has determined its initial position, it can synchronize its audio clock with the transmission infrastructure to perform TOF-based localization, which provides similar position accuracy to TDOA based localization with fewer beacons. Multilateration and trilateration processing for each device's location is offloaded onto a cloud-based solver that can provide localization as a service to ALPS and similar TOF/TDOA based systems.

Traditional hard real-time scheduling algorithms require the use of the worst-case execution times to guarantee that deadlines will be met. Unfortunately, many algorithms with parameters derived from sensing the physical world suffer large variations in execution time, leading to pessimistic overall utilization, such as visual recognition tasks. In this article, we present ZS-QRAM, a scheduling approach that enables the use of flexible execution times and application-derived utility to tasks in order to maximize total system utility. In particular, we provide a detailed description of the algorithm, the formal proofs for its temporal protection, and a detailed, evaluation. Our evaluation uses the Utility Degradation Resilience (UDR) showing that ZS-QRAM is able to obtain 4× as much UDR as ZSRM, a previous overbooking approach, and almost 2× as much UDR as Rate-Monotonic with Period Transformation (RM/TP). We then evaluate a Linux kernel module implementation of our scheduler on an Unmanned Air Vehicle (UAV) platform. We show that, by using our approach, we are able to keep the tasks that render the most utility by degrading lower-utility ones even in the presence of highly dynamic execution times.

Poor posture and incorrect muscle usage are a leading cause of many injuries in sports and fitness. For this reason, non- invasive, fine-grained sensing and monitoring of human motion and muscles is important for mitigating injury and improving fitness efficacy. Current sensing systems either de- pend on invasive techniques or unscalable approaches whose accuracy is highly dependent on body sensor placement. As a result these systems are not suitable for use in active sports or fitness training where sensing needs to be scalable, accurate and un-inhibitive to the activity being performed. We present MARS, a system that detects both body motion and individual muscle group activity during physical human activity by only using unobtrusive, non-invasive in- ertial sensors. MARS not only accurately senses and recreates human motion down to the muscles, but also allows for fast personalized system setup by determining the individual identities of the instrumented muscles, obtained with minimal system training. In a real world human study con- ducted to evaluate MARS, the system achieves greater than 95% accuracy in identifying muscle groups.

In this demonstration, we show an energy measurement system that estimates the energy consumption of individual appliances using a wireless sensor network consisting of contactless electromagnetic field (EMF) sensors deployed near each appliance, and a whole-house power meter. The EMF sensor can detect appliance state transitions within close proximity based on magnetic field fluctuations. Data from these sensors are then relayed back to the main meter using a low-latency wireless sensor networking protocol, where changes in the total power consumption of the house are used to determine the power usage of individual appliances. The sensors are low-cost, easy to deploy and are able to detect current changes associated with the appliance from a few inches away making it possible to externally monitor in-wall wiring to devices like overhead lights or heavy machinery that might operate on multiple phases of the AC distribution system of the building. Appliance-level energy data provide continuous feedback to end users about their consumption patterns and provide building managers accurate information that can be used to target the most effective update and retrofit strategies.

The tight coupling among computation, sensing and control found in Cyber-Physical Systems (CPS) often requires information processing to be completed within strict timing deadlines. Traditional hard real-time scheduling algorithms require the use of the worst-case execution times to guarantee that deadlines will be met. Unfortunately, many algorithms with parameters derived from sensing the physical world suffer from large variations in execution time, which leads to pessimistic overall utilization. For example, object tracking in a computer vision system is highly dependent on the number and size of the objects within the camera's field of view. In this paper, we present the formal description of ZS-QRAM [8], a scheduling approach that allows system designers to flexibly assign execution times and application-derived utility to tasks in order to maximize total system utility even in the presence of highly variable processing estimates. In particular, we provide a detailed description of the algorithm, the formal proofs for its temporal protection and a detail evaluation. Our evaluation uses the Utility Degradation Resilience (UDR) metric presented in [8]. Our results show that ZS-QRAM is able to obtain four times as much UDR as ZSRM, a previous overbooking approach, and almost twice as much UDR as Rate-Monotonic with Period Transformation (RM/TP) even when the latter does not provide temporal protection.

Time synchronization in wireless sensor networks is important for event ordering and efficient communication scheduling. In this paper, we introduce an external hardwarebased clock tuning circuit that can be used to improve synchronization and significantly reduce clock drift over long periods of time without waking up the host MCU. This is accomplished through two main hardware sub-systems. First, we improve upon the circuit presented in [1] that synchronizes clocks using the ambient magnetic fields emitted from power lines. The new circuit uses an electric field front-end as opposed to the original magnetic-field sensor, which makes the design more compact, lower-power, lower-cost, exhibit less jitter and improves robustness to noise generated by nearby appliances. Second, we present a low-cost hardware tuning circuit that can be used to continuously trim a micro-controller's low-power clock at runtime. Most time synchronization approaches require a CPU to periodically adjust internal counters to accommodate for clock drift. Periodic discrete updates can introduce interpolation errors as compared to continuous update approaches and they require the CPU to expend energy during these wake up periods. Our hardware-based external clock tuning circuit allows the main CPU to remain in a deep-sleep mode for extended periods while an external circuit compensates for clock drift. We show that our new synchronization circuit consumes 60% less power than the original design and is able to correct clock drift rates to within 0.01 ppm without power hungry and expensive precision clocks.

As sensor networks gain traction and begin to scale, we will be increasingly faced with challenges associated with managing large-scale time-series data. In this paper, we present a cloud-to-edge partitioned architecture called Respawn that is capable of serving large amounts of time-series data from a continuously updating datastore with access latencies low enough to support interactive real-time visualization. Respawn targets sensing systems where resource-constrained edge node devices may only have limited or intermittent network connections linking them to a cloud-backend. The cloud-backend provides aggregate storage and transparent dispatching of data queries to edge node devices. Data is downsampled as it enters the system creating a multi-resolution representation capable of lowlatency range-base queries. Lower-resolution aggregate data is automatically migrated from edge nodes to the cloud-backend both for improved consistency and caching. In order to further mask latency from users, edge nodes automatically identify and migrate blocks of data that contain statistically interesting features. We show through simulation and micro-benchmarking that Respawn is able to run on ARM-based edge node devices connected to a cloud-backend with the ability to serve thousands of clients and terabytes of data with sub-second latencies.

During low-light conditions, drivers rely mainly on headlights to improve visibility. But in the presence of rain and snow, headlights can paradoxically reduce visibility due to light reflected off of precipitation back towards the driver. Precipitation also scatters light across a wide range of angles that disrupts the vision of drivers in oncoming vehicles. In contrast to recent computer vision methods that digitally remove rain and snow streaks from captured images, we present a system that will directly improve driver visibility by controlling illumination in response to detected precipitation. The motion of precipitation is tracked and only the space around particles is illuminated using fast dynamic control. Using a physics-based simulator, we show how such a system would perform under a variety of weather conditions. We build and evaluate a proof-of-concept system that can avoid water drops generated in the laboratory.

In this paper, we present an indoor ultrasonic location tracking system that can utilize off-the-shelf audio speakers (potentially already in place) to provide fine-grained indoor position data to modern mobile devices like smartphones and tablets. We design and evaluate a communication primitive based on rate-adaptive wide-band linear frequency modulated chirp pulses that utilizes the audio bandwidth just above the human hearing frequency range where mobile devices are still sensitive. Typically transmitting data, even outside of this range, introduces broadband human audible noises (clicks) due to the non-ideal impulse response of speakers. Unlike existing audio modulation schemes, our scheme is optimized based on psychoacoustic properties. For example, all tones exhibit slowly changing power-levels and gradual frequency changes so as to minimize human perceivable artifacts. Chirps also bring the benefit of Pulse Compression, which greatly improves ranging resolution and makes them resilient to both Doppler shifts as well as multi-path propagation that typically plague indoor environments. The scheme also supports the decoding of multiple unique identifier packets being transmitted simultaneously. By applying a Time-Difference-of-Arrival (TDOA) pseudo-ranging technique the mobile devices can localize themselves without tight out-of-band synchronization with the broadcasting infrastructure. This design is not only scalable with respect to the number of transmitters and tracked devices, but also improves user privacy since the mobile devices compute their positions locally. We show through user studies and experimentation on smartphones that we are able to provide sub-meter (95% < 10cm) accurate indoor positioning in a manner that is imperceptible to humans.

In this demonstration, we show an indoor location tracking system that broadcasts ranging data to mobile phones using ultrasonic signals. The system capitalizes on the ability for many smart-phones to detect audio above the human hearing range. This approach provides enhanced ranging capabilities to mobile devices without adding to or modifying their hardware. In [1], we describe a modulation scheme based on rate-adaptive wide-band linear frequency modulated chirp pulses that can be transmitted from standard audio tweeters at just above the human hearing frequency range. Acoustic data transmissions (even outside of the human hearing range), typically introduce audible noises (clicks) due to the non-ideal impulse response of speakers. Our scheme is optimized to avoid these artifacts by using slowly changing power-levels and gradual shifts in frequency. Each speaker simultaneously transmits a uniquely identifiably signature that can be geolocated on a map. By then applying a time-difference-of-arrival (TDOA) pseudo-ranging technique the mobile devices can localize themselves without synchronizing with the broadcasting infrastructure. This design is not only scalable with respect to the number of transmitters and tracked devices, but also improves user privacy since the mobile device can compute its position locally.

In this paper, we present SAGA, a system for building energy management that provides robust multi-hop wireless sensing, actuation, device management, plotting of historical data and a server backend API to support remote access. Information collected from sensor nodes is stored locally for quick retrieval even if outside network connectivity is lost or unavailable. When network connectivity is available, data is pushed using the extensible Message Passing Protocol (XMPP). This enables external server-side archiving of historical events and additional processing as well as secure bi-directional communication from gateways behind firewalls or with dynamic IP address like those found in broadband connected homes. SAGA provides a web interface that allows devices to be easily configured with aliases and grouped together based on sensor type. Individual and groups of sensors can be plotted to show relative comparisons of different sensor values. For example, selecting to plot a group of energy metering devices shows the overall distribution of energy consumption per-device. A local multi-resolution storage system provides optimized access to recent high-resolution data along with quick retrieval of pre-computed long-term averages. SAGA has been deployed in three homes around the Pittsburgh area, collecting data for more than two years.

Time Division Multiple Access (TDMA) communication protocols in wireless sensor networks provide collision-free communication that increases energy-efficiency while maintaining deterministic packet latencies. The TDMA-ASAP[1] protocol proposed stealing neighbor's slots when networks are running at low-duty cycles to reduce the potentially large latencies of the TDMA cycle size. In this paper, we further reduce the end-to-end latencies by intelligently spreading slots across the TDMA cycle and by enhancing the stealing opportunities in the schedule, stealing slots scheduled for downstream (control) and upstream (data) messages. We also provide a practical time synchronization algorithm that operates within the TDMA schedule. In order to evaluate our new schemes, we carried out both simulation studies to show scalability and a test bed implementation of Fire Fly wireless sensor nodes to show feasibility. Our schemes provide higher peak throughput (nearly 2x) as compared to a common low-power-listen contention-based (LPL-CSMA) protocol and improves the average packet latency by as much as 5x as compared to existing TDMA protocols without slot-stealing and up to 2x as compared to TDMA-ASAP.

This paper presents a family ofRate-Harmonized Schedulersthat can be used in reservation-based operating systems to naturally cluster task execution and lump processor idle durations. While traditional approaches to energy management have focused on reducingdynamic switching powerthrough Dynamic Voltage and Frequency Scaling (DVFS), processor technology trends predict a future in whichstatic leakage powerwill begin to dominate. To this end, most modern processors provide built-in support for sleep modes withlow leakage power. However, substantial time is required to switch in/out of such sleep modes due to mechanical oscillator stabilization delays. Significant opportunities for energy saving are potentially missed due to idle gaps between executing tasks that are shorter than the time required to enter the sleep mode. Armed with apriori workload information,reservation-basedoperating systems can potentially eliminate such wasted idle durations using Rate-Harmonized Scheduling. AnEnergy-Saving Rate-Harmonized Schedulerguarantees thateveryidle duration can be used to switch into sleep mode. This paper also provides extensions to Rate-Harmonized Scheduling to support multicore processors. Empirical evaluation results are provided from an implementation in the nano-RK operating system for wireless sensor networks. Energy-Saving Rate-Harmonized Scheduling saves 16.8% energy compared to conventional Rate-Monotonic Scheduling for the task set used inSensor Andrewproject. At low utilization levels, Energy-Saving Rate-Harmonized Scheduling can save up to 39% energy on randomly generated task sets.

In recent years, there has been an increase in video surveillance systems in public and private environments due to a heightened sense of security. The next generation of surveillance systems will be able to annotate video and locally coordinate the tracking of objects while multiplexing hundreds of video streams in real-time. In this paper, we present OmniEye, a wireless distributed real-time surveillance system composed of wireless smart cameras. OmniEye is comprised of custom-designed smart camera nodes called DSPcams that communicate using an IEEE 802.11 mesh network. These cameras provide wide-area coverage and local processing with the ability to direct a sparse number of high-resolution pan, tilt and zoom (PTZ) cameras that can home onto targets of interest. Each DSPcam performs local processing to help classify events and pro-actively draw an operator's attention when necessary. In video-streaming applications, maintaining high network utilization is required in order to maximize image quality as well as the number of cameras. Our experiments show that by using the standard 802.11 DCF MAC protocol for communication, the system does not scale beyond 5-6 cameras while each camera is streaming at 1 Mbps. Also, we see high levels of jitter in video transmissions. This performance degrades further for multi-hop scenarios due to the presence of hidden nodes. In order to improve the system's scalability and reliability, we propose a Time-Synchronized Application- level MAC protocol (TSAM) capable of operating on top of existing 802.11 protocols using commodity off-the-shelf hardware. Through analysis and experimental validation, we show how TSAM is able to improve throughput and provide bounded delay. Unlike traditional CSMA-based systems, TSAM gracefully degrades in a fair manner so that existing streams can still deliver data.

Clock synchronization is highly desirable in many sensor networking applications. It enables event ordering, coordinated actuation, energy-efficient communication and duty cycling. This paper presents a novel low-power hardware module for achieving global clock synchronization by tuning to the magnetic field radiating from existing AC power lines. This signal can be used as a global clock source for battery-operated sensor nodes to eliminate drift between nodes over time even when they are not passing messages. With this scheme, each receiver is frequency-locked with each other, but there is typically a phase-offset between them. Since these phase offsets tend to be constant, a higher-level compensation protocol can be used to globally synchronize a sensor network. We present the design of an LC tank receiver circuit tuned to the AC 60Hz signal which we call a Syntonistor. The Syntonistor incorporates a low-power microcontroller that filters the signal induced from AC power lines generating a pulse-per-second output for easy interfacing with sensor nodes. The hardware consumes less than 58μW which is 2--3 times lower than the idle state of most sensor networking MAC protocols. Next, we evaluate a software clock-recovery technique running on the local microcontroller that minimizes timing jitter and provides robustness to noise. Finally, we provide a protocol that sets a global notion of time by accounting for phase-offsets. We evaluate the synchronization accuracy and energy performance as compared to in-band message passing schemes. The use of out-of-band signals for clock synchronization has the useful property of decoupling the synchronization scheme from any particular MAC protocol. Our experiments show that over a 11 day period, eight nodes distributed across the floor of the CIC building on Carnegie Mellon's campus remained synchronized on an average to less than 1ms without exchanging any radio messages beyond the initialization phase.

Energy consumption continues to be a major concern in multiple application domains including power-hungry data centers, portable and wearable devices, mobile communication devices and wireless sensor networks. While energy-constrained, many such applications must meet timing and QoS constraints for sensing, actuation or multimedia data processing. Many modern power-aware processors and microcontrollers have built-in support for active, idle and sleep operating modes. In sleep mode, substantially more energy savings can be obtained but it requires a significant amount of time to switch into and out of that mode. Hence, a significant amount of energy is lost due to idle gaps between executing tasks that are shorter than the required time for the processor to enter the sleep mode. We present a technique called rate-harmonized scheduling that naturally clusters task execution such that processor idle times are lumped together. We next introduce the energy-saving rate-harmonized scheduler which guarantees that every idle duration on the processor can be used to put the processor into sleep mode. This property can be used to even eliminate the idle power mode in processors but nevertheless it is predictable, analyzable, and saves more energy. We finally evaluate the practical benefits of rate-harmonized scheduling implemented in the nano-RK real-time operating system [1] for wireless sensor networks.

We propose a wireless medium access control (MAC) protocol that provides static-priority scheduling of messages in a guaranteed collision-free manner. Our protocol supports multiple broadcast domains, resolves the wireless hidden node problem and allows for parallel transmissions across a mesh network. Arbitration of messages is achieved without the notion of a master coordinating node, global clock synchronization or out-ofband signalling. The protocol relies on bit-dominance similar to what is used in the CAN bus except that in order to operate on a wireless physical layer, nodes are not required to receive incoming bits while transmitting. The use of bit-dominance efficiently allows for a much larger number of priorities than would be possible using existing wireless solutions. A MAC protocol with these properties enables schedulability analysis of sporadic message streams in wireless multihop networks.

With the advent of CMOS cameras, it is now possible to make compact, cheap and low-power image sensors capable of on-board image processing. These embedded vision sensors provide a rich new sensing modality enabling new classes of wireless sensor networking applications. In order to build these applications, system designers need to overcome challenges associated with limited bandwidth, limited power, group coordination and fusing of multiple camera views with various other sensory inputs. Real-time properties must be upheld if multiple vision sensors are to process data, communicate with each other and make a group decision before the measured environmental feature changes. In this paper, we present FireFly Mosaic, a wireless sensor network image processing framework with operating system, networking and image processing primitives that assist in the development of distributed vision-sensing tasks. Each FireFly Mosaic wireless camera consists of a FireFly (Rowe et al., 2006) node coupled with a CMUcam3 (Rowe et al., 2007) embedded vision processor. The FireFly nodes run the nano-RK (Eswaran et al., 2005) real-time operating system and communicate using the RT-link (Rowe et al., 2006) collision-free TDMA link protocol. Using FireFly Mosaic, we demonstrate an assisted living application capable of fusing multiple cameras with overlapping views to discover and monitor daily activities in a home. Using this application, we show how an integrated platform with support for time synchronization, a collision-free TDMA link layer, an underlying RTOS and an interface to an embedded vision sensor provides a stable framework for distributed real-time vision processing. To the best of our knowledge, this is the first wireless sensor networking system to integrate multiple coordinating cameras performing local processing.

In this paper, we propose micro-climate sensing as an effective means of enhancing conventional RF-based localization. Our system targets people-tracking applications in dynamic indoor environments, such as nursing homes, hospitals and office spaces that require simple deployment and where conventional RF-based tracking using signal strengths alone is very likely to suffer from time-varying signal attenuation and inevitable changes in the environment over time such as new furniture arrangements, people traffic, changing obstacle patterns etc. To help mitigate these effects, we use time-synchronized windows of sensor samples to dynamically associate a mobile node with its nearest beacon nodes. Comparisons and matches are always relative to the ambient attributes at the time of localization, and hence our technique automatically evolves with environmental changes. We consider this property of localization techniques to be a significant contribution and a necessary requirement for any long- lived localization system. In assisted-living environments, sensor networks likely already have basic sensors to collect contextual information about users and to monitor the environment. We propose using light, humidity, temperature and audio data samples over a short window of time to model the micro-climate of a beacon node. Using microclimate matching in conjunction with RF signal strength decreases the worst-case localization error significantly by a factor of more than 3 (from 25 m to 8 m) while making the system more resilient to environment changes. Microclimate data helps ensure at least room level location tracking even in buildings like hospitals with many rooms in close proximity.

Wireless sensor networks have traditionally focused on low duty-cycle applications where sensor data are reported periodically in the order of seconds or even longer. This is due to typically slow changes in physical variables, the need to keep node costs low and the goal of extending battery lifetime. However, there is a growing need to support real-time streaming of audio and/or low-rate video even in wireless sensor networks for use in emergency situations and short-term intruder detection. In this paper, we present FireFly, a time-synchronized sensor network platform for real-time data streaming across multiple hops. FireFly is composed of several integrated layers including specialized low-cost hardware, a sensor network operating system, a real-time link layer and network scheduling which together provide efficient support for applications with timing constraints. In order to achieve high end-to-end throughput, bounded latency and predictable lifetime, we employ hardware-based time synchronization. Multiple tasks including audio sampling, networking and sensor reading are scheduled using the nano-RK RTOS. We have implemented RT-Link, a TDMA-based link layer protocol for message exchange on well-defined time slots and pipelining along multiple hops. We use this platform to support 2-way audio streaming concurrently with sensing tasks. For interactive voice, we investigate TDMA-based slot scheduling with balanced bi-directional latency while meeting audio timeliness requirements. Finally, we describe our experimental deployment of 42 nodes in a coal mine, and present measurements of the end-to-end throughput, jitter, packet loss and voice quality

We propose RT-link, a time-synchronized link protocol for real-time wireless communication in industrial control, surveillance and inventory tracking. RT-link provides predictable lifetime for battery-operated embedded nodes, bounded end-to-end delay across multiple hops, and collision-free operation. We investigate the use of hardware-based time-synchronization for infrastructure nodes by using an AM carrier-current radio for indoors and atomic clock receivers for outdoors. Mobile nodes are synchronized via in-band software synchronization within the same framework. We identify three key observations in the design and deployment of RT-link: (a) hardware-based global-time synchronization is a robust and scalable option to in-band software-based techniques, (b) achieving global time-synchronization is both economical and convenient for indoor and outdoor deployments, (c) RT-link achieves a practical lifetime of over 2 years. Through analysis and simulation, we show that RT-link outperforms energy-efficient link protocols such as B-MAC in terms of node lifetime and end-to-end latency. The protocol supports flexible services such as on-demand end-to-end rate control and logical topology control. We implemented RT-link on the CMU FireFly sensor platform and have integrated it within the nano-RK real-time sensor OS. A 42-node network with sub-20 mus synchronization accuracy has been deployed for 3 weeks in the NIOSH Mining Research Laboratory and within two 5-story campus buildings

Much progress has been achieved in defining methods, techniques, and tools for software architecture reconstruction (SAR). However, less progress has been achieved in constructing reasoning frameworks from existing systems that support organizations in architecture analysis and design decisions. These reasoning frameworks are necessary, for example, to assemble existing components and deploy them in new system configurations. We propose a model-centric approach where this kind of reasoning is driven by the analysis of quality attribute scenarios. The scenarios and the related quality attribute models guide the SAR effort by focusing on the elicitation of model relevant artifacts. The approach further drives the model construction towards the analytical support of What If scenarios that explore responses stimulated by new requirements, such as new deployments of existing components. The paper provides two real-world case studies. The first case study introduces the model-centric reconstruction approach in the context of a large satellite tracking system. The second case study provides the construction of a time performance model for an existing embedded system in the automotive industry. The model allows us to perform cost-efficient predictions of component assemblies in new customer configurations. Copyright © 2005 John Wiley & Sons, Ltd.

Many sensor networking applications such as surveillance and environmental monitoring are time-sensitive in nature. To support such applications, we design and implement Nano-RK, a reservation-based real-time operating system (RTOS) with multi-hop networking support for use in wireless sensor networks. We support fixed-priority preemptive multitasking for guaranteeing that task deadlines are met, along with support for CPU and network bandwidth reservations. Tasks can specify their resource demands and the operating system provides timely, guaranteed and controlled access to CPU cycles and network packets in resource-constrained embedded sensor environments. We also introduce the concept of virtual energy reservations that allows the OS to enforce energy budgets associated with a sensing task by controlling resource accesses. A lightweight wireless networking stack supports packet forwarding, routing and TDMA-based network scheduling. Nano-RK has been implemented on the Atmel ATMEGA128 processor with the Chipcon CC2420 802.15.4 transceiver chip. Our results show that a light-weight embedded resource kernel with rich functionality and timing support is practical and constitutes a simple and alternative paradigm for supporting distributed sensing tasks.

Single chip heterogeneous multiprocessors (SCHMs) are becoming more commonplace, especially in portable devices where reduced energy consumption is a priority. The use of coordinated collections of processors which are simpler or which execute at lower clock frequencies is widely recognized as a means of reducing power while maintaining latency and throughput. A primary limitation of using this approach to reduce power at the system level has been the time to develop and simulate models of many processors at the instruction set simulator level. High-level models, simulators, and design strategies for SCHMs are required to enable designers to think in terms of collections of cooperating, heterogeneous processors in order to reduce power. Toward this end, this paper has two contributions. The first is to extend a unique, preexisting high-level performance simulator, the Modeling Environment for Software and Hardware (MESH), to include power annotations. MESH can be thought of as a thread-level simulator instead of an instruction-level simulator. Thus, the problem is to understand how power might be calibrated and annotated with program fragments instead of at the instruction level. Program fragments are finer-grained than threads and coarser-grained than instructions. Our experimentation found that compilers produce instruction patterns that allow power to be annotated at this level using a single number over all compiler-generated fragments executing on a processor. Since energy is power*time, this makes system runtime (i.e., performance) the dominant factor to be dynamically calculated at this level of simulation. The second contribution arises from the observation that high-level modeling is most beneficial when it opens up new possibilities for organizing designs. Thus, we introduce a design strategy, enabled by the high-level performance power-simulation, which we refer to as spatial voltage scaling. The strategy both reduces overall system power consumption and improves performance in our example. The design space for this design strategy could not be explored without high-level SCHM power-performance simulation.

In this paper we describe a functioning low cost embedded vision system which can perform basic color blob tracking at 16.7 frames per second. This system utilizes a low cost CMOS color camera module and all image data is processed by a high speed, low cost microcontroller. This eliminates the need for a separate frame grabber and high speed host computer typically found in traditional vision systems. The resulting embedded system makes it possible to utilize simple color vision algorithms in applications like small mobile robotics where a traditional vision system would not be practical.