VLDB 2026 Research / reviewers in the wild / expert
Shahriar Nirjon
dblp:162/5547 · also S. M. Shahriar Nirjon
· DBLP profile ↗
69ranked-venue papers
12as first author
21since 2021 · last 2026
0000-0003-1443-1146ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Computer networks · 37 · 8 first-author · 10 since 2021Systems, architecture and hardware · 4 · 2 first-authorHuman-computer interaction and ubiquitous computing · 3 · 1 first-author · 1 since 2021Graphics, computer vision, multimedia, augmented reality and games · 2 · 1 since 2021Artificial intelligence and machine learning · 1 · 1 since 2021Security and privacy · 1Applied, interdisciplinary, general and emerging computing · 1
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | mmAnomaly: Leveraging Visual Context for Robust Anomaly Detection in the Non-Visual World with mmWave RadarabstractmmWave radar enables human sensing in non-visual scenarios—e.g., through clothing or certain types of walls—where traditional cameras fail due to occlusion or privacy limitations. However, robust anomaly detection with mmWave remains challenging, as signal reflections are influenced by material properties, clutter, and multipath interference, producing complex, non-Gaussian distortions. Existing methods lack contextual awareness and misclassify benign signal variations as anomalies. We present mmAnomaly, a multi-modal anomaly detection framework that combines mmWave radar with RGBD input to incorporate visual context. Our system extracts semantic cues—such as scene geometry and material properties—using a fast ResNet-based classifier, and uses a conditional latent diffusion model to synthesize the expected mmWave spectrum for the given visual context. A dual-input comparison module then identifies spatial deviations between real and generated spectra to localize anomalies. We evaluate mmAnomaly on two multi-modal datasets across three applications: concealed weapon localization, through-wall intruder localization, and through-wall fall localization. The system achieves up to 94% F1 score and sub-meter localization error, demonstrating robust generalization across clothing, occlusions, and cluttered environments. These results establish mmAnomaly as an accurate and interpretable framework for context-aware anomaly detection in mmWave sensing. Tarik Reza Toha, Shao-Jung (Louie) Lu, Mahathir Monjur, Shahriar Nirjon |
SenSys | 4 |
| 2026 | EdgeTune: Efficient On-Device LLM Personalization at the EdgeabstractLarge-language models (LLMs) of roughly one billion parameters are now efficient enough to run locally on modern smartphones and embedded GPUs, enabling low-latency, privacy-preserving on-device inference. However, these devices still cannot fine-tune LLMs efficiently, hindering secure, privacy-preserving personalization without sending personal data to the cloud. Low-Rank Adaptation (LoRA) reduces trainable parameters by up to 99% [29], but gradients must still propagate the frozen backbone, leaving about two-thirds of fine-tuning FLOPs and activation traffic intact. This paper presents EdgeTune, the first on-device LLM personalization framework that directly targets these backbone costs. EdgeTune combines GradCut, an importance-aware adapter-placement algorithm that removes low-yield LoRA paths, with an analytically derived, data-free reuse–or–re-tune policy that decides when cached adapters can safely be reused across model releases. Together, they reduce the amortized cost of continual on-device LLM personalization by up to 79%. Across four text reasoning benchmarks and five 0.35–1.1B-parameter edge LLMs on Jetson Orin Nano and Pixel 6a, GradCut cuts LoRA adaptation cost by about 20% and EdgeTune lowers overall time and energy per personalization step by 70–80% while matching always-fine-tune accuracy. A month-long user study with 13 participants using three sensor-enabled mobile apps—mood journal, interactive reader, and email assistant—shows that EdgeTune maintains these savings in deployment and improves user approval ratings by 8.26–28.39% compared to state-of-the-art LoRA baselines. Rana Muhammad Shahroz Khan, Tianlong Chen 0001, Shahriar Nirjon |
SenSys | 4 |
| 2026 | Short Paper: Towards Algorithmically Grounded Embedded AI ModelsabstractEmbedded systems were once built with clarity—each line of code grounded in an algorithm, each behavior traceable and explainable. But as AI models rapidly replace classical methods in sensing, scheduling, decision, and control, we have gained accuracy at the cost of trust. Today’s neural networks are black boxes, assembled by intuition or brute force, leaving us unable to explain, debug, or control their behavior. We argue this is not just a tooling issue, but a design flaw: explainability has long been an afterthought, with networks built first and interpreted later. We advocate a principled approach where networks are grounded in algorithms and designed with internal anchors—expected intermediate behaviors, invariants, or interpretable signals—that support debugging and interpretation. This paper presents Algorithm-Informed Neural Networks (AINN)—architectures shaped by algorithms as implicit inductive bias. By decomposing algorithms into logic blocks, we build modular networks that are easier to train, interpret, and debug. As proof of concept, we present two use cases—fall detectionkeyword spotting problems—showing how algorithmic structure improves training efficiency and enables effective debugging. Md. Yusuf Sarwar Uddin, Shahriar Nirjon |
SenSys | 3 |
| 2026 | mmWEAVER: Environment-Specific mmWave Signal Synthesis from a Photo and Activity DescriptionabstractRealistic signal generation and dataset augmentation are essential for advancing mmWave radar applications such as activity recognition and pose estimation, which rely heavily on diverse, and environment-specific signal datasets. However, mmWave signals are inherently complex, sparse, and high-dimensional, making physical simulation computationally expensive. This paper presents mmWeaver, a novel framework that synthesizes realistic, environment-specific complex mmWave signals by modeling them as continuous functions using Implicit Neural Representations (INRs), achieving up to 49-fold compression. mmWeaver incorporates hypernetworks that dynamically generate INR parameters based on environmental context (extracted from RGB-D images) and human motion features (derived from text-to-pose generation via MotionGPT), enabling efficient and adaptive signal synthesis. By conditioning on these semantic and geometric priors, mmWeaver generates diverse I/Q signals at multiple resolutions, preserving phase information critical for downstream tasks such as point cloud estimation and activity classification. Extensive experiments show that mmWeaver achieves a complex SSIM of 0.88 and a PSNR of 35 dB, outperforming existing methods in signal realism while improving activity recognition accuracy by up to 7% and reducing human pose estimation error by up to 15%, all while operating 6–35→ faster than simulation-based approaches. Mahathir Monjur, Shahriar Nirjon |
WACV | 2 |
| 2025 | mmDefender: A mmWave System for On-Body Localization of Concealed Threats in Moving Persons
Shao-Jung Louie Lu, Mahathir Monjur, Sirajum Munir, Shahriar Nirjon |
EWSN | 4 |
| 2025 | mmCounter: Static People Counting in Dense Indoor Scenarios using mmWave Radar
Tarik Reza Toha, Shao-Jung Louie Lu, Shahriar Nirjon |
EWSN | 3 |
| 2025 | PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model PatchesabstractAs large language models (LLMs) increasingly shape the AI landscape, fine-tuning pretrained models has become more popular than in the pre-LLM era for achieving optimal performance in domain-specific tasks. However, pretrained LLMs such as ChatGPT are periodically evolved (i.e., model parameters are frequently updated), making it challenging for downstream users with limited resources to keep up with fine-tuning the newest LLMs for their domain application. Even though fine-tuning costs have nowadays been reduced thanks to the innovations of parameter-efficient fine-tuning such as LoRA, not all downstream users have adequate computing for frequent personalization. Moreover, access to fine-tuning datasets, particularly in sensitive domains such as healthcare, could be time-restrictive, making it crucial to retain the knowledge encoded in earlier fine-tuned rounds for future adaptation. In this paper, we present PORTLLM, a training-free framework that (i) creates an initial lightweight model update patch to capture domain-specific knowledge, and (ii) allows a subsequent seamless plugging for the continual personalization of evolved LLM at minimal cost. Our extensive experiments cover seven representative datasets, from easier question-answering tasks {BoolQ, SST2} to harder reasoning tasks {WinoGrande, GSM8K}, and models including {Mistral-7B,Llama2, Llama3.1, and Gemma2}, validating the portability of our designed model patches and showcasing the effectiveness of our proposed framework. For instance, PORTLLM achieves comparable performance to LoRA fine-tuning with reductions of up to 12.2× in GPU memory usage. Finally, we provide theoretical justifications to understand the portability of our model update patches, which offers new insights into the theoretical dimension of LLMs’ personalization. Rana Muhammad Shahroz, Pingzhi Li, Sukwon Yun, Shahriar Nirjon, Chau-Wai Wong, Tianlong Chen 0001 |
ICLR | 5 |
| 2025 | FedEXT: Differential Federated Learning with Complementary Extension of Edge Models
Shahriar Nirjon |
INFOCOM | 2 |
| 2025 | Demo Abstract: Human Strategy Meets AI Execution: An LLM-Driven Gaming AgentabstractWe introduce an intelligent mobile agent that leverages large language models (LLMs) and computer vision to interpret user commands and autonomously interact with smartphone applications. This agent continuously captures and analyzes screen content, executes actions such as taps, swipes, and text inputs, and intelligently handles ambiguous situations by prompting users for clarification. To advance this vision, we first develop a prototype focused on automating interactions in low-frame-rate mobile games like 2048 and tic-tac-toe. By taking user-defined strategies as input, the agent automates game interactions, effectively separating strategic decision-making from physical touch-based inputs. This enhances accessibility for users who cannot physically interact with a phone and for those who prefer focusing on strategy rather than execution. George S. Harris, Aidan Lok, Shahriar Nirjon |
SenSys | 3 |
| 2024 | Antler: Exploiting Task Affinity for Efficient Multitask Learning on Low-Resource Systems
Yubo Luo, Le Zhang 0021, Shahriar Nirjon |
EWSN | 4 |
| 2024 | VoiceDirect: mmWave and Audio Signal Fusion for User Localization and Speaking Direction Estimation
Mahathir Monjur, Shahriar Nirjon |
EWSN | 2 |
| 2024 | Explaining the Difference Between Edge Models and High-Accuracy Base Models for Vision Tasks
Shahriar Nirjon |
EWSN | 2 |
| 2024 | Poster: An Empirical Study in mmWave-Based 3D Human Pose Estimation
Shahriar Nirjon |
EWSN | 2 |
| 2023 | SoundSieve: Seconds-Long Audio Event Recognition on Intermittently-Powered SystemsabstractA fundamental problem of every intermittently-powered sensing system is that signals acquired by these systems over a longer period in time are also intermittent. As a consequence, these systems fail to capture parts of a longer-duration event that spans over multiple charge-discharge cycles of the capacitor that stores the harvested energy. From an application's perspective, this is viewed as sporadic bursts of missing values in the input data - which may not be recoverable using statistical interpolation or imputation methods. In this paper, we study this problem in the light of an intermittent audio classification system and design an end-to-end system - SoundSieve - that is capable of accurately classifying audio events that span multiple on-off cycles of the intermittent system. SoundSieve employs an offline audio analyzer that learns to identify and predict important segments of an audio clip that must be sampled to ensure accurate classification of the audio. At runtime, SoundSieve employs a lightweight, energy- and content-aware audio sampler that decides when the system should wake up to capture the next chunk of audio; and a lightweight, intermittence-aware audio classifier that performs imputation and on-device inference. Through extensive evaluations using popular audio datasets as well as real systems, we demonstrate that SoundSieve yields 5%--30% more accurate inference results than the state-of-the-art. Mahathir Monjur, Yubo Luo, Shahriar Nirjon |
MobiSys | 4 |
| 2022 | Demo Abstract: Capuchin: A Neural Network Model Generator for 16-bit MicrocontrollersabstractResource-optimized deep neural networks (DNNs) nowadays run on microcontrollers to perform a wide variety of audio, image and sensor data classification tasks. Despite comprehensive support for deep learning tools for 32-bit microcontrollers, performing deep learning inferences on 16-bit microcontrollers still remains a chal-lenge. Although there are some tools for implementing neural net-works on 16-bit systems, generally, there is a large gap in efficiency between the development tools for 16-bit microcontrollers and 32-bit (or higher) systems. There is also a steep learning curve that discourages beginners inexperienced with microcontrollers and programming in C to develop efficient and effective deep learning models for 16-bit microcontrollers. To fill this gap, we have created a neural network model generator that (1) automatically transfers parameters of a pre-trained DNN or CNN model from commonly used frameworks to a 16-bit microcontroller, and (2) automatically implements the model on the microcontroller to perform on-device inference. The optimization of data transfer saves time and mini-mizes chances of error, and the automatic implementation reduces the complexity to implement DNNs and CNNs on ultra-low-power microcontrollers. Le Zhang 0021, Yubo Luo, Shahriar Nirjon |
IPSN | 3 |
| 2022 | Weight Separation for Memory-Efficient and Accurate Deep Multitask LearningabstractWe propose a new concept called Weight Separation of deep neural networks (DNNs), which enables memory-efficient and accurate deep multitask learning on a memory-constrained embedded system. The goal of weight separation is to achieve extreme packing of multiple heterogeneous DNNs into the limited memory of the system while ensuring the prediction accuracy of the constituent DNNs at the same time. The proposed approach separates the DNN weights into two types of weight-pages consisting of a subset of weight parameters, i.e., shared and exclusive weight-pages. It optimally distributes the weight-pages into two levels of the system memory hierarchy and stores them separately, i.e., the shared weight-pages in primary (level-1) memory (e.g., RAM) and the exclusive weight-pages in secondary (level-2) memory (e.g., flask disk or SSD). First, to reduce the memory usage of multiple DNNs, less critical weight parameters are identified and overlapped onto the shared weight-pages that are deployed in the limited space of the primary (main) memory. Next, to retain the prediction accuracy of multiple DNNs, the essential weight parameters that play a critical role in preserving prediction accuracy are stored intact in the plentiful space of secondary memory storage in the form of exclusive weight-pages without overlapping. We implement two real systems applying the proposed weight separation: 1) a microcontroller-based multitask IoT system that performs multitask learning of 10 scaled-down DNNs by separating the weight parameters into FRAM and flash disk, and 2) an embedded GPU system that performs multitask learning of 10 state-of-the-art DNNs, separating the weight parameters into GPU RAM and eMMC. Our evaluation shows that memory efficiency, prediction accuracy, and execution time of deep multitask learning improve up to 5.9x, 2.0%, and 13.1x, respectively, without any modification of DNN models. Seulki Lee 0002, Shahriar Nirjon |
PerCom | 2 |
| 2021 | WiDeo: One Shot Domain Adaptation for Cross-Environment WiFi-Based Activity Classification Through RF-Visual Joint EmbeddingabstractThe effect of environmental change in WiFi signal is a major obstacle for generalized WiFi-based activity recognition. In this paper, we propose a novel method for adapting a WiFi-based activity classifier, that is trained on a large-scale dataset for an environment, to a new environment with only one labeled sample per activity. To this end, we propose a novel representation extraction algorithm using the supervision of visual data during the training phase. Our proposed feature extraction explicitly learns the corresponding relation between WiFi signal and movement of human body parts. For environment adaptation, we propose a framework that relies on only one labeled sample per activity class in comparison with current state-of-the-art solutions which are not suitable for few-shot adaptation. We collect data from four volunteers from five different environments and show that our proposed solution is able to achieve 28% higher accuracy than state-of-the-art solutions for environmental adaptation. Md Tamzeed Islam, Shahriar Nirjon |
DCOSS | 2 |
| 2021 | SmartON: Just-in-Time Active Event Detection on Energy Harvesting SystemsabstractWe propose SmartON, a batteryless system that learns to wake up proactively at the right moment in order to detect events of interest. It does so by adapting the duty cycle to match the distribution of event arrival times under the constraints of harvested energy. While existing energy harvesting systems either wake up periodically at a fixed rate to sense and process the data, or wake up only in accordance with the availability of the energy source, SmartON employs a three-phase learning framework to learn the energy harvesting pattern as well as the pattern of events at run-time, and uses that knowledge to wake itself up when events are most likely to occur. The three-phase learning framework enables rapid adaptation to environmental changes in both short and long terms. Being able to remain asleep more often than a CTID (charging-then-immediate-discharging) wake-up system and adapt to the event pattern, SmartON is able to reduce energy waste, increase energy efficiency, and capture more events. To realize Smar-tON we have developed a dedicated hardware platform whose power management module activates capacitors on-the-fly to dynamically increase its storage capacitance. We conduct both simulation-driven and real-system experiments to demonstrate that SmartON captures 1X–7X more events and is 8X–17X more energy-efficient than a CTID system. All source code and hardware design files are open-sourced1. Yubo Luo, Shahriar Nirjon |
DCOSS | 2 |
| 2021 | Sound-Adapter: Multi-Source Domain Adaptation for Acoustic Classification Through Domain DiscoveryabstractThe accuracy of an audio classifier drops when it is trained and tested in different conditions aka domains, e.g., different devices, different environments, or their combinations. Previous works have proposed audio domain adaptation techniques for a special case where the training data are recorded with a single source microphone and the model is applied to test data recorded with a different but single target microphone (i.e., single source to single target domain adaptation). In this paper, we solve a more generic and practical problem where the goal is to adapt models that are trained on data from more than one acoustic (i.e., multi-source domain adaptation). Unlike previous works, the proposed method does not assume availability of recording metadata (i.e., domain labels) in the training data---which makes the adaptation problem harder. To solve this, we propose the first multi-task deep neural network architecture to cluster audio samples according to their domain in an unsupervised way. Using the inferred domain information, we perform domain adaptation to remove biases due to domain heterogeneity from the machine learning model. We conduct extensive experiments on an empirical dataset that we collect from five domains as well as on a public dataset. Our results show that the proposed technique has a mean accuracy of 87% for domain discovery in a five domain scenario and its model adaptation step improves acoustic event classification accuracy by up to 21% when compared to state-of-the-art algorithms on datasets containing samples from multiple source domains. Md Tamzeed Islam, Shahriar Nirjon |
IPSN | 2 |
| 2021 | Deep Functional Network (DFN): Functional Interpretation of Deep Neural Networks for Intelligent Sensing SystemsabstractWe introduce Deep Functional Network (DFN) that approximates a black-box Deep Neural Network (DNN) to a functional program consisting of a set of well-known functions and data flows among them. A DFN not only provides a semantic interpretation of a DNN but also enables easy deployment and optimization of the translated program according to the requirements and constraints of the target intelligent sensing system. To interpret a DNN, we propose the DFN framework consisting of two steps: 1) function estimation that estimates the distribution of functions likely to be used in the source DNN and 2) network formation that finds a functional network in the form of a directed acyclic graph (DAG) given the estimated function distribution. Our empirical study conducted with 16 state-of-the-art DNNs demonstrates that the generated DFNs provide semantic understandings of the DNNs along with comparable classification accuracy to the source DNNs. We implement two intelligent sensing systems that use the proposed DFN: 1) a mobile robot that avoids obstacles detected by a camera and 2) a smartphone-based human activity recognizer using IMU sensors, where different sizes of DFNs are generated to complete the task under various resource budgets, i.e., execution time and energy consumption dynamically imposed by run-time scenarios. The experiment result demonstrates that a set of DFNs generated from a single DNN enable both systems to achieve the desired performance under various resource constraints based on semantic understanding of the DNNs. Seulki Lee 0002, Shahriar Nirjon |
IPSN | 2 |
| 2021 | Exploiting scene and body contexts in controlling continuous vision body cameras
Shiwei Fang, Ketan Mayer-Patel, Shahriar Nirjon |
Ad Hoc Networks | 3 |
| 2020 | EyeFi: Fast Human Identification Through Vision and WiFi-based Trajectory MatchingabstractHuman sensing, motion trajectory estimation, and identification are central to a wide range of applications in many domains such as retail stores, surveillance, public safety, public address, smart homes and cities, and access control. Existing solutions either require facial recognition or installation and maintenance of multiple units, or they lack long-term re-identification capability. In this paper, we propose a novel system - called EyeFi- that combines WiFi and camera on a standalone device to overcome these limitations. EyeFi integrates a WiFi chipset to an overhead camera and fuses motion trajectories obtained from both vision and RF modalities to identify individuals. In order to do that, EyeFi uses a student-teacher model to train a neural network to estimate the Angle of Arrival (AoA) of WiFi packets from the CSI values. Based on extensive evaluation using real-world data, we observe that EyeFi improves WiFi CSI based AoA estimation accuracy by more than 30% and offers 3,800 times computational speed over the state-of-the-art solution. In a real-world environment, EyeFi's accuracy of person identification averages 75% when the number of people varies from 2 to 10. Shiwei Fang, Md Tamzeed Islam, Sirajum Munir, Shahriar Nirjon |
DCOSS | 4 |
| 2020 | Wi-Fringe: Leveraging Text Semantics in WiFi CSI-Based Device-Free Named Gesture Recognitionabstractgestures, i.e., activities and gestures that have a semantically meaningful name in English language, as opposed to arbitrary free-form gestures. Given a list of activities (only their names in English text), along with zero or more training examples (WiFi CSI values) per activity, Wi-Fringe is able to detect all activities at runtime. We show for the first time that by utilizing the state-of-the-art semantic representation of English words, which is learned from datasets like the Wikipedia (e.g., Google's word-to-vector [1]) and verb attributes learned from how a word is defined (e.g, American Heritage Dictionary), we can enhance the capability of WiFi-based named gesture recognition systems that lack adequate training examples per class. We propose a novel cross-domain knowledge transfer algorithm between radio frequency (RF) and text to lessen the burden on developers and end-users from the tedious task of data collection for all possible activities. To evaluate Wi-Fringe, we collect data from four volunteers in a multi-person apartment and an office building for a total of 20 activities. We empirically quantify the trade-off between the accuracy and the number of unseen activities. Md Tamzeed Islam, Shahriar Nirjon |
DCOSS | 2 |
| 2020 | SuperRF: Enhanced 3D RF Representation Using Stationary Low-Cost mmWave Radar
Shiwei Fang, Shahriar Nirjon |
EWSN | 2 |
| 2020 | Fast and scalable in-memory deep multitask learning via neural weight virtualizationabstractThis paper introduces the concept of Neural Weight Virtualization - which enables fast and scalable in-memory multitask deep learning on memory-constrained embedded systems. The goal of neural weight virtualization is two-fold: (1) packing multiple DNNs into a fixed-sized main memory whose combined memory requirement is larger than the main memory, and (2) enabling fast in-memory execution of the DNNs. To this end, we propose a two-phase approach: (1) virtualization of weight parameters for fine-grained parameter sharing at the level of weights that scales up to multiple heterogeneous DNNs of arbitrary network architectures, and (2) in-memory data structure and run-time execution framework for in-memory execution and context-switching of DNN tasks. We implement two multitask learning systems: (1) an embedded GPU-based mobile robot, and (2) a microcontroller-based IoT device. We thoroughly evaluate the proposed algorithms as well as the two systems that involve ten state-of-the-art DNNs. Our evaluation shows that weight virtualization improves memory efficiency, execution time, and energy efficiency of the multitask learning systems by 4.1x, 36.9x, and 4.2x, respectively. Seulki Lee 0002, Shahriar Nirjon |
MobiSys | 2 |
| 2020 | Scheduling Computational and Energy Harvesting Tasks in Deadline-Aware Intermittent SystemsabstractThe sporadic nature of harvestable energy and the mutually exclusive computing and charging cycles of intermittently powered batteryless systems pose a unique and challenging real-time scheduling problem. Existing literature focus either on the time or the energy constraints but not both at the same time. In this paper, we propose two scheduling algorithms, named Celebi-Offline and Celebi-Online, for intermittent systems that schedule both computational and energy harvesting tasks by harvesting the required minimum amount of energy while maximizing the schedulability of computational jobs. To evaluate Celebi, we conduct simulation as well as trace-based and real-life experiments. Our results show that the proposed Celebi-Offline algorithm has 92% similar performance as an optimal scheduler, and Celebi-Online scheduler schedules 8% - 22% more jobs than the earliest deadline first (EDF), rate monotonic (RM), and as late as possible (ALAP) scheduling algorithms. We deployed solar-powered batteryless systems where four intermittent applications are executed in the TI-MSP430FR5994 microcontroller and demonstrate that the system with Celebi-Online misses 63% less deadline than a non-realtime system and 8% less deadline than the system with a baseline (as late as possible) scheduler. Bashima Islam, Shahriar Nirjon |
RTAS | 2 |
| 2020 | SubFlow: A Dynamic Induced-Subgraph Strategy Toward Real-Time DNN Inference and TrainingabstractWe introduce SubFlow-a dynamic adaptation and execution strategy for a deep neural network (DNN), which enables real-time DNN inference and training. The goal of SubFlow is to complete the execution of a DNN task within a timing constraint that may dynamically change while ensuring comparable performance to executing the full network by executing a subset of the DNN at run-time. To this end, we propose two online algorithms that enable SubFlow: 1) dynamic construction of a sub-network which constructs the best subnetwork of the DNN in terms of size and configuration, and 2) time-bound execution which executes the sub-network within a given time budget either for inference or training. We implement and open-source SubFlow by extending TensorFlow with full compatibility by adding SubFlow operations for convolutional and fully-connected layers of a DNN. We evaluate SubFlow with three popular DNN models (LeNet-5, AlexNet, and KWS), which shows that it provides flexible run-time execution and increases the utility of a DNN under dynamic timing constraints, e.g., lx-6.7x range of dynamic execution speed with average -3% of performance (inference accuracy) difference. We also implement an autonomous robot as an example system that uses SubFlow and demonstrate that its obstacle detection DNN is flexibly executed to meet a range of deadlines that varies depending on its running sped. Seulki Lee 0002, Shahriar Nirjon |
RTAS | 2 |
| 2020 | Fusing wifi and camera for fast motion tracking and person identification: demo abstractabstractHuman sensing, motion tracking, and identification are at the center of numerous applications such as customer analysis, public safety, smart cities, and surveillance. To enable such capabilities, existing solutions mostly rely on vision-based approaches, e.g., facial recognition that is perceived to be too privacy invasive. Other camera-based approaches using body appearances lack long-term re-identification capability. WiFi-based approaches require the installation and maintenance of multiple units. We propose a novel system - called EyeFi [2] - that overcomes these limitations on a standalone device by fusing camera and WiFi data. We use a three-antenna WiFi chipset to measure WiFi Channel State Information (CSI) to estimate the Angle of Arrival (AoA) using a neural network trained with a novel student-teacher model. Then, we perform cross modal (WiFi, camera) trajectory matching to identify individuals using the MAC address of the incoming WiFi packets. We demonstrate our work using real-world data and showcase improvements over traditional optimization-based methods in terms of accuracy and speed. Shiwei Fang, Sirajum Munir, Shahriar Nirjon |
SenSys | 3 |
| 2020 | Reliable Communication and Latency Bound Generation in Wireless Cyber-Physical SystemsabstractLow-power wireless communication has been widely used in cyber-physical systems that require time-critical data delivery. Achieving this goal is challenging because of link burstiness and interference. Based on significant empirical evidence of 21 days and over 3.6 M packet transmissions per link, we propose both routing and scheduling algorithms that produce latency bounds of the real-time periodic streams and accounts for both link bursts and interference. The solution is achieved through the definition of a new metric B max that characterizes links by their maximum burst length, and by choosing a novel least-burst-route that minimizes the sum of worst-case burst lengths over all links in the route. With extensive data-driven analysis, we show that our algorithms outperform existing solutions by achieving accurate latency bound with much less energy consumption. In addition, a testbed evaluation consisting of 48 nodes spread across a floor of a building shows that we obtain 100% reliable packet delivery within derived latency bounds. We also demonstrate how performance deteriorates and discuss its implications for wireless networks with insufficient high-quality links. Sirajum Munir, Hao-Tsung Yang, Shan Lin 0001, Shahriar Nirjon, Lin Chen 0002, Enamul Hoque 0002, John A. Stankovic, Kamin Whitehouse |
ACM Trans. Cyber Phys. Syst. | 4 |
| 2019 | ZenCam: Context-Driven Control of Autonomous Body CamerasabstractIn this paper, we present - ZenCam, which is an always-on body camera that exploits readily available information in the encoded video stream from the on-chip firmware to classify the dynamics of the scene. This scene-context is further combined with simple inertial measurement unit (IMU)-based activity level-context of the wearer to optimally control the camera configuration at run-time to keep the device under the desired energy budget. We describe the design and implementation of ZenCam and thoroughly evaluate its performance in real-world scenarios. Our evaluation shows a 29.8-35% reduction in energy consumption and 48.1-49.5% reduction in storage usage when compared to a standard baseline setting of 1920×1080 at 30fps while maintaining a competitive or better video quality at the minimal computational overhead. Shiwei Fang, Ketan Mayer-Patel, Shahriar Nirjon |
DCOSS | 3 |
| 2019 | Securing the Insecure Link of Internet-of-Things Using Next-Generation Smart GatewaysabstractSince low-cost IoT devices have limited computing resources and are often unable to guarantee sufficient security, it is imperative to ensure secure communication between IoT gateways and cloud. In this paper, we propose a flow-level adaptive mobile VPN solution specifically tailored for IoT ecosystems, called AdamVPN, which adapts its configuration dynamically at runtime in order to improve IoT gateway's application-level throughput while conforming to the security and privacy guarantees simultaneously. Our deployment experiments in both Wi-Fi and cellular environments demonstrate that AdamVPN significantly improves throughput by 2.75×-3.0× for Wi-Fi and 1.8×-2.16× for cellular networks when compared to OpenVPN. Syed Rafiul Hussain, Shahriar Nirjon, Elisa Bertino |
DCOSS | 2 |
| 2019 | On-device training from sensor data on batteryless platforms: poster abstractabstractIn this paper, we argue that the fusion of machine learning (ML) and batteryless computing systems enables true lifelong learning in mobile devices. The lack of learning from experience in current batteryless systems makes them ignorant of changes in their operating environment. Due to high communication cost, latency, privacy, and dependency issues of offloading computation to an edge device, on-device training is a solution for batteryless systems to learn and adapt in dynamically changing environments. Combining batteryless systems and ML is however a challenging task. Sporadic energy supply and limited resources in a batteryless system cause execution-discontinuity and data-constraints in ML processes. To understand these challenges, we identify suitable ML tasks for such systems and study the energy producers, i.e., harvesters, and consumers, i.e., intermittently executable tasks in a ML pipeline. Using a trace-driven simulation, we demonstrate the feasibility of on-device training of a batteryless learner. Bashima Islam, Yubo Luo, Seulki Lee 0002, Shahriar Nirjon |
IPSN | 4 |
| 2019 | SoundSemantics: exploiting semantic knowledge in text for embedded acoustic event classificationabstractIn this paper, we propose a fundamentally different approach to acoustic event classification that exploits knowledge from the textual domain to deal with a well-known pain point in audio event classification---i.e., the lack of adequate training examples. We show that by exploiting existing context-aware semantic representation of English words (e.g., Google's word-to-vector [33]) that is generated from a massive amount of English texts on the Internet, it is possible to classify acoustic events even when there are a few or no training examples for a wide variety of sounds. Our approach works in application scenarios where a system wants to learn a number of predefined categories of acoustic events, but it does not have enough training examples per category, and/or for some categories, it does not have any training examples at all. We solve this problem by combining a robust audio representation step, followed by a cross-modal projection of the audio representation onto textual representation. Our approach is different from techniques such as one-shot and data augmentation that do not consider the cross-domain knowledge transfers. We develop a generic mobile application for audio event detection where a user can input a list of desired sound types, along with training audio clips for some of those sound types, and the system is able to recognize all types of sounds (at varying level of accuracy depending on the number of classes that do not have any training examples), which is not achievable by any existing audio classifier that we are aware of. We evaluate the performance of the proposed system on an empirical dataset [41] as well as by deploying the application in two real-world scenarios. The accuracy of the classifier lies between 60%-90% for a 6--10 class problem when the number of classes that do not have any training examples is varied between 2--5. Md Tamzeed Islam, Shahriar Nirjon |
IPSN | 2 |
| 2019 | Neuro.ZERO: a zero-energy neural network accelerator for embedded sensing and inference systemsabstractWe introduce Neuro.ZERO---a co-processor architecture consisting of a main microcontroller (MCU) that executes scaled-down versions of a deep neural network1 (DNN) inference task, and an accelerator microcontroller that is powered by harvested energy and follows the intermittent computing paradigm [76]. The goal of the accelerator is to enhance the inference performance of the DNN that is running on the main microcontroller. Neuro.ZERO opportunistically accelerates the run-time performance of a DNN via one of its four acceleration modes: extended inference, expedited inference, ensemble inference, and latent training. To enable these modes, we propose two sets of algorithms: (1) energy and intermittence-aware DNN inference and training algorithms, and (2) a fast and high-precision adaptive fixed-point arithmetic that beats existing floating-point and fixed-point arithmetic in terms of speed and precision, respectively, and achieves the best of both. To evaluate Neuro.ZERO, we implement low-power image and audio recognition applications and demonstrate that their inference speedup increases by 1.6× and 1.7×, respectively, and the inference accuracy increases by 10% and 16%, respectively, when compared to battery-powered single-MCU systems. Seulki Lee 0002, Shahriar Nirjon |
SenSys | 2 |
| 2019 | Improving Pedestrian Safety in Cities Using Intelligent Wearable SystemsabstractWith the prevalence of smartphones, pedestrians and joggers today often walk or run while listening to music. Since they are deprived of their auditory senses that would have provided important cues to dangers, they are at a much greater risk of being hit by cars or other vehicles. In this paper, we build a wearable system that uses multichannel audio sensors embedded in a headset to help detect and locate cars from their honks, engine, and tire noises, and warn pedestrians of imminent dangers of approaching cars. We demonstrate that using a segmented architecture consisting of headset-mounted audio sensors, a front-end hardware platform that performs signal processing and feature extraction, and machine learning-based classification on a smartphone, we are able to provide early danger detection in real time, from up to 60 m away, and alert the user with low latency and high accuracy. To further reduce power consumption of the battery-powered wearable headset, we implement a custom-designed integrated circuit that is able to compute delays between multiple channels of audio with nW power consumption. A regression-based method for sound source localization, angle via polygonal regression, is proposed and used in combination with the IC to improve the granularity and robustness of localization. Stephen Xia, Daniel de Godoy, Bashima Islam, Md Tamzeed Islam, Shahriar Nirjon, Peter R. Kinget, Xiaofan Jiang 0001 |
IEEE Internet Things J. | 5 |
| 2018 | Duty-Cycle-Aware Real-Time Scheduling of Wireless Links in Low Power WANsabstractLow Power Wide Area Networks (LPWANs) are an excellent fit to city-scale IoT applications becuase of their long range and a battery life of several years, and a data rate of 25-50kbps, which is sufficient to carry IoT traffic. However, a practical limitation of a LPWAN-based real-time wireless network is the duty-cycle limit imposed on the sub-1GHz band by the FCC. In this paper, we overcome this challenge by proposing the first duty-cycle-aware wireless link scheduling algorithm for real-time LPWANs that considers the urgency of the packets as well as the availability of the wireless channels. The proposed algorithm is implemented in a five-node, wide-area outdoor test-bed in multiple realworld scenarios. Simulation results are provided to quantify its performance under different settings (e.g. larger networks, variety of workloads, and multiple baselines). In both realworld deployments and simulations, the proposed algorithm outperforms standard scheduling algorithms in terms of link schedulability, deadline misses, and buffer size. Md Tamzeed Islam, Bashima Islam, Shahriar Nirjon |
DCOSS | 3 |
| 2018 | Knowledge Transfer Between Embedded ControllersabstractAlthough many Cyber-Physical Systems (CPS) have similarities among themselves, their control systems are often designed from the scratch. As a result, the knowledge of one expert control system does not come into use in designing and improving other types of control systems. In this paper, we explore the problem of knowledge transfer between two embedded control systems - which enables us to design effective and accurate control systems efficiently and at a large scale. To realize this idea, we formally define the problem of transferring knowledge between two linear time-variant systems. We derive necessary conditions for transferring parameters between two scalar systems as well as two high-order systems. We describe the transfer process which constitutes of a parameter update procedure, a convergence test, and a stability test. We derive a closed-form expression to quantify the performance benefit of the proposed technique in terms of the speed of convergence of system parameter adaptation process. In order to demonstrate the efficacy of the proposed technique, we conduct experiments with a real robotic arm as well as a mobile robot simulator. Our results show that the robotic arm learns its system dynamics 3--5 times faster when it uses transferred knowledge from a well-adapted robotic hand. Similarly, with transferred knowledge, the mobile robot navigates successfully to its target location while making 10 times less learning errors. Seulki Lee 0002, Shahriar Nirjon |
DCOSS | 2 |
| 2018 | Deep Beacon: Image Storage and Broadcast over BLE Using Variational Autoencoder Generative Adversarial NetworkabstractThis paper describes Deep Beacon which uses a set of cheap, low-power, storage-constrained Bluetooth Low Energy (BLE) devices to beacon (i.e. broadcast) a color image over a very long period (months, as opposed to days or weeks). The system employs deep neural network image encoder to encode a given input image and generates an extremely compact representation (as small as 10 bytes) of the image. At the receiver end, another deep neural network decoder runs on a mobile device which decodes (i.e. generates) the original image from the BLE broadcast messages. We evaluate Deep Beacon's performance using hand-written digit images and different types of RGB images that contain objects such as birds, flowers, and traffic signs. We empirically determine the tradeoffs between the system lifetime and the quality of broadcast images, and determine an optimal set of parameters for our system, under user-specified constraints such as the number of available beacon devices, maximum latency, and life expectancy. We develop a smartphone application that takes an image and user-requirements as inputs, shows previews of different quality output images, writes the encoded image into a set of beacons, and reads the broadcasted image back. Our evaluation shows that one beacon device is capable of broadcasting high-quality images (90% structurally similar to original images) for a year-long continuous broadcasting, and both the lifetime and the image quality improve when two beacons are used. Chong Shao, Shahriar Nirjon |
DCOSS | 2 |
| 2018 | A motion-triggered stereo camera for 3D experience capture: demo abstractabstractThis demo is an implementation of our motion-triggered camera system that captures, processes, stores, and transmits 3D visual information of a real-world environment using a low-cost camera-based sensor system that is constrained by its limited processing capability, storage, and battery life. This system can be used in applications such as capturing and sharing 3D content in the social media, training people in different professions, and post-facto analysis of an event. This system uses off-the-shelf hardware and standard computer vision algorithms. Its novelty lies in the ability to optimally control camera data acquisition and processing stages to guarantee the desired quality of captured information and battery life. The design of the controller is based on extensive measurements and modeling of the relationships between the linear and angular motion of a camera and the quality of generated 3D point clouds as well as the battery life of the system. To achieve this, we 1) devise a new metric to quantify the quality of generated 3D point clouds, 2) formulate an optimization problem to find an optimal trigger point for the camera system and prolongs its battery life while maximizing the quality of captured 3D environment and 3) make the model adaptive so that the system evolves and its performance improves over time. Bashima Islam, Md Tamzeed Islam, Shahriar Nirjon |
IPSN | 3 |
| 2018 | Glimpse.3D: a motion-triggered stereo body camera for 3D experience capture and previewabstractThe Glimpse.3D is a body-worn camera that captures, processes, stores, and transmits 3D visual information of a real-world environment using a low-cost camera-based sensor system that is constrained by its limited processing capability, storage, and battery life. The 3D content is viewed on a mobile device such as a smartphone or a virtual reality headset. This system can be used in applications such as capturing and sharing 3D content in the social media, training people in different professions, and post-facto analysis of an event. Glimpse.3D uses off-the-shelf hardware and standard computer vision algorithms. Its novelty lies in the ability to optimally control camera data acquisition and processing stages to guarantee the desired quality of captured information and battery life. The design of the controller is based on extensive measurements and modeling of the relationships between the linear and angular motion of a body-worn camera and the quality of generated 3D point clouds as well as the battery life of the system. To achieve this, we 1) devise a new metric to quantify the quality of generated 3D point clouds, 2) formulate an optimization problem to find an optimal trigger point for the camera system that prolongs its battery life while maximizing the quality of captured 3D environment, and 3) make the model adaptive so that the system evolves and its performance improves over time. Bashima Islam, Md Tamzeed Islam, Shahriar Nirjon |
IPSN | 3 |
| 2018 | Rethinking ranging of unmodified BLE peripherals in smart city infrastructureabstractMobility tracking of IoT devices in smart city infrastructures such as smart buildings, hospitals, shopping centers, warehouses, smart streets, and outdoor spaces has many applications. Since Bluetooth Low Energy (BLE) is available in almost every IoT device in the market nowadays, a key to localizing and tracking IoT devices is to develop an accurate ranging technique for BLE-enabled IoT devices. This is, however, a challenging feat as billions of these devices are already in use, and for pragmatic reasons, we cannot propose to modify the IoT device (a BLE peripheral) itself. Furthermore, unlike WiFi ranging - where the channel state information (CSI) is readily available and the bandwidth can be increased by stitching 2.4GHz and 5GHz bands together to achieve a high-precision ranging, an unmodified BLE peripheral provides us with only the RSSI information over a very limited bandwidth. Accurately ranging a BLE device is therefore far more challenging than other wireless standards. In this paper, we exploit characteristics of BLE protocol (e.g. frequency hopping and empty control packet transmissions) and propose a technique to directly estimate the range of a BLE peripheral from a BLE access point by multipath profiling. We discuss the theoretical foundation and conduct experiments to show that the technique achieves a 2.44m absolute range estimation error on average. Bashima Islam, Mostafa Uddin, Sarit Mukherjee, Shahriar Nirjon |
MMSys | 4 |
| 2018 | Lifelong Learning on Harvested EnergyabstractWe introduce the vision of lifelong and intermittent learning, which will enable batteryless computing platforms to execute a certain class of machine learning tasks. We identify key properties and challenges to learning on harvested energy which relates to the semantics of machine learning tasks. Each of these challenges leads to a new research direction. We envision that a big chunk of research on batteryless IoT devices in the next 5-10 years will be about making them capable of continuously learning throughout their lifetime. Concepts related to intermittent learning will be at the heart of those works. Shahriar Nirjon |
MobiSys | 1 |
| 2018 | AI-Enhanced 3D RF Representation Using Low-Cost mmWave RadarabstractThis paper introduces a system that takes radio frequency (RF) signals from an off-the-shelf, low-cost, 77 GHz mm Wave radar and produces an enhanced 3D RF representation of a scene. Such a system can be used in scenarios where camera and other types of sensors do not work, or their performance is impacted due to bad lighting conditions and occlusions, or an alternate RF sensing system like synthetic aperture radar (SAR) is too large, inconvenient, and costly. The enhanced RF representation can be used in numerous applications such as robot navigation, human-computer interaction, and patient monitoring. We use off-the-shelf parts to capture RF signals and collect our own data set for training and testing of the approach. The novelty of the system lies in its use of AI to generate a fine-grained 3D representation of an RF scene from its sparse RF representation which a mm Wave radar of the same class cannot achieve. Shiwei Fang, Shahriar Nirjon |
SenSys | 2 |
| 2018 | Secure Seamless Bluetooth Low Energy Connection Migration for Unmodified IoT DevicesabstractAt present, Bluetooth Low Energy (BLE) is dominantly used in commercially available Internet of Things (IoT) devices-such as smart watches, fitness trackers, and smart appliances. Compared to classic Bluetooth, BLE has been simplified in many ways that include its connection establishment, data exchange, and encryption processes. Unfortunately, this simplification comes at a cost. For example, only a star topology is supported in BLE environments and a peripheral (an IoT device) can communicate with only one gateway (e.g., a smartphone, or a BLE hub) at any given set time. When a peripheral goes out of range and thus loses connectivity to a gateway, it cannot connect and seamlessly communicate with another gateway without user interventions. In other words, BLE connections are not automatically migrated or handed-off to another gateway. In this paper, we propose SeamBlue1, which brings secure seamless connectivity to BLE-capable mobile IoT devices in an environment that consists of a network of gateways. Our framework ensures that unmodified, commercial off-the-shelf BLE devices seamlessly and securely connect to a nearby gateway without any user intervention. Syed Rafiul Hussain, Shagufta Mehnaz, Shahriar Nirjon, Elisa Bertino |
IEEE Trans. Mob. Comput. | 3 |
| 2017 | Seamless and Secure Bluetooth LE Connection MigrationabstractAt present, Bluetooth Low Energy (BLE) is dominantly used in commercially available Internet of Things (IoT) devices -- such as smart watches, fitness trackers, and smart appliances. Compared to classic Bluetooth, BLE has been simplified in many ways that include its connection establishment, data exchange, and encryption processes. Unfortunately, this simplification comes at a cost. For example, only a star topology is supported in BLE environments and a peripheral (an IoT device) can communicate with only one gateway (e.g. a smartphone, or a BLE hub) at a set time. When a peripheral goes out of range, it loses connectivity to a gateway, and cannot connect and seamlessly communicate with another gateway without user interventions. In other words, BLE connections do not get automatically migrated or handed-off to another gateway. In this paper, we propose a system which brings seamless connectivity to BLE-capable mobile IoT devices in an environment that consists of a network of gateways. Our framework ensures that unmodified, commercial off-the-shelf BLE devices seamlessly and securely connect to a nearby gateway without any user intervention. Syed Rafiul Hussain, Shagufta Mehnaz, Shahriar Nirjon, Elisa Bertino |
CODASPY | 3 |
| 2017 | SeamBlue: Seamless Bluetooth Low Energy Connection Migration for Unmodified IoT Devices
Syed Rafiul Hussain, Shagufta Mehnaz, Shahriar Nirjon, Elisa Bertino |
EWSN | 3 |
| 2017 | SoundSifter: Mitigating Overhearing of Continuous Listening DevicesabstractIn this paper, we study the overhearing problem of continuous acoustic sensing devices such as Amazon Echo, Google Home, or such voice-enabled home hubs, and develop a system called SoundSifter that mitigates personal or contextual information leakage due to the presence of unwanted sound sources in the acoustic environment. Instead of proposing modifications to existing home hubs, we build an independent embedded system that connects to a home hub via its audio input. Considering the aesthetics of home hubs, we envision SoundSifter as a smart sleeve or a cover for these devices. SoundSifter has hardware and software to capture the audio, isolate signals from distinct sound sources, filter out signals that are from unwanted sources, and process the signals to enforce policies such as personalization before the signals enter into an untrusted system like Amazon Echo or Google Home. We conduct empirical and real-world experiments to demonstrate that SoundSifter runs in real-time, is noise resilient, and supports selective and personalized voice commands that commercial voice-enabled home hubs do not. Md Tamzeed Islam, Bashima Islam, Shahriar Nirjon |
MobiSys | 3 |
| 2016 | Years-Long Binary Image Broadcast Using Bluetooth Low Energy BeaconsabstractThis paper describes the first 'image beacon' system that is capable of broadcasting binary images over a very long period (years, as opposed to days or weeks) using a set of cheap, low-power, memory-constrained Bluetooth Low Energy (BLE) beacon devices. We design a patch-based image encoding algorithm to produce encoded images of reasonably high quality, having sizes of as low as 16 bytes -- without any prior knowledge of the test images. We test our system with different types of images that contain hand-written alphanumeric characters, geometric shapes, and arbitrary binary images having complex shapes and curves. We empirically determine the tradeoffs between the system lifetime and the quality of broadcasted images, and determine an optimal set of parameters for our system, under user-specified constraints such as the number of available beacon devices, maximum latency, and life expectancy. We develop a smartphone application that takes an image and user-requirements as inputs, shows previews of different quality output images, writes the encoded image into a set of beacons, and reads the broadcasted image back. Our evaluation shows that a set of 2 -- 3 beacons is capable of broadcasting high-quality images (75% -- 90% structurally similar to original images) for a year-long continuous broadcasting, and both the lifetime and the image quality improve when more beacons are used. Chong Shao, Shahriar Nirjon, Jan-Michael Frahm |
DCOSS | 2 |
| 2016 | SEUS: A Wearable Multi-Channel Acoustic Headset Platform to Improve Pedestrian Safety: Demo AbstractabstractWith the prevalence of smartphones, pedestrians and joggers today often walk or run while listening to music. Since they are deprived of their auditory senses that would have provided important cues to dangers, they are at a much greater risk of being hit by cars or other vehicles. In this demonstration we present SEUS, a wearable system aimed at Sense Enhancement for Urban Safety. SEUS uses a three-stage architecture, consisting of headset mounted audio sensors, an embedded front-end for signal processing and feature extraction, and machine learning based classification on a smartphone, to provide early danger detection for pedestrians in real-time. Rishikanth Chandrasekaran, Daniel de Godoy, Stephen Xia, Md Tamzeed Islam, Bashima Islam, Shahriar Nirjon, Peter R. Kinget, Xiaofan Jiang 0001 |
SenSys | 6 |
| 2015 | TypingRing: A Wearable Ring Platform for Text InputabstractThis paper presents TypingRing, a wearable ring platform that enables text input into computers of different forms, such as PCs, smartphones, tablets, or even wearables with tiny screens. The basic idea of TypingRing is to have a user wear a ring on his middle finger and let him type on a surface - such as a table, a wall, or his lap. The user types as if a standard QWERTY keyboard is lying underneath his hand but is invisible to him. By using the embedded sensors TypingRing determines what key is pressed by the user. Further, the platform provides visual feedback to the user and communicates with the computing device wirelessly. This paper describes the hardware and software prototype of TypingRing and provides an in-depth evaluation of the platform. Our evaluation shows that TypingRing is capable of detecting and sending key events in real-time with an average accuracy of 98.67%. In a field study, we let seven users type a paragraph with the ring, and we find that TypingRing yields a reasonable typing speed (e.g., 33-50 keys per minute) and their typing speed improves over time. Shahriar Nirjon, Jeremy Gummeson, Dan Gelb, Kyu-Han Kim |
MobiSys | 1 |
| 2015 | Demo: AsthmaGuide: An Ecosystem for Asthma Monitoring and AdviceabstractAsthmaGuide is a smartphone and cloud based asthma system in which a smart phone is used as a hub for collecting a comprehensive collection of information. The data, including data over time, is then displayed in a cloud web application for both patients and healthcare providers to view. AsthmaGuide also provides an advice and alarm infrastructure based on the collected data and parameters set by healthcare providers. With these components, AsthmaGuide provides a comprehensive ecosystem that allows patients to be involved in their own health and also allows doctors to provide more effective day to day care. Using real asthma patient wheezing sounds we develop a new combination of classifiers that is 96% accurate at automatically detecting wheezing. This abstract provides an overview of the design and implementation of AsthmaGuide and provides empirical evidence that AsthmaGuide is 3% - 11% more accurate in detecting wheezing sounds than standard techniques. Ho-Kyeong Ra, Asif Salekin, Hee-Jung Yoon, Jeremy Kim, Shahriar Nirjon, David J. Stone, Sujeong Kim, Jong-Myung Lee, Sang Hyuk Son, John A. Stankovic |
SenSys | 5 |
| 2014 | KinSpace: Passive Obstacle Detection via Kinect
Chris Greenwood, Shahriar Nirjon, John A. Stankovic, Hee-Jung Yoon, Ho-Kyeong Ra, Sang Hyuk Son, Taejoon Park |
EWSN | 2 |
| 2014 | RESONATE: reverberation environment simulation for improved classification of speech models
Robert F. Dickerson, Enamul Hoque 0002, Philip Asare, Shahriar Nirjon, John A. Stankovic |
IPSN | 4 |
| 2014 | COIN-GPS: indoor localization from direct GPS receivingabstractDue to poor signal strength, multipath effects, and limited on-device computation power, common GPS receivers do not work indoors. This work addresses these challenges by using a steerable, high-gain directional antenna as the front-end of a GPS receiver along with a robust signal processing step and a novel location estimation technique to achieve direct GPS-based indoor localization. By leveraging the computing power of the cloud, we accommodate longer signals for acquisition, and remove the requirement of decoding timestamps or ephemeris data from GPS signals. We have tested our system in 31 randomly chosen spots inside five single-story, indoor environments such as stores, warehouses and shopping centers. Our experiments show that the system is capable of obtaining location fixes from 20 of these spots with a median error of less than 10 m, where all normal GPS receivers fail. Shahriar Nirjon, Jie Liu 0001, Gerald DeJean, Bodhi Priyantha, Yuzhe Jin, Ted Hart |
MobiSys | 1 |
| 2014 | Kintense: A robust, accurate, real-time and evolving system for detecting aggressive actions from streaming 3D skeleton dataabstractKintense is a robust, accurate, real-time, and evolving system for detecting aggressive actions such as hitting, kicking, pushing, and throwing from streaming 3D skeleton joint coordinates obtained from Kinect sensors. Kintense uses a combination of: (1) an array of supervised learners to recognize a predefined set of aggressive actions, (2) an unsupervised learner to discover new aggressive actions or refine existing actions, and (3) human feedback to reduce false alarms and to label potential aggressive actions. This paper describes the design and implementation of Kintense and provides empirical evidence that the system is 11% - 16% more accurate and 10% - 54% more robust to changes in distance, body orientation, speed, and person when compared to standard techniques such as dynamic time warping (DTW) and posture based gesture recognizers. We deploy Kintense in two multi-person households and demonstrate how it evolves to discover and learn unseen actions, achieves up to 90% accuracy, runs in real-time, and reduces false alarms with up to 13 times fewer user interactions than a typical system. Shahriar Nirjon, Chris Greenwood, Stefanie Zhou, John A. Stankovic, Hee-Jung Yoon, Ho-Kyeong Ra, Can Basaran, Taejoon Park, Sang Hyuk Son |
PerCom | 1 |
| 2014 | MultiNets: A system for real-time switching between multiple network interfaces on mobile devicesabstractMultiNets is a system supporting seamless switch-over between wireless interfaces on mobile devices in real-time. MultiNets is configurable to run in three different modes: (i) Energy Saving mode --for choosing the interface that saves the most energy based on the condition of the device, (ii) Offload mode --for offloading data traffic from the cellular to WiFi network, and (iii) Performance mode --for selecting the network for the fastest data connectivity. MultiNets also provides a powerful API that gives the application developers: (i) the choice to select a network interface to communicate with a specific server, and (ii) the ability to simultaneously transfer data over multiple network interfaces. MultiNets is modular, easily integrable, lightweight, and applicable to various mobile operating systems. We implement MultiNets on Android devices as a show case. MultiNets does not require any extra support from the network infrastructure and runs existing applications transparently. To evaluate MultiNets, we first collect data traces from 13 actual Android smartphone users over three months. We then use the collected traces to show that, by automatically switching to WiFi whenever it is available, MultiNets can offload on average 79.82% of the data traffic. We also illustrate that, by optimally switching between the interfaces, MultiNets can save on average 21.14 KJ of energy per day, which is equivalent to 27.4% of the daily energy usage. Using our API, we demonstrate that a video streaming application achieves 43--271% higher streaming rate when concurrently using WiFi and 3G interfaces. We deploy MultiNets in a real-world scenario and our experimental results show that depending on the user requirements, it outperforms the state-of-the-art Android system either by saving up to 33.75% energy, achieving near-optimal offloading, or achieving near-optimal throughput while substantially reducing TCP interruptions due to switching. Shahriar Nirjon, Angela Nicoara, Cheng-Hsin Hsu, Jatinder Pal Singh, John A. Stankovic |
ACM Trans. Embed. Comput. Syst. | 1 |
| 2013 | Poster abstract: a mobile-cloud service for physiological anomaly detection on smartphonesabstractThere is a growing number of examples that use the microphones in phone for various acoustic processing tasks as mobile phones become increasingly computationally powerful. However, there is no general physiological acoustic anomaly detection service on smartphones. To this end, we propose a physiological acoustic anomaly detection service which contains classifiers that can be used to detect irregularity and anomalies in lung sounds and notifies the user. We also present and discuss on some preliminary results. Dezhi Hong, Shahriar Nirjon, John A. Stankovic, David J. Stone, Guobin Shen |
IPSN | 2 |
| 2013 | Auditeur: a mobile-cloud service platform for acoustic event detection on smartphonesabstractAuditeur is a general-purpose, energy-efficient, and context-aware acoustic event detection platform for smartphones. It enables app developers to have their app register for and get notified on a wide variety of acoustic events. Auditeur is backed by a cloud service to store user contributed sound clips and to generate an energy-efficient and context-aware classification plan for the phone. When an acoustic event type has been registered, the smartphone instantiates the necessary acoustic processing modules and wires them together to execute the plan. The phone then captures, processes, and classifies acoustic events locally and efficiently. Our analysis on user-contributed empirical data shows that Auditeur's energy-aware acoustic feature selection algorithm is capable of increasing the device lifetime by 33.4%, sacrificing less than 2% of the maximum achievable accuracy. We implement seven apps with Auditeur, and deploy them in real-world scenarios to demonstrate that Auditeur is versatile, 11.04% - 441.42% less power hungry, and 10.71% - 13.86% more accurate in detecting acoustic events, compared to state-of-the-art techniques. We present a user study to demonstrate that novice programmers can implement the core logic of interesting apps with Auditeur in less than 30 minutes, using only 15 - 20 lines of Java code. Shahriar Nirjon, Robert F. Dickerson, Philip Asare, Qiang Li 0025, Dezhi Hong, John A. Stankovic, Pan Hu 0003, Guobin Shen, Xiaofan Jiang 0001 |
MobiSys | 1 |
| 2013 | KinSpace: to provide fall prevention using KinectabstractFalls are a significant problem for the elderly living independently in the home. Many falls occur due to household objects left in open spaces. We present KinSpace, a system that uses real-time depth data and human-in-the-loop feedback to adjust its understanding of the open space of an environment. We present results showing the effectiveness of our underlying technical solutions in identifying open spaces and obstacles. The results for both lab testing and a small deployment in an actual home show over 80% accuracy for open space detection and 70% accuracy in obstacle detection even in the presence of many real world issues. Chris Greenwood, Shahriar Nirjon, John A. Stankovic, Hee-Jung Yoon, Ho-Kyeong Ra, Taejoon Park, Sang Hyuk Son |
SenSys | 2 |
| 2013 | Kintense: a robust, accurate, real-time and evolving system for detecting aggressive actions from streaming 3D skeleton dataabstractKintense is a robust, accurate, real-time, and evolving system for detecting aggressive actions such as hitting, kicking, pushing, and throwing from streaming 3D skeleton joint coordinates obtained from Kinect sensors. Kintense uses a combination of: (1) an array of supervised learners to recognize a predefined set of aggressive actions, (2) an unsupervised learner to discover new aggressive actions or refine existing actions, and (3) human feedback to reduce false alarms and to label potential aggressive actions. This abstract provides an overview of the design and implementation of Kintense and provides empirical evidence that Kintense is 11% -- 16% more accurate when compared to standard techniques such as dynamic time warping (DTW) and posture based gesture recognizers. Shahriar Nirjon, Chris Greenwood, Stefanie Zhou, John A. Stankovic, Hee-Jung Yoon, Ho-Kyeong Ra, Can Basaran, Taejoon Park, Sang Hyuk Son |
SenSys | 1 |
| 2013 | High-sensitivity cloud-offloaded instant GPS for indoor environmentsabstractDue to poor signal-strength, multi-path effects, and a lack of adequate visible satellite vehicles (SV), GPS receivers do not work indoors. This work addresses these challenges by using a mechanically steerable, high-gain directional antenna as the front-end of the GPS receiver along with a robust signal processing technique to acquire satellites in indoor environments. Our experiment on a local warehouse shows that, the system is capable of acquiring 5 or more SVs (which is a requirement for instant GPS technique [4]), whereas a Garmin device barely sees 3, and the system is capable of estimating indoor locations with 3--18 m errors when compared to the ground truth. Shahriar Nirjon, Jie Liu 0001, Bodhi Priyantha, Gerald DeJean |
SenSys | 1 |
| 2012 | Kinsight: Localizing and Tracking Household Objects Using Depth-Camera SensorsabstractWe solve the problem of localizing and tracking household objects using a depth-camera sensor network. We design and implement Kin sight that tracks household objects indirectly -- by tracking human figures, and detecting and recognizing objects from human-object interactions. We devise two novel algorithms: (1) Depth Sweep -- that uses depth information to efficiently extract objects from an image, and (2) Context Oriented Object Recognition -- that uses location history and activity context along with an RGB image to recognize object sat home. We thoroughly evaluate Kinsight's performance with a rich set of controlled experiments. We also deploy Kinsightin real-world scenarios and show that it achieves an average localization error of about 13 cm. Shahriar Nirjon, John A. Stankovic |
DCOSS | 1 |
| 2012 | SEPTIMU: continuous in-situ human wellness monitoring and feedback using sensors embedded in earphonesabstractA mobile phone, as a pervasive device, has great potential in human wellness monitoring. In this demo, we first present the design and implementation of our hardware - SEPTIMU. SEPTIMU consists of a small baseboard and a pair of tiny sensor boards embedded inside conventional earphones. The baseboard provides power conversion and data communication through the normal audio jack interface. The embedded sensor board is 1×1cm2 and integrates 3-axis accelerometer, gyroscope, thermometer, photodiode and microphone. Secondly, we evaluate SEPTIMU using a mobile application that continuously monitors body posture and provides feedback to the user. Dezhi Hong, Ben Zhang 0003, Qiang Li 0025, Shahriar Nirjon, Robert F. Dickerson, Guobin Shen, Xiaofan Jiang 0001, John A. Stankovic |
IPSN | 4 |
| 2012 | MultiNets: Policy Oriented Real-Time Switching of Wireless Interfaces on Mobile DevicesabstractIn this paper we present Multi Nets, a system which is capable of switching between wireless network interfaces on mobile devices in real-time. Multi Nets is motivated by the need of smart phone platforms to save energy, offload data traffic, and achieve higher throughput. We describe the architecture of Multi Nets and demonstrate the methodology to perform switching in Linux based mobile OSes such as Android. Our analysis on mobile data traces collected from real users shows that with real-time switching we can save 27.4% of the energy, offload 79.82% of the data traffic, or achieve 7 times more throughput on average. We deploy Multi Nets in a real world scenario and our experimental results show that depending on the user requirements, it outperforms the state-of-the-art Android system either by saving up to 33.75% energy, or achieving near-optimal offloading, or achieving near-optimal throughput while substantially reducing TCP interruptions due to switching. Shahriar Nirjon, Angela Nicoara, Cheng-Hsin Hsu, Jatinder Pal Singh, John A. Stankovic |
IEEE Real-Time and Embedded Technology and Applications Symposium | 1 |
| 2012 | Septimu2 - earphones for continuous and non-intrusive physiological and environmental monitoringabstractMobile phones have become an ideal platform for physiological and environmental sensing. A number of research and commercial smartphone "accessories" have emerged in recent years that try to extend the sensing capabilities of a mobile phone. However, the major drawback of these devices is that they either require the user to act in some specific way or change their lifestyle and habit to some extent. In this demo, we present Septimu V2 (Septimu2) -- a novel non-intrusive physiological and environmental sensing platform which is fully embedded in a conventional earphone, works with existing smartphones, and does not require the user to change habits in any way. Septimu2 is a continuation of [1], and integrates a suite of new sensors. In addition to 3-axis accelerometer and gyroscope, Septimu2 incorporates remote IR temperature sensor, IR LED, IR photodiode and two additional microphones. The baseboard performs signal condition and sends the data to cellphone via Bluetooth. Septimu2 enables a number of applications, including heart-rate monitoring, fine grained posture detection, and external sound source localization and classification. Pan Hu 0003, Guobin Shen, Xiaofan Jiang 0001, Shao-Fu Shih, Donghuan Lu, Feng Zhao 0001, Dezhi Hong, Qiang Li 0025, Shahriar Nirjon, Robert F. Dickerson, John A. Stankovic |
SenSys | 9 |
| 2012 | MusicalHeart: a hearty way of listening to musicabstractMusicalHeart is a biofeedback-based, context-aware, automated music recommendation system for smartphones. We introduce a new wearable sensing platform, Septimu, which consists of a pair of sensor-equipped earphones that communicate to the smartphone via the audio jack. The Septimu platform enables the MusicalHeart application to continuously monitor the heart rate and activity level of the user while listening to music. The physiological information and contextual information are then sent to a remote server, which provides dynamic music suggestions to help the user maintain a target heart rate. We provide empirical evidence that the measured heart rate is 75% -- 85% correlated to the ground truth with an average error of 7.5 BPM. The accuracy of the person-specific, 3-class activity level detector is on average 96.8%, where these activity levels are separated based on their differing impacts on heart rate. We demonstrate the practicality of MusicalHeart by deploying it in two real world scenarios and show that MusicalHeart helps the user achieve a desired heart rate intensity with an average error of less than 12.2%, and its quality of recommendation improves over time. Shahriar Nirjon, Robert F. Dickerson, Qiang Li 0025, Philip Asare, John A. Stankovic, Dezhi Hong, Ben Zhang 0003, Xiaofan Jiang 0001, Guobin Shen, Feng Zhao 0001 |
SenSys | 1 |
| 2010 | Addressing burstiness for reliable communication and latency bound generation in wireless sensor networksabstractAs wireless sensor networks mature, they are increasingly being used in real-time applications. Many of these applications require reliable transmission within latency bounds. Achieving this goal is very difficult because of link burstiness and interference. Based on significant empirical evidence of 21 days and over 3,600,000 packets transmission per link, we propose a scheduling algorithm that produces latency bounds of the real-time periodic streams and accounts for both link bursts and interference. The solution is achieved through the definition of a new metric Bmax that characterizes links by their maximum burst length, and by choosing a novel least-burst-route that minimizes the sum of worst case burst lengths over all links in the route. A testbed evaluation consisting of 48 nodes spread across a floor of a building shows that we obtain 100% reliable packet delivery within derived latency bounds. We also demonstrate how performance deteriorates and discuss its implications for wireless networks with insufficient high quality links. Sirajum Munir, Shan Lin 0001, Enamul Hoque 0002, Shahriar Nirjon, John A. Stankovic, Kamin Whitehouse |
IPSN | 4 |
| 2009 | Heuristics for scheduling periodic real-time streams in wireless sensor networksabstractSimultaneous transmissions in the same radio range of a wireless sensor network causes interference and packets are lost. Knowing the interference pattern in advance, the transmission links can be scheduled so that no packet is lost due to interference and all streams meet their deadlines. This problem is NP-hard in general and therefore we resort to heuristics. In this paper, we present a set of heuristics for scheduling periodic, real-time data streams over a wireless sensor network. Simulation results show that our heuristics produce feasible schedules in almost all cases. Shahriar Nirjon, John A. Stankovic, Kamin Whitehouse |
SenSys | 1 |
| 2008 | Bagging and Boosting Negatively Correlated Neural NetworksabstractIn this paper, we propose two cooperative ensemble learning algorithms, i.e., NegBagg and NegBoost, for designing neural network (NN) ensembles. The proposed algorithms incrementally train different individual NNs in an ensemble using the negative correlation learning algorithm. Bagging and boosting algorithms are used in NegBagg and NegBoost, respectively, to create different training sets for different NNs in the ensemble. The idea behind using negative correlation learning in conjunction with the bagging/boosting algorithm is to facilitate interaction and cooperation among NNs during their training. Both NegBagg and NegBoost use a constructive approach to automatically determine the number of hidden neurons for NNs. NegBoost also uses the constructive approach to automatically determine the number of NNs for the ensemble. The two algorithms have been tested on a number of benchmark problems in machine learning and NNs, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, satellite, soybean, and waveform problems. The experimental results show that NegBagg and NegBoost require a small number of training epochs to produce compact NN ensembles with good generalization. Xin Yao 0001, Shahriar Nirjon, Muhammad Asiful Islam, Kazuyuki Murase |
IEEE Trans. Syst. Man Cybern. Part B | 3 |