Weight difference estimation is crucial in various applications, particularly for identifying items being picked up and put back when people interact with the shelf while shopping in autonomous stores, ensuring precise cost estimation. However, the conventional approach of estimating weight changes requires specialized weight-sensing shelves, which are densely deployed weight scales, incurring intensive sensor consumption and maintenance costs. Prior works explored the vibration-based weight sensing method, but they are limited to the object that can generate vibration through motion. This work demonstrates a system leveraging active vibration sensing for weight difference estimation on shelves at different locations. The main intuition of the system is that the weight placed on the shelf influences the dynamic vibration response of the shelf, thus altering the shelf vibration patterns. Our system achieves a mean absolute error 9.23 grams and mean absolute percentage error 7.9% on the real-store shelf layout.

Object classification is a vital technology that is widely used to track and identify misplaced and out-of-stock items in shopping centers. While there have been a number of studies utilizing various sensing modalities such as computer vision, RFID, and vibration sensors, these methods are limited in their use due to privacy concerns, scalability, and the inability to identify stationary objects. To overcome these limitations, we propose a novel active vibration-sensing approach for object classification by utilizing music as an excitation source. Different objects can induce different deformations of the surface and further change the surface structural response. Therefore, we leverage vibrations from music on a store shelf and measure the structural responses on the surface when different objects are placed. Our evaluation of a store shelf demonstrates that distinct object characteristics lead to unique vibration responses, enabling accurate classification of 98.6% accuracy in distinguishing five common store objects. This study provides a promising avenue for a reliable, privacy-preserving, and scalable object classification system in various settings beyond shopping centers.

IoT (Internet of Things) devices, such as network-enabled wearables, are carried by increasingly more people throughout daily life. Information from multiple devices can be aggregated to gain insights into a person’s behavior or status. For example, an elderly care facility could monitor patients for falls by combining fitness bracelet data with video of the entire class. For this aggregated data to be useful to each person, we need a multi-modality association of the devices’ physical ID (i.e., location, the user holding it, visual appearance) with a virtual ID (e.g., IP address/available services). Existing approaches for multi-modality association often require intentional interaction or direct line-of-sight to the device, which is infeasible for a large number of users or when the device is obscured by clothing. We present IDIoT , a calibration-free passive sensing approach that fuses motion sensor information with camera footage of an area to estimate the body location of motion sensors carried by a user. We characterize results across three baselines to highlight how different fusing methodology results better than earlier IMU-vision fusion algorithms. From this characterization, we determine IDIoT is more robust to errors such as missing frames or miscalibration that frequently occur in IMU-vision matching systems.

Recent work has shown that two wearable devices worn on the same user can exploit gait as a secret source to generate a common key for secure pairing. The main threat of using gait comes from side-channel attackers who can use cameras to record the walking user and extract accelerations from the video to pair with legitimate devices. We propose a novel pre-step that uses a CNN-LSTM deep learning model to classify the acceleration measurement devices, i.e., between IMU vs. Camera. We prototype the pre-step and evaluate it using real subjects. Our results show that the proposed pre-step can achieve high classification success rates. The experiments with different cut-off frequencies show that the higher acceleration frequencies appear to contain more distinguishable features to classify camera from IMU.

Large-scale micro-aerial vehicle (MAV) swarms provide promising solutions for situational awareness in applications such as environmental monitoring, urban surveillance, search and rescue, and so on. However, these scenarios do not provide localization infrastructure and limit cost and size of on-board capabilities of individual nodes, which makes it challenging for nodes to autonomously navigate to suitable preassigned locations. In this article, we present H-DrunkWalk , a collaborative and adaptive technique for heterogeneous MAV swarm navigation in environments not formerly preconditioned for operation. Working with heterogeneous MAV swarm, the H-DrunkWalk achieves high accuracy through collaboration but still maintains a low cost of the entire swarm. The heterogeneous MAV swarm consists of two types of nodes: (1) basic MAVs with limited sensing, communication, computing capabilities and (2) advanced MAVs with premium sensing, communication, computing capabilities. The key focus behind this networked MAV swarm research is to (1) rely on collaboration to overcome limitations of individual nodes and efficiently achieve system-wide sensing objectives and (2) fully take advantage of advanced MAVs to help basic MAVs improve their performance. The evaluations based on real MAV testbed experiments and large-scale physical-feature-based simulations show that compared to the traditional non-collaborative and non-adaptive method (dead reckoning with map bias), our system achieves up to 6× reductions in location estimation errors, and as much as 3× improvements in navigation success rate under the given time and accuracy constraints. In addition, by comprehensively considering the environment, heterogeneous structure, and quality of location estimation, our H-DrunkWalk brings 2× performance improvement (on average) as that of a hardware upgrade.

Secure pairing is an important problem especially due to large number of IoT devices. In this paper, we propose PosePair++, to enable a camera to securely pair with IoT devices which are equipped with IMU sensors. Existing context-based pairing approaches do not adequately address this problem due to differing sensing modalities. To address this challenge, we propose to translate the signals from heterogeneous sensing modalities to a common space, namely 2D acceleration. In this demo, we present PosePair++'s robustness against different types of attackers (i.e., attackers that observe the user's motion, or attackers performing mimicking attack).

In the Internet of Things (IoT) paradigm, it is important to easily setup and control devices, which is achieved by pairing. In this work, we present a novel pairing scheme that utilizes heterogeneous sensing. The core idea is that devices with different sensing capabilities can still detect common information about their user. We demonstrate this idea through an example application consisting of a camera and IoT devices with inertial sensors. As the user holds a device and moves it around, the camera captures the human's pose and compares it to the IoT device motion. If the motion features are similar enough, the device can be successfully paired to the camera's network.

Unmanned aerial vehicle (UAV) swarms provide situation awareness in tasks such as emergency response, search and rescue, etc. However, most of these scenarios take place in GPS-denied environments, where accurately localizing each UAV is challenging. Heterogeneous UAV swarms, in which only a subset of the drones carry cameras, face the additional challenge of identifying each individual UAV to avoid sending position updates to the wrong drone, thus crashing. This work presents an identification mechanism based on the correlation between motion observed from external camera, and acceleration measured on each UAV's accelerometer.

Indoor emergency response situations, such as urban fire, are characterized by dangerous constantly changing operating environments with little access to situational information for first responders. In situ information about the conditions, such as the extent and evolution of an indoor fire, can augment rescue efforts and reduce risk to emergency personnel. Static sensor networks that are pre-deployed or manually deployed have been proposed but are less practical due to need for large infrastructure, lack of adaptivity, and limited coverage. Controlled-mobility in sensor networks, that is, the capability of nodes to move as per network needs can provide the desired autonomy to overcome these limitations. In this article, we present SensorFly, a controlled-mobile aerial sensor network platform for indoor emergency response application. The miniature, low-cost sensor platform has capabilities to self deploy, achieve three-dimensional sensing, and adapt to node and network disruptions in harsh environments. We describe hardware design trade-offs, the software architecture, and the implementation that enables limited-capability nodes to collectively achieve application goals. Through the indoor fire monitoring application scenario, we validate that the platform can achieve coverage and sensing accuracy that matches or exceeds static sensor networks and provide higher adaptability and autonomy.

Resilient localization and navigation for autonomous Unmanned Aerial Vehicles (UAVs) still remains a challenge in certain scenarios, like GPS-deprived environments such as indoors or urban canyons. In this work, we explore a heterogeneous UAV swarm design, in which a small number of sensor and computationally powerful UAVs collaborate with the remaining resource-constrained UAVs to guarantee optimal localization accuracy.

Micro-aerial vehicle (MAV) swarms are a new class of mobile sensor networks with many applications, including search and rescue, urban surveillance, radiation monitoring, etc. These sensing applications require autonomously navigating a high number of low-cost, low-complexity MAV sensor nodes in hazardous environments. The lack of preexisting localization infrastructure and the limited sensing, computing, and communication abilities of individual nodes makes it challenging for nodes to autonomously navigate to suitable preassigned locations. In this paper, we present a collaborative and adaptive algorithm for resource-constrained MAV nodes to quickly and efficiently navigate to preassigned locations. Using radio fingerprints between flying and landed MAVs acting as radio beacons, the algorithm detects intersections in trajectories of mobile nodes. The algorithm combines noisy dead-reckoning measurements from multiple MAVs at detected intersections to improve the accuracy of the MAVs' location estimations. In addition, the algorithm plans intersecting trajectories of MAV nodes to aid the location estimation and provide desired performance in terms of timeliness and accuracy of navigation. We evaluate the performance of our algorithm through a real testbed implementation and large-scale physical feature based simulations. Our results show that, compared to existing autonomous navigation strategies, our algorithm achieves up to 6X reduction in location estimation errors, and as much as 3X improvement in navigation success rate under the given time and accuracy constraints.