Crowd monitoring in sports stadiums is important to enhance public safety and improve audience experience. Existing approaches mainly rely on cameras and microphones, which can cause significant disturbances and often raise privacy concerns. In this paper, we sense floor vibration, which provides a less disruptive and more non-intrusive way of crowd sensing, to predict crowd behavior. However, since the vibration-based crowd monitoring approach is newly developed, one main challenge is the lack of training data due to sports stadiums are usually large public spaces with complex physical activities.

Monitoring vital signs and activities in pig pens during the farrowing period is crucial for reducing pre-weaning piglet mortality and enhancing farm productivity. Traditional methods focus on either vital signs or activities separately, falling short in the dynamically changing farm environment. This paper introduces a multiscale vibration sensing method which dynamically adjusts sensor amplifier gain to detect both vital signs (e.g. heartbeats and respirations) and larger-scale activities (e.g. walking, eating, nursing, etc.). Preliminary trials demonstrate the system's potential to adapt to rapidly changing conditions by switching between high sensitivity for vital signs and reduced sensitivity for activity sensing depending on the detected vibration signal.

Assessing urban infrastructure and environment quality at scale remains a challenge. This work presents a multimodal sensing framework that integrates computer vision-based infrastructure analysis with mobile air quality monitoring to explore urban conditions beyond the camera's field of view. A vehicle-mounted system captures video data for semantic segmentation of roads and buildings while an air intake unit collects temperature, humidity, CO2, TVOC, and AQI levels. A preliminary test drive in Ann Arbor demonstrated expected correlations between CO2 and TVOC spikes in dense urban areas, providing a proof of concept for linking environmental sensing with visual urban analysis. Future work will refine sensor calibration, adaptive sampling strategies, and predictive modeling to improve accuracy and scalability.

Weight difference estimation is crucial in various applications, particularly for identifying items being picked up and put back when people interact with the shelf while shopping in autonomous stores, ensuring precise cost estimation. However, the conventional approach of estimating weight changes requires specialized weight-sensing shelves, which are densely deployed weight scales, incurring intensive sensor consumption and maintenance costs. Prior works explored the vibration-based weight sensing method, but they are limited to the object that can generate vibration through motion. This work demonstrates a system leveraging active vibration sensing for weight difference estimation on shelves at different locations. The main intuition of the system is that the weight placed on the shelf influences the dynamic vibration response of the shelf, thus altering the shelf vibration patterns. Our system achieves a mean absolute error 9.23 grams and mean absolute percentage error 7.9% on the real-store shelf layout.

This paper addresses challenges in monitoring human behavior in crowds through floor vibration sensing, overcoming limitations like subjective manual observation, visual occlusions, and audio interference. Our approach involves tackling limited-data vibration signal tasks by conducting pre-training across modalities, leveraging publicly available audio datasets. By leveraging self-supervised representation learning to pre-train on publicly available audio datasets, our approach reduces data requirements, improves robustness, and minimizes the need for human labeling efforts. Evaluation using in-game stadium vibration data with YouTube audio dataset demonstrates up to 5.8 × error reduction for crowd behavior.

Computer vision has been used more ubiquitously in recent years to understand and measure the environment around us, particularly in our neighborhoods. However, many city-wide sensing applications using vision require large labeling efforts, making various applications difficult on a wide scale. We propose a framework for labeling and self-training of in-car video to detect trash on the roads. Our approach requires minimal manual labeling to identify items not meant to be in the street, sidewalk, or public places, from a front-viewing car camera. Our system provides each frame of a video with a score indicating the amount of trash. To prevent overfitting, due to minimal available data, we remove data with high certainty of trash from the training dataset. The results show that our prediction with manually labeled ground truth yield an R2 of 0.66.

Precision Swine Farming has the potential to directly benefit swine health and industry profit by automatically monitoring the growth and health of pigs. We introduce the first system to use structural vibration to track animals and the first system for automated characterization of piglet group activities, including nursing, sleeping, and active times. PigSense uses physical knowledge of the structural vibration characteristics caused by pig-activity-induced load changes to recognize different behaviors of the sow and piglets. For our system to survive the harsh environment of the farrowing pen for three months, we designed simple, durable sensors for physical fault tolerance, then installed many of them, pooling their data to achieve algorithmic fault tolerance even when some do stop working. The key focus of this work was to create a robust system that can withstand challenging environments, has limited installation and maintenance requirements, and uses domain knowledge to precisely detect a variety of swine activities in noisy conditions while remaining flexible enough to adapt to future activities and applications. We provided an extensive analysis and evaluation of all-round swine activities and scenarios from our one-year field deployment across two pig farms in Thailand and the USA. To help assess the risk of crushing, farrowing sicknesses, and poor maternal behaviors, PigSense achieves an average of 97.8% and 94% for sow posture and motion monitoring, respectively, and an average of 96% and 71% for ingestion and excretion detection. To help farmers monitor piglet feeding, starvation, and illness, PigSense achieves an average of 87.7%, 89.4%, and 81.9% in predicting different levels of nursing, sleeping, and being active, respectively. In addition, we show that our monitoring of signal energy changes allows the prediction of farrowing in advance, as well as status tracking during the farrowing process and on the occasion of farrowing issues. Furthermore, PigSense also predicts the daily pattern and weight gain in the lactation cycle with 89% accuracy, a metric that can be used to monitor the piglets’ growth progress over the lactation cycle.

Heart rate is a critical metric for human cardiovascular health. Most common methods for measuring human heart rate involve wearable devices (e.g., electrocardiography, smart watches). However, such devices can cause discomfort to some patients, especially the elderly or young children. This paper presents FreePulse, a heart rate monitoring system for seated subjects using ambient vibrations. FreePulse builds on our past work using vibrations in the building structures around us to measure human activities and health. As people’s hearts beat, they push on the surfaces the body is touching, creating vibrations in those structures. We combine structure response characterization with human pulse modelling to identify these pulse-induced vibrations from ambient vibration. In testing, FreePulse has shown up to 96% pulse rate accuracy on average, competitive with consumer-grade wearable devices.

Object classification is a vital technology that is widely used to track and identify misplaced and out-of-stock items in shopping centers. While there have been a number of studies utilizing various sensing modalities such as computer vision, RFID, and vibration sensors, these methods are limited in their use due to privacy concerns, scalability, and the inability to identify stationary objects. To overcome these limitations, we propose a novel active vibration-sensing approach for object classification by utilizing music as an excitation source. Different objects can induce different deformations of the surface and further change the surface structural response. Therefore, we leverage vibrations from music on a store shelf and measure the structural responses on the surface when different objects are placed. Our evaluation of a store shelf demonstrates that distinct object characteristics lead to unique vibration responses, enabling accurate classification of 98.6% accuracy in distinguishing five common store objects. This study provides a promising avenue for a reliable, privacy-preserving, and scalable object classification system in various settings beyond shopping centers.

Monitoring cardiovascular risk factors requires a method for con-tinuous heart rate monitoring. However, this typically relies on a wearable device which must be issued to, then charged and worn by the user. Ambient sensors can address these challenges, but have only been usable in scenarios where the body is in proximity or contact with these sensors, such as on a bed. In this paper, we present SeatBeats, a system that can monitor heart rate using ambient vibrations in the surface a user sits on. The system relies on the surface's response to heart beat-induced vibrations and utilizes autocorrelation to capture and extract the repeated pattern beats. Our experiments in a real office environment show up to 96% accuracy relative to a smartwatch heart rate sensor.

Pig vital sign monitoring (e.g., estimating the heart rate (HR) and respiratory rate (RR)) is essential to understand the stress level of the sow and detect the onset of parturition. It helps to maximize peri-natal survival and improve animal well-being in swine production. The existing approach mainly relies on manual measurement, which is labor-intensive and only provides a few points of information. Other sensing modalities such as wearables and cameras are developed to enable more continuous measurement, but are still limited due to animal discomfort, data transfer, and storage challenges. In this paper, we introduce PigV2, the first system to monitor pig heart rate and respiratory rate through ground vibrations. Our approach leverages the insight that both heartbeat and respiration generate ground vibrations when the sow is lying on the floor. We infer vital information by sensing and analyzing these vibrations. The main challenge in developing PigV2 is the overlap of vital- and non-vital-related information in the vibration signals, including pig movements, pig postures, pig-to-sensor distances, and so on. To address this issue, we first characterize their effects, extract their current status, and then reduce their impact by adaptively interpolating vital rates over multiple sensors. PigV2 is evaluated through a real-world deployment with 30 pigs. It has 3.4% and 8.3% average errors in monitoring the HR and RR of the sows, respectively.

Automated monitoring of livestock behavior can help farmers economically by detecting changes in animal welfare. Prior approaches use video, which requires light and high storage capability, or motion detection, which has difficulty separating subtle activities. Wearable sensors can address these issues but are vulnerable to destruction by the animals. To the best of our knowledge, we present the first system that uses structural vibration to track animal behavior, and the first system to automatically detect piglet nursing. PigNet uses vibration sensors attached to a pig pen to sense the unique vibration patterns and changes in structural response caused by the animals' movement and position within the pen. Combined with our knowledge of pig behavior, we use this physical knowledge of vibration characteristics to detect pig activities and track piglet growth in a real farm environment. Our system is designed to be robust to the harsh environment, which can create unpredictable noise, as well as physically damage or disconnect sensor nodes. When deployed in a real-world farm environment, our system was able to achieve a daily pen-level status profile of up to 90% accuracy, which tracks nursing activity, sow lying activity, and changes in piglet growth over the weeks-long pre-weaning period.

This paper presents the Active Structural Occupant Detector, an active vibration sensing system that detects stationary occupants through injection of vibration signals into the floor. Many smart buildings require occupant detection in order to provide personalized services. Some examples include optimized energy usage and security. Several methods currently exist for occupant detection, each with their own drawbacks, such as installation requirements. Structural vibration sensing overcomes many of these drawbacks by measuring impulses created by occupants to infer their movements, but cannot detect stationary occupants. The ASOD utilizes active vibration sources, which inject acoustic waves into the structure then measure how the structure responds to them. Any occupants present interact with these waves, causing changes to the measured signal. By characterizing the changes in how the waves travel, we can predict the presence or lack of an occupant with up to a 97.7% accuracy, as demonstrated by experiments in a real- world environment.