Imagine placing your smartphone on a table in a noisy restaurant and clearly capturing the voices of friends seated around you, or recording a lecturer’s voice with clarity in a reverberant auditorium. We introduce SonicSieve, the first intelligent directional speech extraction system for smartphones using a bio-inspired acoustic microstructure. Our passive design embeds directional cues onto incoming speech without any additional electronics. It attaches to the in-line mic of low-cost wired earphones which can be attached to smartphones. We present an end-to-end neural network that processes the raw audio mixtures in real-time on mobile devices. Our results show that SonicSieve achieves a signal quality improvement of 5.0 dB when focusing on a 30° angular region. Additionally, the performance of our system based on only two microphones exceeds that of conventional 5-microphone arrays.

We present LubDubDecoder, a system that enables fine-grained monitoring of micro-cardiac vibrations associated with the opening and closing of heart valves across a range of hearables. Our system transforms the built-in speaker, the only transducer common to all hearables, into an acoustic sensor that captures the coarse “lub-dub” heart sounds, leverages their shared temporal and spectral structure to reconstruct the subtle seismocardiography (SCG) and gyrocardiography (GCG) waveforms, and extract the timing of key micro-cardiac events. In an IRB-approved feasibility study with 25 users, our system achieves correlations of 0.88–0.95 compared to chest-mounted reference measurements in within-user and cross-user evaluations, and generalizes to unseen hearables using a zero-effort adaptation scheme with a correlation of 0.91. Our system is robust across remounting sessions and music playback.

In this paper, our goal is to enable quantitative feedback on muscle fatigue during exercise to optimize exercise effectiveness while minimizing injury risk. We seek to capture fatigue by monitoring surface vibrations that muscle exertion induces. Muscle vibrations are unique as they arise from the asynchronous firing of motor units, producing surface micro-displacements that are broadband, nonlinear, and seemingly stochastic. Accurately sensing these noise-like signals requires new algorithmic strategies that can uncover their underlying structure. We present GigaFlex the first contactless system that measures muscle vibrations using mmWave radar to infer muscle force and detect fatigue. GigaFlex draws on algorithmic foundations from Chaos theory to model the deterministic patterns of muscle vibrations and extend them to the radar domain. Specifically, we design a radar processing architecture that systematically infuses principles from Chaos theory and nonlinear dynamics throughout the sensing pipeline, spanning localization, segmentation, and learning, to estimate muscle forces during static and dynamic weight-bearing exercises. Across a 23-participant study, GigaFlex estimates maximum voluntary isometric contraction (MVIC) root mean square error (RMSE) of 5.9\%, and detects one to three Repetitions in Reserve (RIR), a key quantitative muscle fatigue metric, with an AUC of 0.83 to 0.86, performing comparably to a contact-based IMU baseline. Our system can enable timely feedback that can help prevent fatigue-induced injury, and opens new opportunities for physiological sensing of complex, non-periodic biosignals.

Professional networking at in-person events plays a crucial role in career growth. However, many professionals struggle to maintain and transition these connections to online platforms like LinkedIn. Missed opportunities often arise due to challenges in tracking interactions, forgetting to add contacts, and the absence of tools that facilitate seamless networking. This paper examines how professionals network at events and the factors influencing their connection decisions. Through a comprehensive survey, we analyze networking behaviors, the use of social platforms, and the challenges faced in transitioning from offline to online transitions. Based on that analysis, we propose a mobile application that leverages Bluetooth and activity sensors to record social proximity, detect meaningful interactions, and provide intelligent recommendations for adding relevant connections. By bridging the gap between offline encounters and online professional networks, our approach enhances networking efficiency, ensuring that valuable connections made at events are sustained beyond the physical space.

We present a low-cost, soft tactile sensor using common, easily sourced materials that can be integrated with existing robotic gripper systems without requiring complex fabrication techniques or expensive components. Our approach includes two designs: a flexible linear sensor constructed from a rubber tube and a planar sensor made with a rubber membrane stretched over an enclosure. Both sensors contain an embedded speaker and microphones that leverage active acoustic sensing to map the unique acoustic resonant response of the cavity’s structure to deformations that occur when the robotic gripper is grasping an object. Experimental results demonstrate that, using a support vector machine, the linear sensor achieves contact point estimation with an RMSE of 6 mm, while the planar sensor achieves an RMSE of 0.57–0.62 mm. Additionally, the planar sensor classifies six objects with an accuracy of 97.7%. These results demonstrate the potential for active acoustics to be an accessible method for enabling tactile sensing capabilities for robotic systems.

Professional networking at in-person events plays a crucial role in career growth. However, many professionals struggle to maintain and transition these connections to online platforms like LinkedIn. Missed opportunities often arise due to challenges in tracking interactions, forgetting to add contacts, and the absence of tools that facilitate seamless networking. This paper examines how we can track those missed opportunities to connect with people from events by developing a protocol to find and record proximity interactions using Bluetooth while maintaining security and privacy of individual and device data, and then implementing that protocol in a proximity networking application developed for mobile phones. By bridging the gap between offline encounters and online professional networks, we believe that our approach enhances networking efficiency, ensuring that valuable connections made at events are sustained beyond the physical space.

AI-augmented data processing systems (DPSs) integrate large language models (LLMs) into query pipelines, allowing powerful semantic operations on structured and unstructured data. However, the reliability (a.k.a. trust) of these systems is fundamentally challenged by the potential for LLMs to produce errors, limiting their adoption in critical domains. To help address this reliability bottleneck, we introduce semantic integrity constraints (SICs) —a declarative abstraction for specifying and enforcing correctness conditions over LLM outputs in semantic queries. SICs generalize traditional database integrity constraints to semantic settings, supporting common types of constraints, such as grounding, soundness, and exclusion, with both reactive and proactive enforcement strategies. We argue that SICs provide a foundation for building reliable and auditable AI-augmented data systems. Specifically, we present a system design for integrating SICs into query planning and runtime execution and discuss its realization in AI-augmented DPSs. To guide and evaluate our vision, we outline several design goals—covering criteria around expressiveness, runtime semantics, integration, performance, and enterprise-scale applicability—and discuss how our framework addresses each, along with open research challenges.

We present the first neural network model to achieve real-time and streaming target sound extraction. To accomplish this, we propose Waveformer, an encoder-decoder architecture with a stack of dilated causal convolution layers as the encoder, and a transformer decoder layer as the decoder. This hybrid architecture uses dilated causal convolutions for processing large receptive fields in a computationally efficient manner, while also leveraging the generalization performance of transformer-based architectures. Our evaluations show as much as 2.2–3.3 dB improvement in SI-SNRi compared to the prior models for this task while having a 1.2–4x smaller model size and a 1.5–2x lower runtime. We provide code, dataset, and audio samples: https://waveformer.cs.washington.edu/.

We present the first wireless earbud hardware that can perform hearing screening by detecting otoacoustic emissions. The conventional wisdom has been that detecting otoacoustic emissions, which are the faint sounds generated by the cochlea, requires sensitive and expensive acoustic hardware. Thus, medical devices for hearing screening cost thousands of dollars and are inaccessible in low and middle income countries. We show that by designing wireless ear-buds using low-cost acoustic hardware and combining them with wireless sensing algorithms, we can reliably identify otoacoustic emissions and perform hearing screening. Our algorithms combine frequency modulated chirps with wideband pulses emitted from a low-cost speaker to reliably separate otoacoustic emissions from in-ear reflections and echoes. We conducted a clinical study with 50 ears across two healthcare sites. Our study shows that the low-cost earbuds detect hearing loss with 100% sensitivity and 89.7% specificity, which is comparable to the performance of a $8000 medical device. By developing low-cost and open-source wearable technology, our work may help address global health inequities in hearing screening by democratizing these medical devices.

The emergence of water-proof mobile and wearable devices (e.g., Garmin Descent and Apple Watch Ultra) designed for underwater activities like professional scuba diving, opens up opportunities for underwater networking and localization capabilities on these devices. Here, we present the first underwater acoustic positioning system for smart devices. Unlike conventional systems that use floating buoys as anchors at known locations, we design a system where a dive leader can compute the relative positions of all other divers, without any external infrastructure. Our intuition is that in a well-connected network of devices, if we compute the pairwise distances, we can determine the shape of the network topology. By incorporating orientation information about a single diver who is in the visual range of the leader device, we can then estimate the positions of all the remaining divers, even if they are not within sight. We address various practical problems including detecting erroneous distance estimates, addressing rotational and flipping ambiguities as well as designing a distributed timestamp protocol that scales linearly with the number of devices. Our evaluations show that our distributed system running on underwater deployments of 4--5 commodity smart devices can perform pairwise ranging and localization with median errors of 0.5--0.9 m and 0.9--1.6 m. Project page with code: https://underwatergps.cs.washington.edu/

Imagine being able to listen to the birds chirping in a park without hearing the chatter from other hikers, or being able to block out traffic noise on a busy street while still being able to hear emergency sirens and car honks. We introduce semantic hearing, a novel capability for hearable devices that enables them to, in real-time, focus on, or ignore, specific sounds from real-world environments, while also preserving the spatial cues. To achieve this, we make two technical contributions: 1) we present the first neural network that can achieve binaural target sound extraction in the presence of interfering sounds and background noise, and 2) we design a training methodology that allows our system to generalize to real-world use. Results show that our system can operate with 20 sound classes and that our transformer-based network has a runtime of 6.56 ms on a connected smartphone. In-the-wild evaluation with participants in previously unseen indoor and outdoor scenarios shows that our proof-of-concept system can extract the target sounds and generalize to preserve the spatial cues in its binaural output. Project page with code: https://semantichearing.cs.washington.edu

In this MobiSys demo, we present the demo of our SIGCOMM 2022 paper on underwater messaging system for existing mobile devices like smartphones and smart watches. Our software-only solution leverages audio sensors, i.e., microphones and speakers, ubiquitous in today's devices to enable acoustic underwater communication between mobile devices. To achieve this, we design a communication system that in real-time adapts to differences in frequency responses across mobile devices, changes in multipath and noise levels at different locations and dynamic channel changes due to mobility. Our demo will allow MobiSys attendees to test our system in realtime in a water tank using several demo smart devices in a waterproof pouch. We will also distribute the Android executable of the system via a QR code to attendees who wish to install and try the system on their own smart devices.

In this MobiSys demo we show a low-cost earphone based system that can screen for hearing loss with a cost of $10. Our system is designed to detect otoacoustic emissions (OAE) which are sounds generated when the outer hair cells move in a healthy cochlea and provide information about their function. OAE testing is commonly used as part of universal infant hearing screening protocols in high-income countries [1]. OAE equipment however is expensive hindering early hearing screening in developing countries that bear the disproportionate brunt of disabling hearing loss. Our design sends two pure tones through each of the headphone's ear-buds and records the distortion-product OAEs generated by the cochlea using a microphone. By running algorithms on a smartphone connected to the earphones using its headphone jack, we can detect distortion-product OAEs. Our device has been validated in a clinical study on 201 pediatric ears at oto-laryngology, hearing, and craniofacial clinics, across three different sites and achieved accuracies comparable to a commercial OAE device. In our demo, users will be encouraged to perform this quick test by listening to some tones in their ear through our device to check for the presence of OAEs in their ears.

In this MobiSys demo we present a system to determine fluid properties using the LiDAR sensors present on modern smartphones, as presented in our ACM IMWUT 2022 paper [1]. Traditional methods of measuring properties like viscosity require expensive laboratory equipment or a relatively large amount of fluid. In contrast, our smartphone-based method is accessible, contactless and works with just a single drop of liquid. Our design works by targeting a coherent LiDAR beam from the phone onto the liquid. Using the phone's camera, we capture the characteristic laser speckle pattern that is formed by the interference of light reflecting from light-scattering particles. By correlating the fluctuations in speckle intensity over time, we can characterize the Brownian motion within the liquid which is correlated with its viscosity. Our demo will allow MobiSys attendees to distinguish between liquids of different viscosities, milks of different fat contents, and adulterated milk.

Since its inception, underwater digital acoustic communication has required custom hardware that neither has the economies of scale nor is pervasive. We present the first acoustic system that brings underwater messaging capabilities to existing mobile devices like smartphones and smart watches. Our software-only solution leverages audio sensors, i.e., microphones and speakers, ubiquitous in today's devices to enable acoustic underwater communication between mobile devices. To achieve this, we design a communication system that in real-time adapts to differences in frequency responses across mobile devices, changes in multipath and noise levels at different locations and dynamic channel changes due to mobility. We evaluate our system in six different real-world underwater environments with depths of 2--15 m in the presence of boats, ships and people fishing and kayaking. Our results show that our system can in real-time adapt its frequency band and achieve bit rates of 100 bps to 1.8 kbps and a range of 30 m. By using a lower bit rate of 10--20 bps, we can further increase the range to 100 m. As smartphones and watches are increasingly being used in underwater scenarios, our software-based approach has the potential to make underwater messaging capabilities widely available to anyone with a mobile device.

MOTIVATION: Quaternary structure determination for transmembrane/soluble proteins requires a reliable computational protocol that leverages observed distance restraints and/or cyclic symmetry (Cn symmetry) found in most homo-oligomeric transmembrane proteins. RESULTS: We survey 118 X-ray crystallographically solved structures of homo-oligomeric transmembrane proteins (HoTPs) and find that ∼97% are Cn symmetric. Given the prevalence of Cn symmetric HoTPs and the benefits of incorporating geometry restraints in aiding quaternary structure determination, we introduce two new filters, the distance-restraints (DR) and the Symmetry-Imposed Packing (SIP) filters. SIP relies on a new method that can rebuild the closest ideal Cn symmetric complex from docking poses containing a homo-dimer without prior knowledge of the number (n) of monomers. Using only the geometrical filter, SIP, near-native poses of 7 HoTPs in their monomeric states can be correctly identified in the top-10 for 71% of all cases, or 29% among 31 HoTP structures obtained through homology modeling, while ZDOCK alone returns 14 and 3%, respectively. When the n is given, the optional n-mer filter is applied with SIP and returns the near-native poses for 76% of the test set within the top-10, outperforming M-ZDOCK's 55% and Sam's 47%. While applying only SIP to three HoTPs that comes with distance restraints, we found the near-native poses were ranked 1st, 1st and 10th among 54 000 possible decoys. The results are further improved to 1st, 1st and 3rd when both DR and SIP filters are used. By applying only DR, a soluble system with distance restraints is recovered at the 1st-ranked pose. AVAILABILITY AND IMPLEMENTATION: https://github.com/capslockwizard/drsip. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

As connected devices continue to decrease in size, we explore the idea of leveraging everyday surfaces such as tabletops and walls to augment the wireless capabilities of devices. Specifically, we introduce Surface MIMO, a technique that enables MIMO communication between small devices via surfaces coated with conductive paint or covered with conductive cloth. These surfaces act as an additional spatial path that enables MIMO capabilities without increasing the physical size of the devices themselves. We provide an extensive characterization of these surfaces that reveal their effect on the propagation of EM waves. Our evaluation shows that we can enable additional spatial streams using the conductive surface and achieve average throughput gains of 2.6-3x for small devices. Finally, we also leverage the wideband characteristics of these conductive surfaces to demonstrate the first Gbps surface communication system that can directly transfer bits through the surface at up to 1.3Gbps.

We present the first wireless physical analytics system for 3D printed objects using commonly available conductive plastic filaments. Our design can enable various data capture and wireless physical analytics capabilities for 3D printed objects, without the need for electronics. To achieve this goal, we make three key contributions: (1) demonstrate room scale backscatter communication and sensing using conductive plastic filaments, (2) introduce the first backscatter designs that detect a variety of bi-directional motions and support linear and rotational movements, and (3) enable data capture and storage for later retrieval when outside the range of the wireless coverage, using a ratchet and gear system. We validate our approach by wirelessly detecting the opening and closing of a pill bottle, capturing the joint angles of a 3D printed e-NABLE prosthetic hand, and an insulin pen that can store information to track its use outside the range of a wireless receiver.

This paper enables data storage and interaction with smart fabric, without the need for onboard electronics or batteries. To do this, we present the first smart fabric design that harnesses the ferromagnetic properties of conductive thread. Specifically, we manipulate the polarity of magnetized fabric and encode different forms of data including 2D images and bit strings. These bits can be read by swiping a commodity smartphone across the fabric, using its inbuilt magnetometer. Our results show that magnetized fabric retains its data even after washing, drying and ironing. Using a glove made of magnetized fabric, we can also perform six gestures in front of a smartphone, with a classification accuracy of 90.1%. Finally, using magnetized thread, we create fashion accessories like necklaces, ties, wristbands and belts with data storage capabilities as well as enable authentication applications.

Our goal is to 3D print wireless sensors, input widgets and objects that can communicate with smartphones and other Wi-Fi devices, without the need for batteries or electronics. To this end, we present a novel toolkit for wireless connectivity that can be integrated with 3D digital models and fabricated using commodity desktop 3D printers and commercially available plastic filament materials. Specifically, we introduce the first computational designs that 1) send data to commercial RF receivers including Wi-Fi, enabling 3D printed wireless sensors and input widgets, and 2) embed data within objects using magnetic fields and decode the data using magnetometers on commodity smartphones. To demonstrate the potential of our techniques, we design the first fully 3D printed wireless sensors including a weight scale, flow sensor and anemometer that can transmit sensor data. Furthermore, we 3D print eyeglass frames, armbands as well as artistic models with embedded magnetic data. Finally, we present various 3D printed application prototypes including buttons, smart sliders and physical knobs that wirelessly control music volume and lights as well as smart bottles that can sense liquid flow and send data to nearby RF devices, without batteries or electronics.

poster Share on Poster: 3D Printing Your Wireless Coverage Authors: Justin Chan Dartmouth College, Hanover, NH, USA Dartmouth College, Hanover, NH, USAView Profile , Changxi Zheng Columbia University, New York, NY, USA Columbia University, New York, NY, USAView Profile , Xia Zhou Dartmouth College, Hanover, NH, USA Dartmouth College, Hanover, NH, USAView Profile Authors Info & Claims MobiCom '15: Proceedings of the 21st Annual International Conference on Mobile Computing and NetworkingSeptember 2015 Pages 227–229https://doi.org/10.1145/2789168.2795164Published:07 September 2015Publication History 0citation210DownloadsMetricsTotal Citations0Total Downloads210Last 12 Months7Last 6 weeks3 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access