RF hardware design is a complicated, time-consuming, and expertise-bound process, which constrains the development and adoption of hardware innovation. Manual workflows do not scale to emerging wireless applications, while existing design generation strategies, e.g., learning-based approaches, lack training efficiency and generalizability across hardware types, frequency bands, operation modes, and substrate materials. In this paper, we present AutoRF, the first agentic framework for automated RF hardware design, supporting metasurfaces and antennas. It allows users to specify their demands and generate corresponding designs. We introduce extensible design abstractions to enable a modular framework. The core of the framework is an efficient and generalizable algorithm for design search and optimization, which utilizes LLMs to drive both circuit model simulator and EM simulator. To boost reliability and performance, we propose custom programming interfaces and a rule reviewer agent as feedback sources to train a specialized LLM. Evaluation demonstrates high success rate and significant optimization speedup; case studies with fabricated metasurface and antennas, ranging from 2.4 GHz to sub-THz, illustrate an ability to derive novel designs for next-generation wireless infrastructure.

Low Earth Orbit (LEO) Synthetic Aperture Radar (SAR) satellites conventionally rely on dedicated communication links, which impose prohibitive hardware, spectrum, and power overhead as satellite constellations scale. This paper presents CommSAR, a novel system that reuses existing SAR imaging waveforms to enable bidirectional communication without modifying satellite hardware or compromising imaging performance. For the downlink, data are embedded by modulating the starting frequency offset of the imaging waveform, preserving the waveform structure and imaging quality. For the uplink, we propose a compact, low-cost programmable metasurface to replace conventional large, expensive antennas, significantly lowering the barrier for dense ground station deployment. To handle extreme satellite dynamics, we employ an opposite-slope waveform as a pilot to compensate for mobility-induced effects. We implement a ground station prototype of CommSAR and validate its performance using an in-orbit commercial SAR satellite and a UAV SAR platform. Experimental results show that CommSAR preserves imaging performance without degradation while achieving downlink and uplink data rates of up to 105 kbps and 112 kbps, respectively, significantly outperforming the state of the art and demonstrating utility-grade performance.

Wireless charging is a cornerstone technology for next-generation mobile and ubiquitous computing. However, its practical deployment has long been constrained by short range, poor flexibility, and lack of support for dynamic multi-device scenarios. In this paper, we propose ChargeX—a system that enables long-range and mobility-resilient wireless charging for multiple small devices. ChargeX pioneers the integration of metasurface-assisted magnetic beamforming, a high-frequency compact transceiver design, and a real-time closed-loop feedback-control mechanism. It further advances the field by introducing a joint optimization framework for dynamically allocating energy across mobile receivers with heterogeneous priorities and spatial-temporal demands. Experimental results demonstrate that it achieves meter-level charging distance, real-time response to device movement, and efficient coordination among multiple receivers, significantly outperforming state-of-the-art prototypes.

Protecting privacy-sensitive hardware usage on mobile devices is crucial. Although mobile operating systems (OSs) and smartphone manufacturers have set the permission settings, attackers can evade these defenses using covert methods, enabling malicious camera recording, microphone eavesdropping, and screen capture. Electronic devices emit unique yet weak electromagnetic interference (EMI) signals when accessing privacy-sensitive hardware. But, these signals are easily affected by foreground application activities and geomagnetic fluctuations caused by device movement. Our prior work showed that supervised learning can extract EMI features correlated with privacy hardware states from complex magnetometer readings, but it requires substantial labeled data, limiting practical deployment to new device models or OS versions. To eliminate this reliance on labeled data, this paper proposes a multimodal contrastive learning framework that leverages the device's built-in magnetometer and synchronized system logs as dual-modal inputs. Through self-supervised training, the framework can learn the intrinsic associations between EMI features and the operating states of privacy-sensitive hardware. Building on this, we design an EMI-based eavesdropping classifier that can analyze a user device's magnetometer readings offline to detect covert eavesdropping activities. Experimental results show that the proposed method can effectively identify eavesdropping behavior related to access to camera, microphone, and screen recording data. Testing across ten diverse mobile devices achieved an average classification accuracy of 89.1% on Android devices and 88.5% on iOS devices for identifying the specific hardware being eavesdropped upon.

Understanding the nature of user-device interactions (e.g., who is using the device and what he/she is doing with it) is critical for many applications including time management, user profiles, and privacy protection. However, in scenarios where mobile devices are shared among family members or multiple employees in a company, conventional account-based statistics are not meaningful. This poses an even bigger problem when dealing with sensitive data. Moreover, fingerprint readers and front-facing cameras were not designed to continuously identify users. In this study, we developedMagPrint++, a novel approach to fingerprint users based on unique patterns in the electromagnetic (EM) signals associated with the specific use patterns of users. Initial experiments showed that time-varying EM patterns are unique to individual users. They are also temporally and spatially consistent, which makes them suitable for fingerprinting.MagPrint++has a number of advantages over existing schemes: i) Non-intrusive fingerprinting, ii) implementation both on COTS mobile phones and a small and easy-to-deploy device, and iii) high accuracy thanks to the proposed classification algorithm. In experiments involving 30 users,MagPrint++achieves$94.3\%$accuracy in classifying users from these traces, which represents a$10.9\%$improvement over the state-of-the-art classification method.

Non-invasive continuous glucose monitoring for diabetes patients remains challenging despite ongoing interest. This paper presents CGMM, a novel non-invasive wireless glucose monitoring system integrated into wearable devices. It features a specially designed metasurface that couples with the wearable's antenna and tissue fluid beneath the skin, amplifying frequency response changes caused by subtle glucose concentration variations. To address individual tissue variability and optimize the passive metasurface design, we develop a tunable metasurface and a one-shot calibration method to obtain the impedance for optimal resonance in glucose sensing environments with unknown parameters. The calibrated impedance is then used for the inverse design and fabrication of an economical passive metasurface. We implement prototypes of CGMM and conduct extensive experimental evaluations. In human experiments involving ten participants using the prototype with LibreVNA, the overall performance is quantified with relative errors ranging from -5.02% to 6.93% and an RMSE of 9.65 mg/dL.

Diffractive neural networks (NNs) have garnered attention for directly implementing wireless signal processing at the physical layer. However, they are limited by a constrained weight learning space and activation functions, which restricts their data processing capabilities. To address this, we propose an RF circuit-based weighted diffraction NN (WDNN) that rivals digital NNs in processing ability. We design a weighted asymmetric RF coupler unit that, when stacked into a network, enables diffractive propagation with arbitrary connection weights. Additionally, an activation module is introduced that utilizes RF amplifiers operating in their nonlinear regions. We validate the effectiveness of the proposed WDNN through three tasks: 32-level amplitude modulated (AM) signal decoding, 31-class angle of arrival (AoA) estimation, and 2-class Wi-Fi based fall detection. After training, WDNN achieves the accuracy of 98.5%, 93.7%, and 90.8% in the AM decoding, AoA estimation, and fall detection tasks, respectively; while the diffractive NN SOTA achieves only 21.6%, 16.9%, and 63.3%. We also implement the prototypes of WDNN and SOTA, and real-world experimental results demonstrate that our method achieves an average accuracy improvement of up to 76.85% across various tasks compared to SOTA.

Bluetooth Low Energy (BLE) direction finding, a feature introduced in BLE version 5.1, enables precise localization through Angle of Arrival (AoA) estimation. However, this advancement introduces new risk to BLE direction finding based localization system. Specifically, the AoA estimation based on phase sampling of constant-tone-extension (CTE) is susceptible to the signal injection attack. This paper presents SaDiF, a feasible spoofing attack mechanism to mislead the locators into mistaking the positioning result as a continuous path. By eavesdropping on BLE packets and injecting attack signals containing pre-designed disturbing phase shift, SaDiF subtly alters the AoA estimation without detection, thus interfere the localization results. Moreover, SaDiF address the challenges posed by hardware imperfections by proposing an injection timing optimization to improve attack robustness. Extensive experiments demonstrates the effectiveness of SaDiF in successfully attacking multiple BLE targets in real-time scenarios. In conclusion, our findings reveal critical security risks in BLE direction finding feature and provide insights into strengthening its defenses.

This paper presents MODepth, a multi-frame monocular depth estimation system based on the controlled motion of an optical image stabilization (OIS) module. By actively injecting acoustic signals, we induce regular translational movements of the OIS lens, resulting in controllable camera pose changes and simplifying inter-frame pose estimation. Leveraging multi-frame images captured under OIS-controlled lens movements, we design a high-precision depth estimation network, MODNet, and introduce the principal point offset estimation module and pose estimation modules to fully exploit geometric information across frames. To validate the effectiveness of our approach, we collect a new dataset MODdata with 1100 samples in nearly 220 indoor scenarios and benchmark our model as an OIS-based multi-frame depth estimation method, comparing it to ground truth obtained from a depth sensor and other state-of-the-art monocular depth estimation algorithms. Our method achieves competitive or superior performance compared to fully supervised baselines, reaching an RMSE of 0.439, which outperforms all evaluated methods, demonstrating that self-supervised fine-tuning with OIS-induced parallax is a viable alternative to ground-truth supervision. Code and dataset are available at: https://github.com/liangjindeamo-yuer/MODEPTH

Quick Response (QR) codes have become increasingly popular for screen-camera communication due to their swift readability and widespread smartphone use. Nevertheless, they are vulnerable to privacy invasions from unauthorized photography. Addressing this, we propose a novel Moiré encryption technique-based secure screen-camera communication system, named MoiréComm. The Moiré encryption can enhance security by using distinct spatial frequency patterns for camouflage. The original QR code is revealed as a Moiré pattern only when the camera in a designated position, e.g., directly in front and 30 cm from the screen. From any other positions, only the camouflaged QR code can be seen. Decryption schemes are customized for different scenarios. The multi-frame approach achieves a decryption success of over 98.6% within 13.2 frames in handheld scenarios. Conditional generative adversarial network (cGAN)-based decryption method decodes the Moiré QR code images with a 98.8% success rate in 0.02 s within three frames and is also applicable in handheld scenarios. For fixed screen-camera setups, our fast decryption scheme achieves 99.4% success within two frames, with average 0.4 s latency. Significantly, the decryption rate plunges to 0% for surveillance cameras displaced by 20$^\circ$or more than$\ge$10 cm from the target position, demonstrating MoiréComm's resilience against attacks.

Wearable devices, including smartwatches, are increasingly popular among consumers due to their user-friendly services. However, transmitting sensitive data like social media messages and payment QR codes via commonly used low-power Bluetooth exposes users to privacy breaches and financial losses. This study introducesTouchHBC, a secure and reliable communication scheme leveraging a smartwatch's built-in electrodes. This system establishes a touch-based human communication system utilizing a laptop's leakage current. As the transmitting device, the laptop modulates this current via the CPU. Simultaneously, the smartwatch, equipped with built-in electrodes, captures the current traversing the human body and decodes it. The modulation and decoding processes involve techniques such as amplitude modulation, variational mode decomposition, channel estimation, and retransmission mechanisms.TouchHBCfacilitates communication between laptops and smartwatches. Real-world tests demonstrate that our prototype achieves a throughput of$19.83bps$. Moreover,TouchHBCoffers the potential for enhanced interaction, including improved gaming experiences through vibration feedback and secure touch login for smartwatch applications by synchronizing with a laptop. Furthermore, the system can be integrated with high-throughput communication protocols such as Bluetooth, enhancing its scalability while maintaining a strong foundation of security.

Various characteristics of mobile applications (apps) and associated in-app services can reveal potentially-sensitive user information; however, privacy concerns have prompted third-party apps to restrict access to data related to mobile app usage. This paper outlines a novel approach to extracting detailed app usage information by analyzing electromagnetic (EM) signals emitted from mobile devices during app-related tasks. The proposed system, MagSpy, recovers user privacy information from magnetometer readings that do not require access permissions. This EM leakage becomes complex when multiple apps are used simultaneously and is subject to interference from geomagnetic signals generated by device movement. To address these challenges, MagSpy employs multiple techniques to extract and identify signals related to app usage. Specifically, the geomagnetic offset signal is canceled using accelerometer and gyroscope sensor data, and a Cascade-LSTM algorithm is used to classify apps and in-app services. MagSpy also uses CWT-based peak detection and a Random Forest classifier to detect PIN inputs. A prototype system was evaluated on over 50 popular mobile apps with 30 devices. Extensive evaluation results demonstrate the efficacy of MagSpy in identifying in-app services (96% accuracy), apps (93.5% accuracy), and extracting PIN input information (96% top-3 accuracy).

As a promising paradigm, federated learning has been applied to multimodal sensing tasks due to its deployment convenience. However, the recent advances in multimodal federated learning emphasize learning a high-quality multimodal model but overlook the model usage requirements of massive unimodal clients. Moreover, the privacy risk in model sharing and client data heterogeneity impact the efficacy of federated learning. In this paper, we propose a novel multimodal federated learning system named MASA. As a departure from existing approaches, MASA simultaneously enhances the model learning efficiency of both multimodal and unimodal clients while ensuring their data privacy. First, we employ a gated cross-modal distillation scheme to achieve performance-aware knowledge transfer across modality-heterogeneous clients. To enhance the system security, MASA integrates a lightweight split-shuffle mechanism to realize the anonymization and encryption of model aggregation. Moreover, to reach personalized collaboration while protecting privacy, MASA features an attention-based spontaneous client clustering mechanism to form client cluster structures securely and distributedly. We evaluate our MASA on four public multimodal datasets for human activity recognition. The results show that our MASA outperforms leading multimodal federated learning methods on the model performance of both multimodal and unimodal clients.

Air writing technology enhances text input for IoT, VR, and AR devices, offering a spatially flexible alternative to physical keyboards. Addressing the demand for such innovation, this paper presents MagicWrite, a novel system utilizing acoustic-based 1D tracking, which is suitable for mobile devices with existing speaker and microphone infrastructure. Compared to 2D or 3D tracking of the finger, 1D tracking eliminates the need for multiple microphones and/or speakers and is more universally applicable. However, challenges emerge when using 1D tracking for recognizing handwritten letters due to trajectory loss and inter-user writing variability. To address this, we develop a general conversion technique that transforms image-based text datasets (e.g., MNIST) into 1D tracking trajectory data, generating artificial datasets of tracking traces (referred to asTrackMNISTs) to bolster system robustness and scalability. These tracking datasets facilitate the creation of personalized user databases that align with individual writing habits. Combined with a kNN classifier, our proposed MagicWrite ensures high accuracy and robustness in text input recognition while simultaneously reducing computational load and energy consumption. Extensive experiments validate that our proposed MagicWrite achieves exceptional classification accuracy for unseen users and inputs in five languages, marking it as a robust solution for air writing.

Acoustic tracking technology, leveraging the ubiquitous presence of speakers and microphones in commercial off-the-shelf (COTS) mobile devices, has become a versatile tool across various applications. However, current phase-based acoustic tracking methods encounter significant limitations in tracking fast movements, thereby restricting their practical utility. This paper identifies three practical challenges to enable fast hand motion tracking using acoustic signals: 1) high mobility, 2) low signal-to-noise ratio (SNR), and 3) variations in hardware frequency response. The high mobility introduces Doppler shift and phase ambiguity which is the primary cause of failure in fast movement tracking, while the latter two factors can further impair the tracking performance in practical scenarios involving high mobility. To address the high mobility issue, we effectively compensate the Doppler shift in the Channel Impulse Response (CIR) for better selection of channel taps and then propose a novel phase derivative approach to mitigate the phase ambiguity. To enhance the real-world robustness, we integrate multiple algorithms including an SNR enhancement algorithm inspired by time-domain beamforming and a hardware frequency response compensation approach that addresses both amplitude and phase distortions. Additionally, an LSTM-based distance reconstruction algorithm is further implemented to correct residual phase noise. Implemented on Android platforms under the name SwiftTrack+, our system demonstrates superior performance in tracking fast movements. Through extensive evaluations, SwiftTrack+ proves its efficacy across diverse scenarios, significantly broadening the scope and reliability of acoustic tracking applications.

Wireless charging has achieved promising progress in recent years. However, the severe bottlenecks are the small charging range and poor flexibility. This paper presents ChargeX to enable smart and long-range wireless charging for small mobile devices. ChargeX incorporates emerging smart metasurface into the magnetic resonance coupling-based wireless charging to extend the charging range and accommodates the mobility of charging device. Unlike previous endeavors in metasurface-assisted wireless charging that focused on simulation, ChargeX makes efforts across software and hardware to meet three crucial requirements for a practical wireless charging system: (i) realize high-freedom and accurate metasurface control under the premise of low loss; (ii) obtain real-time feedback from the receiver and make effective manipulation for transmitted magnetic flux; and (iii) generate a proper AC signal source at the desired frequency band. We developed a prototype of ChargeX, and evaluated its performance through controlled experiments and real-world phone charging. Extensive experiments demonstrate the great potential of ChargeX for long-range and flexible wireless charging with a compact receiver design.

mmWave networks offer wide bandwidth for high-speed wireless communication but suffer from limited range and susceptibility to blockage. Existing coverage provisioning solutions not only incur high costs but also require significant expert knowledge and manual efforts. In this paper, we present AutoMS, an automated service framework to optimize mmWave coverage by strategically designing and placing low-cost passive metasurfaces. Our approach consists of three key components: (1) joint optimization of metasurface phase configurations and placement as well as access point beamforming codebooks. (2) a fast 3D ray-tracing simulator for accelerated large-scale metasurface channel modeling. (3) a metasurface design amenable to ultra-low-cost hot stamping fabrication, featuring high reflectivity, near 2π phase control, and wideband support. Simulation and testbed experiments show that AutoMS can increase the median received signal strength by 11 dB in target rooms and over 20 dB at previous blind spots, and improve the median throughput by over 3× in real-world scenarios.

Microwave ovens have become an essential cooking appliance owing to their convenience and efficiency. However, microwave ovens suffer from uneven distribution of energy, which causes prolonged delays, unpleasant cooking experiences, and even safety concerns. Despite significant research efforts, current solutions remain inadequate. In this paper, we first conduct measurement studies to understand the energy distribution for 10 microwave ovens and show their energy distribution in both 2D and 3D is very skewed, with notably lower energy levels at the center of the microwave cavity, where food is commonly placed. To tackle this challenge, we propose a novel methodology to enhance the performance of microwave ovens. Our approach begins with the development of a measurement driven model of a microwave oven. We construct a detailed 3D model in the High Frequency Structure Simulator (HFSS) and use real temperature measurements from a microwave to derive critical parameters relevant to the appliance's functionality (e.g., operating frequency, waveguide specifications). We then develop a novel approach that optimizes the design and placement of a low-cost passive metasurface for a given heating objective. Using extensive experiments, we demonstrate the efficacy of our approach across diverse food, optimization objectives, and microwave ovens.

Global Navigation Satellite System (GNSS) is extensively utilized for outdoor positioning and navigation. However, achieving high-precision indoor positioning is challenging due to the significant attenuation of GNSS signals indoors. To address this issue, we propose an innovative indoor GNSS positioning system called GPMS, which uses passive metasurface technology to redirect GNSS signals from outdoors into indoor spaces. These passive metasurfaces are strategically optimized for indoor coverage by steering and scattering the GNSS signals across a wide range of incident angles. We further develop a novel localization algorithm that can determine which metasurface the signal goes through and localize the user using the set of metasurfaces as anchor points. A distinct advantage of our localization algorithm is that it can be implemented on existing mobile devices without any hardware modifications. We implement the prototype of GPMS, and deploy six metasurfaces in two indoor environments, a 10×50 m2 office floor and a 15×20 m2 lecture room, to evaluate system performance. In terms of coverage, our GPMS increases the C/N0 from 9.1 dB-Hz to 23.2 dB-Hz and increases the number of visible satellites from 3.6 to 21.5 in the office floor. In terms of indoor positioning accuracy, our proposed system decreases the absolute positioning error from 30.6 m to 3.2 m in the office floor, and from 11.2 m to 2.7 m in the lecture room, demonstrating the feasibility and benefits of metasurface-assisted GNSS for indoor positioning.

The demand for ultra-high-resolution imaging in mobile phone photography is continuously increasing. However, the image resolution of mobile devices is typically constrained by the size of the CMOS sensor. Although deep learning-based super-resolution (SR) techniques have the potential to overcome this limitation, existing SR neural network models require large computational resources, making them unsuitable for real-time SR imaging on current mobile devices. Additionally, cloud-based SR systems pose privacy leakage risks. In this paper, we propose M3Cam, an innovative and lightweight SR imaging system for mobile phones. M3Cam can ensure high-quality 16× SR image (4× in both height and width) visualization with almost negligible latency. In detail, we utilize an optical image stabilization (OIS) module for lens control and introduce a new modality of data, namely gyroscope readings, to achieve high-precision and compact optical flow estimation modules. Building upon this concept, we design a multi-frame-based SR model utilizing the Swin Transformer. Our proposed system can generate a 16× SR image from four captured low-resolution images in real-time, with low computational load, low inference latency, and minimal reliance on runtime RAM. Through extensive experiments, we demonstrate that our proposed multi-modal optical flow model significantly enhances pixel alignment accuracy between multiple frames and delivers outstanding 16× SR imaging results under various shooting scenarios. Code and dataset are available at: https://github.com/liangjindeamo-yuer/M3CAM

The convenient text input system is a pain point for devices such as AR glasses, and it is difficult for existing solutions to balance portability and efficiency. This paper introduces HandPad, the system that turns the hand into an on-the-go touchscreen, which realizes interaction on the hand via human capacitance. HandPad achieves keystroke and handwriting inputs for letters, numbers, and Chinese characters, reducing the dependency on capacitive or pressure sensor arrays. Specifically, the system verifies the feasibility of touch point localization on the hand using the human capacitance model and proposes a handwriting recognition system based on Bi-LSTM and ResNet. The transfer learning-based system only needs a small amount of training data to build a handwriting recognition model for the target user. Experiments in real environments verify the feasibility of HandPad for keystroke (accuracy of 100%) and handwriting recognition for letters (accuracy of 99.1%), numbers (accuracy of 97.6%) and Chinese characters (accuracy of 97.9%).

Motivated by many potential applications that could be enabled by acoustic motion tracking, in this paper we systematically examine the factors that limit the accuracy of acoustic tracking in practical scenarios. We identify three main challenges: (i) high mobility, (ii) low SNR, and (iii) hardware frequency response. We further show that the last two issues may exacerbate the performance issue under high mobility. We develop effective approaches to address the issues. In particular, to address high mobility, we tackle phase wrap-around using the derivative of the phase; we further estimate the Doppler shift under diverse scenarios and compensate the Doppler in channel impulse response (CIR). To address low SNR, we use a novel approach to estimate the phase shift between consecutive time intervals to effectively support time-domain beamforming and increase SNR. To tackle the uneven frequency response, we show that it is important to estimate and compensate the phase as well as the amplitude of the frequency response. Our extensive evaluation shows that each of our techniques is effective and putting them together significantly enhances the accuracy of acoustic motion tracking in general scenarios.

Low Earth Orbit (LEO) satellite communication is essential for wireless communication. While manufacturing and launching LEO satellites have become efficient and cost-effective, ground stations remain expensive due to complex designs for handling severe path losses and precise beam tracking. Hence, it is important to develop low cost and high-performance ground stations for widespread adoption of LEO satellite communication. Towards realizing this goal, we design a passive metasurface-enhanced LEO ground station system, named PMSat, combining metasurface's fine-grained beamforming capability with a small-size phased array's adaptive steering and focusing. For uplink, we jointly optimize the phase array codebook and uplink metasurface phase profile, and realize electronic steering by switching the codeword. We further jointly optimize the downlink metasurface phase profile to improve the focusing performance and enhance the received signal strength (RSS) over a wide range of incident angles. Our PMSat prototype consists of a single passive metasurface with 21 × 21 elements for uplink and 22 × 22 for downlink, along with 1 × 4 receiving and 1 × 4 transmitting phased array antennas. The effectiveness of our proposed PMSat is validated through extensive experiments, and results demonstrate that the optimized metasurface improves the SNR by 8.32 dB and 16.57 dB for uplink and downlink, respectively.

Mobile devices have emerged as the most popular platforms to access information. However, they have also become a major concern of privacy violation and previous researches have demonstrated various approaches to infer user privacy based on mobile devices. In this paper, we study the electromagnetic (EM) emission of a laptop that could be harvested by a commercial-off-the-shelf (COTS) mobile device, e.g., a smartphone. We proposeMagAttack, which exploits the electromagnetic side channel of a laptop to guess user activities, i.e., application launching and application operation. The key insight ofMagAttackis that applications are discrepant in essence due to the different compositions of instructions, which can be reflected on the CPU power consumption, and thus the corresponding EM emissions.MagAttackis challenging since that EM signals are noisy due to the dynamics of applications and the limited sampling rate of the built-in magnetometers in COTS mobile devices. We overcome these challenges and convert noisy coarse-grained EM signals to robust fine-grained features. We implementMagAttackon both an iOS and an Android smartphone without any hardware modification, and evaluate its performance with 30 popular applications, 30 YouTube videos, and 50 top websites in China. The results demonstrate thatMagAttackcan recognize aforementioned 30 applications with an average accuracy of 98.6 percent, and identify which video out of the 30 candidates being played with an average accuracy of 97.5 percent and visiting which website among the 50 candidates with an average accuracy of 90.4 percent.

Near-field communication (NFC) technology emerges as a vital role with appealing benefits for users to improve mobile device’s functionality. Although today’s most smartphones and smartwatches come with NFC support, other mobile devices (e.g., PC and laptops) and IoT devices that don’t equip with dedicated radio modules cannot take advantage of wide-scale NFC capability. We design and developMagneComm+, an NFC-like implementation scheme without dedicated hardware and propose a novel near-field communication protocol that is applicable to almost all mobile devices and IoT devices. The key idea is to utilize the electromagnetic induction (EMI) signal emitted from the computing devices (e.g., CPUs) and captured by magnetometers on mobile devices for communication. We tackle challenges indata encoding/decoding,preamble detection,retransmission and error correction,multi-transmitter, andfull-duplexschemes, to efficiently generate and reliably receive EMI signal with the hardware available on devices. We prototypeMagneComm+on both between laptops and smartphones, as well as between two laptops with an external magnetometer. Extensive evaluation results show that ourMagneComm+supports around$10~cm$10cmcommunication distance with average110 bps(bit per second) data rate on the normal-speed mode, and maximum17.28 kbpson the full-speed mode.

Quick response (QR) codes have been widely used in mobile applications, especially mobile payments, such as Alipay, WeChat, PayPal, etc due to their convenience and the pervasive built-in cameras on smartphones. Recently, however, attacks against QR codes have been reported and attackers can capture a QR code of the victim and replay it to achieve a fraudulent transaction or intercept private information, just before the original QR code is scanned. In this study, we enhance the security of a QR code by identifying its authenticity. We propose ScreenID, which embeds a QR code with information of the screen which displays it, thereby the QR code can reveal whether it is reproduced by an adversary or not. In ScreenID, PWM frequency of screens is exploited as the unique screen fingerprint. To improve the estimation accuracy of PWM frequency, ScreenID incorporates a model for the interaction between the camera and screen in the temporal and spatial domains. Extensive experiments demonstrate that ScreenID can differentiate screens of different models, types, and manufacturers and thus improve the security of QR codes.

Multi-frame super-resolution methods can generate high resolution images by combining multiple captures of the same scene; however, the performance of merged results are susceptible to degradation due to a lack of precision in image registration. In this study, we sought to develop a robust multi-frame super resolution method (called OISSR) for use on smartphone cameras with a optical image stabilizer (OIS). Acoustic injection is used to alter the readings from the built-in MEMS gyroscope to control the lens motion in the OIS module (note that the image sensor is fixed). We employ a priori knowledge of the induced lens motion to facilitate optical flow estimation with sub-pixel accuracy, and the output high-precision pixel alignment vectors are utilized to merge the multiple frames to reconstruct the final super resolution image. Extensive experiments on a OISSR prototype implemented on a Xiaomi 10Ultra demonstrate the high performance and effectiveness of the proposed system in obtaining the quadruple enhanced resolution imaging.

Optical image stabilizers (OIS) are widely used in digital cameras to counteract motion blur caused by camera shakes in capturing videos and photos. In this paper, we sought to expand the applicability of the lens-shift OIS technology for metric depth estimation, i.e., let a RGB camera to achieve the similar function of a time-of-flight (ToF) camera. Instead of having to move the entire camera for depth estimation, we propose DoCam, which controls the lens motion in the OIS module to achieve 3D reconstruction. After controlling the lens motion by altering the MEMS gyroscopes readings through acoustic injection, we improve the traditional bundle adjustment algorithm by establishing additional constraints from the linearity of the lens control model for high-precision camera pose estimation. Then, we elaborate a dense depth reconstruction algorithm to compute depth maps at real-world scale from multiple captures with micro lens motion (i.e., ≤ 3 mm). Extensive experiments demonstrate that our proposed DoCam can enable a 2D color camera to estimate high-accuracy depth information of the captured scene by means of controlling lens motion in the OIS. DoCam is suitable for a variety of applications that require depth information of the scenes, especially when only a single color camera is available and located at a fixed position.

This study reveals that on-board hardware modules leak electromagnetic (EM) emissions whenever audio or camera data is accessed, and proposes Magdefender scheme that explores the possibility of using the magnetometer built into mobile devices to detect eavesdropping instances by malicious apps and even the unscrupulous phone vendors. However, the target EM signals generated by accessing multimedia data is weak and tends to be buried beneath other noisy EM signals from apps running in the foreground. It is also subject to the external interference from geomagnetic signals generated by the device movement. To cope with the challenges, we adopt a generative adversarial networks (GAN) based model to facilitate the extraction of target EM signals indicating the occurrence of eavesdropping from the overall magnetometer readings. We also develop a neural network-based classifier with triplet loss embedding to identify the EM signals from the camera and/or microphones. Empirical results demonstrate the efficacy of MagDefenderin recognizing instances of eavesdropping on cameras/microphones data, with average accuracy of 97.3% when applied to the trained devices, and average 91.5% on unseen mobile devices.

Quick response (QR) codes have been widely used in mobile applications due to its convenience and the pervasive built-in cameras on smartphones. Recently, however, attacks against QR codes have been reported that attackers can capture a QR code of the victim and replay it to achieve a fraudulent transaction or intercept private information, just before the original QR code is scanned. In this study, we enhance the security of a QR code by identifying its authenticity. We propose SCREENID, which embeds a QR code with information of the screen which displays it, thereby the QR code can reveal whether it is reproduced by an adversary or not. In SCREENID, PWM frequency of screens is exploited as the unique screen fingerprint. To improve the estimation accuracy of PWM frequency, SCREENID incorporates a model for the interaction between the camera and screen in the temporal and spatial domains. Extensive experiments demonstrate that SCREENID can differentiate screens of different models, types, and manufacturers, thus improve the security of QR codes.

Text input systems based on device-free finger tracking technologies have attracted considerable attention in the use scenarios of mobile and the Internet-of-Things (IoT) devices. Issues pertaining to 2D tracking have prompted interest in using 1D finger trajectories for the recognition of handwritten letters. Nonetheless, 1D tracking imposes two major challenges: (i) Trajectory information loss from 2D to 1D; and (ii) Inter-user diversity in writing traits. These challenges could possibly be overcome by collecting a large training dataset for every user; however, this would impose an unacceptable burden on users. This paper presents a text input system with multi-language support without training using acoustic-based 1D finger tracking technology. We developed a novel data augmentation scheme, in which the handwritten image dataset MNISTs are used to create artificial datasets (called TrackMNISTs). We compensate for the trajectory information loss of 1D by creating personal dataset (from TrackMNIST) to match the writing habits of individual users. The proposed data augmentation mechanism is also applicable to multilingual letter recognition. In experiments, MagicInput achieved outstanding classification accuracy on unseen users: 10 digits (98.3%), 26 uppercase/lowercase English letters (97.8%/95.3%), 49 Japanese characters (91.4%), and the 30 commonly used Chinese characters (93.8%).

Various characteristics of mobile applications (apps) and associated in-app services have been used reveal potentially-sensitive user information; however, privacy concerns have prompted third-party apps to rigorously restrict access to data related to mobile app usage. This paper outlines a novel approach to the extraction of detailed app usage information based on analysis of the electromagnetic (EM) signals emitted from mobile devices when executing app-related tasks. Note that this type of EM leakage becomes high-complex when multiple apps are used simultaneously and is subject to interference from geomagnetic signals generated by device movement. This paper proposes a deep learning-based multi-label classification system to identify apps and in-app services based on magnetometer readings. The proposed MAGTHIEF system uses accelerometer and gyroscope data to cancel out the offset in geomagnetic signals followed by an elaborate deep region convolution neural network (DRCNN) to differentiate among multiple apps and the corresponding inapp services. Experiments on 50 apps demonstrated the efficacy of MAGTHIEF in identifying multiple apps and in-app services, achieving high average macro F1 scores of 0.87 and 0.95, respectively. MAGTHIEF also achieved time duration accuracy of 89.5% in recognizing app trajectory in the real-world scene.

Understanding the nature of user-device interactions (e.g., who is using the device and what he/she is doing with it) is critical to many applications including time management, user profiles, and privacy protection. However, in scenarios where mobile devices are shared among family members or multiple employees in a company, conventional account-based statistics are not meaningful. This poses an even bigger problem when dealing with sensitive data. Moreover, fingerprint readers and front-facing cameras were not designed to continuously identify users. In this study, we developed MagPrint, a novel approach to fingerprint users based on unique patterns in the electromagnetic (EM) signals associated with the specific use patterns of users. Initial experiments showed that time-varying EM patterns are unique to individual users. They are also temporally and spatially consistent, which makes them suitable for fingerprinting. MagPrint has a number of advantages over existing schemes: i) Non-intrusive fingerprinting, ii) implementation using a small and easy-to-deploy device, and iii) high accuracy thanks to the proposed classification algorithm. In experiments involving 30 users, MagPrint achieves 94.3% accuracy in classifying users from these traces, which represents an 10.9% improvement over the state-of-the-art classification method.

Quick response (QR) codes have been widely used in mobile applications, due to its convenience and the pervasive built-in cameras on smartphones. Recently, however, QR codes have been reported suffering attacks for being sniffed just before the QR code is scanned, which lead to financial loss. In this study, we propose ScreenID, for enhancing the QR code security by identifying its authenticity, which embeds a QR code with information of unique screen fingerprint - PWM frequency. PWM frequencies are adjusted to different values by screen manufacturers, therefore can successfully differentiate screens. To improve the estimation accuracy of PWM frequency, ScreenID incorporates a model for the interaction between the camera and screen in the temporal and spatial domains. Extensive experiments demonstrate that ScreenID can differentiate screens of different models, types and manufacturers and thus improve the security of QR codes.

Mobile devices have emerged as the most popular platforms to access information. However, they have also become a major concern of privacy violation and previous researches have demonstrated various approaches to infer user privacy based on mobile devices. In this paper, we study a new side channel of a laptop that could be harvested by a commercial-off-the-shelf (COTS) mobile device, eg, a smartphone. We propose MagAttack, which exploits the electromagnetic (EM) side channel of a laptop to infer user activities, i.e., application launching and application operation. The key insight of MagAttack is that applications are discrepant in essence due to the different compositions of instructions, which can be reflected on the CPU power consumption, and thus the corresponding EM emissions. MagAttack is challenging since that EM signals are noisy due to the dynamics of applications and the limited sampling rate of the built-in magnetometers in COTS mobile devices. We overcome these challenges and convert noisy coarse-grained EM signals to robust fine-grained features. We implement MagAttack on both an iOS and an Android smartphone without any hardware modification, and evaluate its performance with 13 popular applications and 50 top websites in China. The results demonstrate that MagAttack can recognize aforementioned 13 applications with an average accuracy of 98.6%, and figure out the visiting operation among 50 websites with an average accuracy of 84.7%.

Quick response (QR) codes are becoming pervasive due to their rapid readability and the popularity of smartphones with built-in cameras. QR codes are also gaining importance in the retail sector as a convenient mobile payment method. However, researchers have concerns regarding the security of QR codes, which leave users susceptible to financial loss or private information leakage. In this study, we addressed this issue by developing a novel QR code (called mQRCode), which exploits patterns presenting a specific spatial frequency as a form of camouflage. When the targeted receiver holds a camera in a designated position (e.g., directly in front at a distance of 30 cm from the camouflaged QR code), the original QR code is revealed in form of a Moire pattern. From any other position, only the camouflaged QR code can be seen. In experiments, the decryption rate of mQRCode was > 98.6% within 10.2 frames via a multi-frame decryption method. The decryption rate for cameras positioned 20° off axis or > 10cm away from the designated location dropped to 0%, indicating that mQRCode is robust against attacks.

Quick response (QR) codes are becoming pervasive due to their rapid readability and the popularity of smartphones with built-in cameras. QR codes are also gaining importance in the retail sector as a convenient mobile payment method. However, researchers have concerns regarding the security of QR codes, which leave users susceptible to financial loss or private information leakage. In this study, we address this issue by developing a novel QR code (called mQR code), which exploits patterns presenting a specific spatial frequency as a form of camouflage. When the targeted receiver holds a camera in a designated position (e.g., directly in front at a distance of 30 cm from the camouflaged QR code), the original QR code is revealed in form of a Moiré pattern. From any other position, only the camouflaged QR code can be seen. In experiments, the decryption rate of mQR codes is $> 98%$. The decryption rate for cameras positioned $20\degree$ off axis or $> 10cm$ from the designated location drops to $0%$, indicating that any attackers will be unable to steal a usable image.

Near-field communication (NFC) plays a crucial role in the operation of mobile devices to enhance applications such as payment, social networks, private communication, gaming, and etc. Despite of the convenience, existing NFC standards like ISO-13157 require additional hardware (e.g., loop antenna and dedicated chip) and thereby hindering their wide-scale applications. In this work, we seek to propose a novel near-field communication protocol, MagneComm, which utilizes Magnetic Induction (MI) signals emitted from CPUs and captured by magnetometers on mobile devices for communication. Since CPUs and magnetometers are readily available components in mobile devices, MagneComm eliminates the requirement for special hardware and complements existing near-field communication protocols by providing additional bandwidth. We systematically analyze the characteristics of magnetic signals of CPUs and facilitate MagneComm with one-way communication, full-duplex communication, and multi-transmitter schemes in accordance with the hardware availability on devices. We prototype MagneComm on both laptops and smartphones. Extensive evaluation results show that MagneComm achieves up to 110bps within 10cm.