Time series is an important form of data generated by Internet-of-Things (IoT) devices. Closed-form continuous-time (CFC) neural networks offer superior expressivity for modeling time series data compared with recurrent neural networks. Additionally, their lower training and inference overhead make them well-suited for deployment on microcontroller-based IoT devices. This paper introduces FedCFC, an on-device federated learning network that operates based on the CFC models distributed across IoT devices. FedCFC incorporates a novel and communication-efficient aggregation strategy designed to mitigate the effects of class distribution imbalances across the participating IoT devices’ training data. The strategy is designed based on a new property of CFC identified in this paper, i.e., the insensitivity of a sub-model of CFC with respect to training data’s class distribution shift. Extensive evaluation with multiple time series datasets demonstrates that FedCFC attains comparable or superior accuracy while achieving a 7.6× to 11× reduction in communication overhead compared with recent federated learning approaches designed to address the class distribution skew problem. Furthermore, deployments of FedCFC on four IoT platforms highlight its suitability for resource-constrained devices with as little as 256 kB of memory or even less.

Stochastic differential equation networks (SDENets), a subset of continuous-time neural networks, offer the natural ability to model continuous time-series data with greater expressivity than discrete-time neural networks. However, SDENets face challenges related to stability and high computational overhead. In this paper, we introduce SE-SDENet, a stable and efficient variant of SDENet. Leveraging the inherent capability of SDENets to model randomness in time-series data, we establish a theoretical framework that ensures SE-SDENet's stability during training by regulating the dynamics of each neuron. Additionally, we propose an efficient training and inference framework that enables SE-SDENet to achieve low forward-pass complexity and to dynamically adjust its complexity at run-time. Evaluation with four datasets and four edge devices demonstrates that SE-SDENet achieves a 6.6x higher throughput than the solver-based SDENet and exhibits improved stability in handling noisy data and long-term predictions. A validation on a robot vehicle shows that SE-SDENet can dynamically adjust its complexity at run-time to meet varying resource constraints.

Closed-form continuous-time (CFC) neural networks have superior expressivity in modeling time series data compared with recurrent neural networks. CFC’s lower training and inference overheads also make it appealing for microcontroller-based platforms. This paper proposes FedCFC, which advances CFC from the centralized learning setting to the federated learning paradigm. FedCFC features a novel and communication-efficient aggregation strategy to address the problem of class distribution skews across clients’ training data. The strategy is designed based on a new empirical property of CFC identified in this paper, i.e., involatility of a sub-network of CFC with respect to training data’s class distribution. Extensive evaluation based on multiple time series datasets shows that FedCFC achieves higher or similar accuracy with 7.6× to 11× reduction in communication overhead, compared with recent federated learning approaches designed to address the class distribution skew problem. Implementations of FedCFC on four microcontroller platforms show its portability to low-end computing devices with 256kB memory and even less.

Camera-based computer vision is essential to autonomous vehicle's perception. This paper presents an attack that uses light-emitting diodes and exploits the camera's rolling shutter effect to create adversarial stripes in the captured images to mislead traffic sign recognition. The attack is stealthy because the stripes on the traffic sign are invisible to human. For the attack to be threatening, the recognition results need to be stable over consecutive image frames. To achieve this, we design and implement GhostStripe, an attack system that controls the timing of the modulated light emission to adapt to camera operations and victim vehicle movements. Evaluated on real testbeds, GhostStripe can stably spoof the traffic sign recognition results for up to 94% of frames to a wrong class when the victim vehicle passes the road section. In reality, such attack effect may fool victim vehicles into life-threatening incidents. We discuss the countermeasures at the levels of camera sensor, perception model, and autonomous driving system.

Camera-based computer vision is crucial for autonomous vehicle perception. We demonstrate GhostStripe [5], an attack system that uses light-emitting diodes and exploits the camera's rolling shutter effect to generate adversarial stripes that are invisible to humans while misleading traffic sign recognition. To maintain stable attack effectiveness, GhostStripe controls the timing of the modulated light emission, adapting to both the camera's framing operation and the movement of the victim vehicle. Evaluated on real testbeds, GhostStripe can stably spoof traffic sign recognition results for up to 97% of frames to a wrong class when the victim vehicle passes the road section.

Tracking interpersonal distances is essential for real-time social distancing management and ex-post contact tracing to prevent spreads of contagious diseases. Bluetooth neighbor discovery has been employed for such purposes in combating COVID-19, but does not provide satisfactory spatiotemporal resolutions. This paper presents ImmTrack, a system that uses a millimeter wave radar and exploits the inertial measurement data from user-carried smartphones or wearables to track interpersonal distances. By matching the movement traces reconstructed from the radar and inertial data, the pseudo identities of the inertial data can be transferred to the radar sensing results in the global coordinate system. The re-identified, radar-sensed movement trajectories are then used to track interpersonal distances. In a broader sense, ImmTrack is the first system that fuses data from millimeter wave radar and inertial measurement units for simultaneous user tracking and re-identification. Evaluation with up to 27 people in various indoor/outdoor environments shows ImmTrack’s decimeters-seconds spatiotemporal accuracy in contact tracing, which is similar to that of the privacy-intrusive camera surveillance and significantly outperforms the Bluetooth neighbor discovery approach.

Eavesdropping on private conversations is one of the most common yet detrimental threats to privacy. A number of recent works have explored side-channels on smart devices for recording sounds without permission. This paper presents LidarPhone, a novel acoustic side-channel attack through the lidar sensors equipped in popular commodity robot vacuum cleaners. The core idea is to repurpose the lidar to a laser-based microphone that can sense sounds from subtle vibrations induced on nearby objects. LidarPhone carefully processes and extracts traces of sound signals from inherently noisy laser reflections to capture privacy sensitive information (such as speech emitted by a victim's computer speaker as the victim is engaged in a teleconferencing meeting; or known music clips from television shows emitted by a victim's TV set, potentially leaking the victim's political orientation or viewing preferences). We implement LidarPhone on a Xiaomi Roborock vacuum cleaning robot and evaluate the feasibility of the attack through comprehensive real-world experiments. We use the prototype to collect both spoken digits and music played by a computer speaker and a TV soundbar, of more than 30k utterances totaling over 19 hours of recorded audio. LidarPhone achieves approximately 91% and 90% average accuracies of digit and music classifications, respectively.

Private conversations are an attractive target for malicious actors intending to conduct audio eavesdropping attacks. Previous works discovered unexpected vectors for these attacks, such as analyzing high-speed video of objects adjacent to sound sources, or using WiFi signal information. We propose LidarPhone, a novel side-channel attack that exploits the lidar sensors in commodity robot vacuum cleaners to perform acoustic eavesdropping attacks. LidarPhone is able to detect the minute vibrations induced on objects that are near audio sources, and extract meaningful signals from inherently noisy raw lidar returns. We evaluate a realistic scenario for potential victims: recovering privacy-sensitive digits (e.g., credit card numbers, social security numbers) emitted by computer speakers during teleconferencing calls. We implement LidarPhone on a Xiaomi Roborock vacuum cleaning robot and perform a comprehensive series of real-world experiments to determine its performance. LidarPhone achieves up to 91% accuracy for digit classification.