Robust speech recognition systems rely on cloud service providers for inference. It needs to ensure that an untrustworthy provider cannot deduce the sensitive content in speech. Sanitization can be done on speech content keeping in mind that it has to avoid compromising transcription accuracy. Realizing the under-utilized capabilities of tiny speech foundation models (FMs), for the first time, we propose a novel use: enhancing speech privacy on resource-constrained devices. We introduce SpeechShield, an edge/cloud privacy preserving speech inference engine that can filter sensitive entities without compromising transcript accuracy. We utilize a timestamp based on-device masking approach that utilizes a token to entity prediction model to filter sensitive entities. Our choice of mask strategically conceals parts of the input and hides sensitive data. The masked input is sent to a trusted cloud service or to a local hub to generate the masked output. The effectiveness of SpeechShield hinges on how well the entity time segments are masked. Our recovery is a confidence score based approach that chooses the best prediction between the cloud and the on-device model.

Proto is a new instructional OS that runs on commodity, portable hardware. It showcases modern features, including per-app address spaces, threading, commodity filesystems, USB, DMA, multicore support, self-hosted debugging, and a window manager. It supports rich applications such as 2D/3D games, music and video players, and a blockchain miner. Unlike traditional instructional systems, Proto emphasizes engaging, media-rich apps that go beyond basic terminal programs. Our method breaks down a full-featured OS into a set of incremental, self-contained prototypes. Each prototype introduces a minimal set of OS mechanisms, driven by the needs of specific apps. The construction process then progressively enables these apps by bringing up one mechanism at a time.

Voice assistive technologies have given rise to extensive privacy and security concerns. In this paper we investigate whether robust automatic speech recognition (ASR) can be done in a privacy preserving manner on resource constrained devices. For inference, speech recognition systems rely on cloud service providers that host state of the art (SOTA) foundation models (FM) demanding high memory usage and computation complexity. Considering the infeasibility to run these SOTA FM on-device, users are forced to offload their sensitive content to cloud for better inference compromising privacy. Sanitization can be done on speech content at the edge keeping in mind that it has to avoid compromising the ASR accuracy. Our solution is based on the observation that although tiny on-device FM cannot perform robust ASR, they are powerful enough to mask the sensitive segments in speech. Given this observation, we introduce SpeechGuard - a privacy-preserving, hybrid edge-cloud ASR inference engine that is compute inexpensive. We present a novel times-tamp based entity filtering mechanism at the edge and a corresponding confidence score based recovery approach. Our results denote that SpeechGuard offers ASR accuracy and privacy guarantee comparable to existing SOTA FM, while achieving significant speedup and reduction in memory usage.

This paper addresses spoken language understanding (SLU) on microcontroller-like embedded devices, integrating on-device execution with cloud offloading in a novel fashion. We leverage temporal locality in the speech inputs to a device and reuse recent SLU inferences accordingly. Our idea is simple: let the device match incoming inputs against cached results, and only offload inputs not matched to any cached ones to the cloud for full inference. Realization of this idea, however, is non-trivial: the device needs to compare acoustic features in a robust yet low-cost way.