Real-time audio streaming transmission and processing play a crucial role in time-sensitive applications such as food delivery services and ride-hailing platforms, where rapid response is essential. However, existing server-based audio streaming architectures struggle to handle the high concurrency of massive mobile devices efficiently. Traditional compression methods like MP3 and AAC offer limited compression ratios, while deep learning-based approaches often fail to meet the real-time transmission demands of edge computing environments. In this paper, we propose a novel edge-to-server audio streaming architecture that leverages Mel filter bank spectral features to achieve ultra-high compression efficiency. Our system integrates audio denoising, Mel feature extraction, and quantization-based compression at the edge, effectively suppressing environmental and device-induced noise while achieving an extreme compression ratio of 0.39% relative to the original uncompressed audio. Compared to conventional methods like MP3, our approach further reduces the file size by 96.1%. The decompressed Mel features remain task-independent, enabling seamless support for various general-purpose audio processing tasks in the server. We evaluate our system across three key audio tasks: speech recognition, speech emotion recognition, and audio classification. Extensive experiments on five different mobile devices demonstrate a 93.10% reduction in transmission latency at 1 Mbps bandwidth compared to 64 kbps MP3 audio, while maintaining task performance within a 5% deviation from state-of-the-art (SOTA) models across six mainstream audio datasets. These results highlight the efficiency, robustness, and scalability of our approach for real-time edge-to-server audio processing.

We introduce M 2 Silent, which enables multi-user silent speech interactions in shared spaces using multi-directional speakers.Ensuring privacy during interactions with voice-controlled systems presents significant challenges, particularly in environments with multiple individuals, such as libraries, offices, or vehicles.M 2 Silent addresses this by allowing users to communicate silently, without producing audible speech, using acoustic sensing integrated into directional speakers.We leverage FMCW signals as audio carriers, simultaneously playing audio and sensing the user's silent speech.

With the increase in motor vehicles, more convenient and accurate interactions are expected while retaining a high standard of safe driving. However, complex and dynamic vehicle environments challenge sensing tasks such as breathing monitor and hand gesture recognition. In this paper, we propose DASIV, which utilizes the highly directional nature of ultrasonic signals to achieve fine-grained directional acoustic sensing in vehicle environments. Due to air nonlinearity, the system enables synchronized directional acoustic communication to transmit information (e.g., navigation) to the driver without affecting other passengers. By optimizing the frequency of the Frequency Modulated Continuous Wave (FMCW) signals, DASIV avoids mutual interference between the sensing and communication signals and achieves breathing detection and hand gesture recognition for the driver. Specifically, the system extracts breathing-induced weak thoracic bullying through the signal phase, captures and analyses breathing patterns using bandpass and Gaussian filters, and develops a breathing model. Then, the system defines 10 interaction hand gestures to meet daily interaction needs, uses spectral features to mine complex and fast hand movement features, and proposes a hand gesture recognition model. Extensive experiments in real environments show that DASIV achieves high-precision breathing monitor (Pearson correlation coefficient of 0.89) and hand gesture recognition (Precision of 91.7%).

This paper introduces a novel multi-directional speaker, named MuDiS, which utilizes a parametric array to generate highly focused sound beams in multiple directions. The system capitalizes on air nonlinearity to reproduce sound from ultrasounds, successfully overcoming challenges inherent in traditional parametric arrays, such as transducer size and wavefront shape. It supports three important features simultaneously: independent beams, wide-angle digital steering, and unintended leakage suppression. To address these challenges, we designed a specialized cell structure that connects ultrasonic transducers, redirecting an approximately omnidirectional wavefront with optimal interspacing. An optimization-based algorithm is developed to minimize unintended leakages, and a nonlinear distortion reduction scheme is proposed to enhance sound quality. The paper showcases a prototype demonstrating the system's capabilities as a multidirectional speaker with a wide sound projection angle. Experimental results validate the effectiveness of our approach. The proposed multi-beam projection system rivals the performance of commercially available single-beam projection directional speakers, and improved steering angle and sound fidelity compared to multi-beamforming performance using traditional parametric arrays.

The convenient text input system is a pain point for devices such as AR glasses, and it is difficult for existing solutions to balance portability and efficiency. This paper introduces HandPad, the system that turns the hand into an on-the-go touchscreen, which realizes interaction on the hand via human capacitance. HandPad achieves keystroke and handwriting inputs for letters, numbers, and Chinese characters, reducing the dependency on capacitive or pressure sensor arrays. Specifically, the system verifies the feasibility of touch point localization on the hand using the human capacitance model and proposes a handwriting recognition system based on Bi-LSTM and ResNet. The transfer learning-based system only needs a small amount of training data to build a handwriting recognition model for the target user. Experiments in real environments verify the feasibility of HandPad for keystroke (accuracy of 100%) and handwriting recognition for letters (accuracy of 99.1%), numbers (accuracy of 97.6%) and Chinese characters (accuracy of 97.9%).