Conventional capacitive touchscreens in mobile devices often malfunction in the presence of moisture, dust, or gloved hands, leading to unreliable user interactions. To overcome these limitations, we present TouchAI, a multi-modal touch detection and localization system that leverages built-in inertial sensors and audio interfaces. Running as a background process, TouchAI constantly monitors inertial measurements for potential touch input and then activates acoustic sensing only upon detected events, thus reducing power consumption and privacy concerns associated with continuous audio recording. We introduce a novel event identification method that distinguishes touch-start and touch-end instances, providing precise timing for data sampling. For fine-grained touch localization, TouchAI employs a lightweight Transformer-based classification model to capture spatiotemporal features from combined acoustic-inertial signals. Experiments on commercial smartphones demonstrate true positive rates of 95.3% and 99.1% for touch detection and event identification, respectively, at false positive rates only less than 3%. Ultimately, TouchAI achieves up to 97.0% and 86.5% localization accuracy on 4×2 and 6×3 touch input grids, respectively, confirming its practicality and effectiveness as an alternative interface under touchscreen malfunction scenarios.

Acoustic communication using microphones and speakers of smart devices is one of the most spotlighted wireless technologies in recent years. In particular, chirp-based acoustic communication is widely adopted for smart device applications because of its robustness to frequency selectivity. Since many chirp-based acoustic applications run in the background on mobile devices, the power consumption of chirp-based acoustic communication is a critically important issue. Using energy detectors (EDs), which determine the existence of a valid acoustic signal based on energy level, applications can reduce power consumption by working only when a valid signal exists. However, conventional ED fails to distinguish between valid signals and high-energy noise in everyday life. In this paper, we propose No Entry, a novel ED for chirp-based acoustic communication systems. No Entry avoids not only high-energy noises but also a different modulation-based acoustic signal by utilizing the frequency sweeping characteristic of chirp signals. We implement prototype Android applications to evaluate the detection accuracy and power consumption. Compared with the state-of-the-art schemes, No Entry reduces energy consumption by 30% while achieving a greater detection performance.

In this paper, we propose a near-ultrasound chirp signal-based communication for the TV's 2nd screen services. While near-ultrasound (with under 20 kHz frequency) communication has been developed for various applications recently, none of the previous work provides a perfect solution for 2nd screen services between TVs and smart devices. This is due mainly to the following real world challenges. The embedded signal in TV contents should be successfully received in a typical TV-watching environment by (i) delivering information at least at 15 bps with significantly low volume to avoid human perception, (ii) despite the presence of ambient noise, e.g., a tick, a snap, or a knock. To fulfill (i), we design chirp quaternary orthogonal keying (QOK) symbols. Especially, we aim to minimize inter-symbol interference (ISI) effects by symbol design in order to completely eliminate guard intervals. To resolve (ii), we propose the novel J-shape detection algorithms for both frame synchronization and carrier sensing. The proposed modem achieves almost zero frame error rate on a smartphone 2.7 m away from the TV even with minimal receive sound pressure level of 35 dBSPL, i.e., the noise level in a very quiet room. Moreover, throughout experiments and log analysis of 2nd screen service deployed in a nation-wide TV broadcasting system, J-shape detection algorithms are proven to achieve highly resilient performance for both frame synchronization and carrier sensing compared to previous schemes.