We propose a lightweight neural front-end framework for on-device speech generation and highlight its benefits towards low-resource language scaling. While data-driven models have shown potential in front-end literature, especially since they can enable fast language expansion, they are often extremely large and of high latency. There is limited work focusing on their usability in real-time settings, and none for on-device TTS applications. At the cost of small performances trade-offs, we build lightweight neural Grapheme-to-Phoneme and verbalization models which achieve, on average across three languages, a p90 latency reduction of 95.98% per token on single-threaded [email protected], with respect to a traditional transformer-based baseline, while having 99.26% less parameters. Additionally, leveraging pre-trained teacher models to bootstrap lightweight students, we enable low-resource language scaling on both Grapheme-to-Phoneme conversion and verbalization.

Recently proposed pre-trained language models can be easily fine-tuned to a wide range of downstream tasks. However, fine-tuning requires a large training set. This PhD project introduces novel natural language processing (NLP) use cases in the healthcare domain where obtaining a large training dataset is difficult and expensive. To this end, we propose data-efficient algorithms to fine-tune NLP models in low-resource settings and validate their effectiveness. We expect the outcomes of this PhD project could contribute to the NLP research and low-resource application domains.