Low-resource language varieties used by specific groups remain neglected in the development of Multilingual Language Models.A great deal of cross-lingual research focuses on inter-lingual language transfer which strives to align allied varieties and minimize differences between them.However, for low-resource varieties, linguistic dissimilarity is also an important cue allowing generalization to unseen varieties.Unlike prior approaches, we propose a two-stage Language Generalization framework that focuses on capturing variety-specific cues while also exploiting rich overlap offered by high-resource source variety.First, we propose TOPPing, a source-selection method specifically designed for low-resource varieties.Second, we suggest a lightweight VAÇAÍ-Bowl architecture that learns variety-specific attributes with one branch while a parallel branch captures variety-invariant attributes using adversarial training.We evaluate our framework on structural prediction tasks, which are among the few tasks available, as proxy for performance on other downstream tasks.Using VAÇAÍ-Bowl with TOPPing yields an average 54.62% improvement in the dependency parsing task, which serves as a proxy for performance on other downstream tasks across 10 low-resource varieties.1 How can we transfer knowledge from source variety to target variety?Alignment Only variety-specific features are collapsed variety-specific features are preserved Variety Representation Variety-Aware Generalization Training Process

Existing zero-shot cross-lingual NER approaches require substantial prior knowledge of the target language, which is impractical for low-resource languages.In this paper, we propose a novel approach to NER using phonemic representation based on the International Phonetic Alphabet (IPA) to bridge the gap between representations of different languages.Our experiments show that our method significantly outperforms baseline models in extremely low-resource languages, with the highest average F1 score (46.38%) and lowest standard deviation (12.67), particularly demonstrating its robustness with non-Latin scripts.Our codes are available at https://github.com/Gabriel819/zeroshot_ner.git

Geewook Kim, Hodong Lee, Daehee Kim, Haeji Jung, Sanghee Park, Yoonsik Kim, Sangdoo Yun, Taeho Kil, Bado Lee, Seunghyun Park. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.