Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, Jimmy Huang. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Sara Shatnawi, Sawsan Alqahtani, Hanan Aldarmaki. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Pre-trained language models (PLM) have advanced the state-of-the-art across NLP applications, but lack domain-specific knowledge that does not naturally occur in pre-training data.Previous studies augmented PLMs with symbolic knowledge for different downstream NLP tasks.However, knowledge bases (KBs) utilized in these studies are usually large-scale and static, in contrast to small, domain-specific, and modifiable knowledge bases that are prominent in real-world task-oriented dialogue (TOD) systems.In this paper, we showcase the advantages of injecting domain-specific knowledge prior to fine-tuning on TOD tasks.To this end, we utilize light-weight adapters that can be easily integrated with PLMs and serve as a repository for facts learned from different KBs.To measure the efficacy of proposed knowledge injection methods, we introduce Knowledge Probing using Response Selection (KPRS) -a probe designed specifically for TOD models.Experiments 1 on KPRS and the response generation task show improvements of knowledge injection with adapters over strong baselines. * Work performed while at AWS AI Labs 1 https://github.com/amazon-research/ domain-knowledge-injection

In many languages like Arabic, diacritics are used to specify pronunciations as well as meanings. Such diacritics are often omitted in written text, increasing the number of possible pronunciations and meanings for a word. This results in a more ambiguous text making computational processing on such text more difficult. Diacritic restoration is the task of restoring missing diacritics in the written text. Most state-of-the-art diacritic restoration models are built on character level information which helps generalize the model to unseen data, but presumably lose useful information at the word level. Thus, to compensate for this loss, we investigate the use of multi-task learning to jointly optimize diacritic restoration with related NLP problems namely word segmentation, part-of-speech tagging, and syntactic diacritization. We use Arabic as a case study since it has sufficient data resources for tasks that we consider in our joint modeling. Our joint models significantly outperform the baselines and are comparable to the state-of-the-art models that are more complex relying on morphological analyzers and/or a lot more data (e.g. dialectal data).

Sawsan Alqahtani, Ajay Mishra, Mona Diab. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Diacritic restoration is the task of assigning diacritics (accents) for each character in a given segment. The typical input levels that have been previously used in diacritic restoration models are word and/or character units. In this paper, we investigate the use of subwords as input units along with their diacritic patterns (combinations of adjacent diacritics) as output, as an alternative to word or character-based models. Our experiments show that characters provide the optimal level of information for sequence-based diacritic restoration models across different languages. We additionally improved our diacritic restoration model by maximizing over the output diacritic sequence using a Conditional Random Field (CRF). Adding a CRF layer improves the performance on observed and unobserved words substantially for Arabic and marginally for Yoruba.