Lütfi Kerem Senel, Besnik Fetahu, Davis Yoshida, Zhiyu Chen, Giuseppe Castellucci, Nikhita Vedula, Jason Ingyu Choi, Shervin Malmasi. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Lütfi Kerem Senel, Benedikt Ebing, Konul Baghirova, Hinrich Schuetze, Goran Glavaš. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Recent progress in pretraining language models on large corpora has resulted in large performance gains on many NLP tasks.These large models acquire linguistic knowledge during pretraining, which helps to improve performance on downstream tasks via fine-tuning.To assess what kind of knowledge is acquired, language models are commonly probed by querying them with 'fill in the blank' style cloze questions.Existing probing datasets mainly focus on knowledge about relations between words and entities.We introduce WDLMPro (Word Definition Language Model Probing) to evaluate word understanding directly using dictionary definitions of words.In our experiments, three popular pretrained language models struggle to match words and their definitions.This indicates that they understand many words poorly and that our new probing task is a difficult challenge that could help guide research on LMs in the future.

Ayyoob ImaniGooghari, Masoud Jalili Sabet, Lutfi Kerem Senel, Philipp Dufter, François Yvon, Hinrich Schütze. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.

Abstract As a ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words, but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related to a predefined concept to take larger values along a specified dimension, while leaving the original semantic learning mechanism mostly unaffected. In other words, we align words that are already determined to be related, along predefined concepts. Therefore, we impart interpretability to the word embedding by assigning meaning to its vector dimensions. The predefined concepts are derived from an external lexical resource, which in this paper is chosen as Roget’s Thesaurus. We observe that alignment along the chosen concepts is not limited to words in the thesaurus and extends to other related words as well. We quantify the extent of interpretability and assignment of meaning from our experimental results. Manual human evaluation results have also been presented to further verify that the proposed method increases interpretability. We also demonstrate the preservation of semantic coherence of the resulting vector space using word-analogy/word-similarity tests and a downstream task. These tests show that the interpretability-imparted word embeddings that are obtained by the proposed framework do not sacrifice performances in common benchmark tests.

A central limitation of multiple-acquisition magnetic resonance imaging (MRI) is the degradation in scan efficiency as the number of distinct datasets grows. Sparse recovery techniques can alleviate this limitation via randomly undersampled acquisitions. A frequent sampling strategy is to prescribe for each acquisition a different random pattern drawn from a common sampling density. However, naive random patterns often contain gaps or clusters across the acquisition dimension that, in turn, can degrade reconstruction quality or reduce scan efficiency. To address this problem, a statistically segregated sampling method is proposed for multiple-acquisition MRI. This method generates multiple patterns sequentially while adaptively modifying the sampling density to minimize k-space overlap across patterns. As a result, it improves incoherence across acquisitions while still maintaining similar sampling density across the radial dimension of k-space. Comprehensive simulations and in vivo results are presented for phase-cycled balanced steady-state free precession and multi-echo [Formula: see text]-weighted imaging. Segregated sampling achieves significantly improved quality in both Fourier and compressed-sensing reconstructions of multiple-acquisition datasets.

Cross-lingual studies attract a growing interest in natural language processing (NLP) research, and several studies showed that similar languages are more advantageous to work with than fundamentally different languages in transferring knowledge. Different similarity measures for the languages are proposed by researchers from different domains. However, a similarity measure focusing on semantic structures of languages can be useful for selecting pairs or groups of languages to work with, especially for the tasks requiring semantic knowledge such as sentiment analysis or word sense disambiguation. For this purpose, in this work, we leverage a recently proposed word embedding based method to generate a language similarity atlas for 76 different languages around the world. This atlas can help researchers select similar language pairs or groups in cross-lingual applications. Our findings suggest that semantic similarity between two languages is strongly correlated with the geographic proximity of the countries in which they are used.

Dense word embeddings, which encode meanings of words to low-dimensional vector spaces, have become very popular in natural language processing (NLP) research due to their state-of-the-art performances in many NLP tasks. Word embeddings are substantially successful in capturing semantic relations among words, so a meaningful semantic structure must be present in the respective vector spaces. However, in many cases, this semantic structure is broadly and heterogeneously distributed across the embedding dimensions making interpretation of dimensions a big challenge. In this study, we propose a statistical method to uncover the underlying latent semantic structure in the dense word embeddings. To perform our analysis, we introduce a new dataset (SEMCAT) that contains more than 6500 words semantically grouped under 110 categories. We further propose a method to quantify the interpretability of the word embeddings. The proposed method is a practical alternative to the classical word intrusion test that requires human intervention.