Lexical substitution is the task of generating meaningful substitutes for a word in a given textual context.Contextual word embedding models have achieved state-of-the-art results in the lexical substitution task by relying on contextual information extracted from the replaced word within the sentence.However, such models do not take into account structured knowledge that exists in external lexical databases.We introduce LexSubCon, an end-to-end lexical substitution framework based on contextual embedding models that can identify highly-accurate substitute candidates.This is achieved by combining contextual information with knowledge from structured lexical resources.Our approach involves: (i) introducing a novel mix-up embedding strategy to the target word's embedding through linearly interpolating the pair of the target input embedding and the average embedding of its probable synonyms; (ii) considering the similarity of the sentence-definition embeddings of the target word and its proposed candidates; and, (iii) calculating the effect of each substitution on the semantics of the sentence through a fine-tuned sentence similarity model.Our experiments show that LexSubCon outperforms previous state-of-the-art methods by at least 2% over all the official lexical substitution metrics on LS07 and CoInCo benchmark datasets that are widely used for lexical substitution tasks.

George Michalopoulos, Yuanxin Wang, Hussam Kaka, Helen Chen, Alexander Wong. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Peritoneal dialysis (PD) removes waste products from blood when the kidney is malfunctioned. Since there is no clear criterion for PD recommendation for patients with kidney disease, existing machine learning models (ML), which rely on credible decision criterion, are ineffective in making PD eligibility decisions, especially when the correlated traits or indicators (patterns) inherent in the PD data are diverse and subtle. Furthermore, the lack of interpretable transparency in traditional ML also weakens the credibility of the decision they produce. Hence, an in-depth knowledge of the patients' characteristics is needed to render a clearer picture of the decision-making process and model to detect the rare PD eligibility cases. In this paper, we extend our previous work (Attribute-Value-Association Discovery and Disentanglement (ADD)), to an extended ADD for PD data analysis (PD-ADD) to overcome these problems. We show that PD-ADD is able to discover association patterns of patient profiles and symptoms to reveal PD characteristics and detect eligible rare cases. Experimental results show that PDADD is much superior to existing unsupervised clustering (with accuracy of 89.87% vs 73.37% of K-Means). It also enables straightforward interpretation of the underlying relations of patient characteristics in an unsupervised setting.