We present a novel retrofitting method to induce emotion aspects into pre-trained language models (PLMs) such as BERT and RoBERTa.Our method updates pre-trained network weights using contrastive learning so that the text fragments exhibiting similar emotions are encoded nearby in the representation space, and the fragments with different emotion content are pushed apart.While doing so, it also ensures that the linguistic knowledge already present in PLMs is not inadvertently perturbed.The language models retrofitted by our method, i.e., BERTEmo and RoBERTaEmo, produce emotion-aware text representations, as evaluated through different clustering and retrieval metrics.For the downstream tasks on sentiment analysis and sarcasm detection, they perform better than their pre-trained counterparts (about 1% improvement in F1-score) and other existing approaches.Additionally, a more significant boost in performance is observed for the retrofitted models over pretrained ones in few-shot learning setting.

Word embeddings learned using the distributional hypothesis (e.g., GloVe, Word2vec) are good at encoding various lexical-semantic relations. However, they do not capture the emotion aspects of words. We present a novel retrofitting method for updating the vectors of emotion bearing words like fun, offence, angry, etc. The retrofitted embeddings achieve better inter-cluster and intra-cluster distance for words having the same emotions, e.g., the joy cluster containing words like fun, happiness, etc., and the anger cluster with words like offence, rage, etc., as evaluated through different cluster quality metrics. For the downstream tasks on sentiment analysis and sarcasm detection, simple classification models, such as SVM and Attention Net, learned using our retrofitted embeddings perform better than their pre-trained counterparts (about 1.5 % improvement in F1-score) as well as other benchmarks. Furthermore, the difference in performance is more pronounced in the limited data setting.

Devices like solar cells, batteries etc. often comprise of a host of material types including organic, inorganic and hybrid materials. The fabrication procedures for these devices involve screening or designing the right set of materials and then subjecting them to a sequence of operations under very specific conditions. The performance characteristics of a device critically depend on the materials used in its fabrication, the specific operations carried out, their operating conditions and the specific sequence in which they are carried out. The space of potential materials, operations and operating conditions is vast, and selecting the right combination thereof to achieve the desired characteristics is a knowledge intensive activity. A large amount of such device fabrication knowledge is available in the form of publications, patents, company reports and so on. In this paper, we present a system that systematically extracts this knowledge from materials science literature. The extracted knowledge is represented as knowledge graphs conforming to an ontology that can be queried to make informed decisions in device fabrication procedures. The system first identifies the set of relevant paragraphs that contain fabrication knowledge. It then employs state of the art entity and relation extraction models to identify instances of operations, methods, materials, etc. and relations between them. The system then applies an unsupervised algorithm to identify sequences of operations representing fabrication procedures. We applied our system on solar cell fabrication knowledge extraction and achieved good performance. We believe our results provide much needed impetus for further work in this area.

We present a novel retrofitting model that can leverage relational knowledge available in a knowledge resource to improve word embeddings. The knowledge is captured in terms of relation inequality constraints that compare similarity of related and unrelated entities in the context of an anchor entity. These constraints are used as training data to learn a non-linear transformation function that maps original word vectors to a vector space respecting these constraints. The transformation function is learned in a similarity metric learning setting using Triplet network architecture. We applied our model to synonymy, antonymy and hypernymy relations in WordNet and observed large gains in performance over original distributional models as well as other retrofitting approaches on word similarity task and significant overall improvement on lexical entailment detection task.