We probe pre-trained transformer language models for bridging inference.We first investigate individual attention heads in BERT and observe that attention heads at higher layers prominently focus on bridging relations incomparison with the lower and middle layers, also, few specific attention heads concentrate consistently on bridging.More importantly, we consider language models as a whole in our second approach where bridging anaphora resolution is formulated as a masked token prediction task (Of-Cloze test).Our formulation produces optimistic results without any finetuning, which indicates that pre-trained language models substantially capture bridging inference.Our further investigation shows that the distance between anaphor-antecedent and the context provided to language models play an important role in the inference.

We introduce a composite deep neural network architecture for supervised and language independent context sensitive lemmatization.The proposed method considers the task as to identify the correct edit tree representing the transformation between a word-lemma pair.To find the lemma of a surface word, we exploit two successive bidirectional gated recurrent structures -the first one is used to extract the character level dependencies and the next one captures the contextual information of the given word.The key advantages of our model compared to the state-of-the-art lemmatizers such as Lemming and Morfette are -(i) it is independent of human decided features (ii) except the gold lemma, no other expensive morphological attribute is required for joint learning.We evaluate the lemmatizer on nine languages -Bengali, Catalan, Dutch, Hindi, Hungarian, Italian, Latin, Romanian and Spanish.It is found that except Bengali, the proposed method outperforms Lemming and Morfette on the other languages.To train the model on Bengali, we develop a gold lemma annotated dataset 1 (having 1, 702 sentences with a total of 20, 257 word tokens), which is an additional contribution of this work.

This paper presents an approach for identifying reader specific difficult words while someone is reading a textual document. The work is motivated by the need of developing human-document interaction systems, in general and creating person-specific online educational content, in particular. Eye gaze information gives person specific behavior whereas textual content is analyzed to get general linguistic aspect of the document content. These two pieces of information are fused together through machine learning algorithms to identify the set of difficult words for a particular reader reading a particular document. An annotated dataset has been created where each word in a document is marked with its bounding box information and each reader identifies a set of difficult words while reading the document. The dataset consists of sixteen documents and each document is read by five subjects. The method is evaluated through recall-precision analysis. The impressive precision at high recall attests the feasibility of building a practical application based on this research. The experiment further brings out several interesting facts about human reading behaviour.