Liqiang Xiao, Jun Ma, Xin Luna Dong, Pascual Martínez-Gómez, Nasser Zalmout, Wei Chen, Tong Zhao, Hao He, Yaohui Jin. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.

Sequence-to-sequence models have shown strong performance in a wide range of NLP tasks, yet their applications to sentence generation from logical representations are underdeveloped.In this paper, we present a sequence-to-sequence model for generating sentences from logical meaning representations based on event semantics.We use a semantic parsing system based on Combinatory Categorial Grammar (CCG) to obtain data annotated with logical formulas.We augment our sequence-to-sequence model with masking for predicates to constrain output sentences.We also propose a novel evaluation method for generation using Recognizing Textual Entailment (RTE).Combining parsing and generation, we test whether or not the output sentence entails the original text and vice versa.Experiments showed that our model outperformed a baseline with respect to both BLEU scores and accuracies in RTE.

Hitomi Yanaka, Koji Mineshima, Pascual Martínez-Gómez, Daisuke Bekki. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

Pascual Martínez-Gómez, Koji Mineshima, Yusuke Miyao, Daisuke Bekki. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017.

In the logic approach to Recognizing Textual Entailment, identifying phrase-tophrase semantic relations is still an unsolved problem.Resources such as the Paraphrase Database offer limited coverage despite their large size whereas unsupervised distributional models of meaning often fail to recognize phrasal entailments.We propose to map phrases to their visual denotations and compare their meaning in terms of their images.We show that our approach is effective in the task of Recognizing Textual Entailment when combined with specific linguistic and logic features.

Determining semantic textual similarity is a core research subject in natural language processing.Since vector-based models for sentence representation often use shallow information, capturing accurate semantics is difficult.By contrast, logical semantic representations capture deeper levels of sentence semantics, but their symbolic nature does not offer graded notions of textual similarity.We propose a method for determining semantic textual similarity by combining shallow features with features extracted from natural deduction proofs of bidirectional entailment relations between sentence pairs.For the natural deduction proofs, we use ccg2lambda, a higherorder automatic inference system, which converts Combinatory Categorial Grammar (CCG) derivation trees into semantic representations and conducts natural deduction proofs.Experiments show that our system was able to outperform other logicbased systems and that features derived from the proofs are effective for learning textual similarity.

This paper presents a system that compositionally maps outputs of a wide-coverage Japanese CCG parser onto semantic representations and performs automated inference in higher-order logic.The system is evaluated on a textual entailment dataset.It is shown that the system solves inference problems that focus on a variety of complex linguistic phenomena, including those that are difficult to represent in the standard first-order logic.

FixFix is a web-based tool for editing reading gaze fixation datasets. The purpose is to provide gaze researchers focusing on reading an easy-to-use interface that will facilitate manual interpretation, but even more so to create gold standard datasets for machine learning and data mining. It allows the users to identify fixations, then move them either singly or in groups, in order to correct both variable and systematic gaze sampling errors.

We present a higher-order inference system based on a formal compositional semantics and the wide-coverage CCG parser.We develop an improved method to bridge between the parser and semantic composition.The system is evaluated on the FraCaS test suite.In contrast to the widely held view that higher-order logic is unsuitable for efficient logical inferences, the results show that a system based on a reasonably-sized semantic lexicon and a manageable number of non-first-order axioms enables efficient logical inferences, including those concerned with generalized quantifiers and intensional operators, and outperforms the state-of-the-art firstorder inference system.

Professional human translation is necessary to meet high quality standards in industry and governmental agencies. Translators engage in multiple activities during their task, and there is a need to model their behavior, with the objective to understand and optimize the translation process. In recent years, user interfaces enabled us to record user events such as eye-movements or keystrokes. Although there have been insightful descriptive analysis of the translation process, there are multiple advantages in enabling quantitative inference. We present methods to classify sequences of fixations and keystrokes into activities and model translation sessions with the objective to recognize translator expertise. We show significant error reductions in the task of recognizing certified translators and their years of experience, and analyze the characterizing patterns.

The reading act is an intimate and elusive process that is important to understand. Psycholinguists have long studied the effects of task, personal or document characteristics on reading behavior. An essential factor in the success of those studies lies in the capability of analyzing eye-movements. These studies aim to recognize causal effects on patterns of eye-movements, by contriving variations in task, personal or document characteristics. In this work, we follow the opposite direction. We present a formal framework to recognize reader's level of understanding and language skill given measurements of reading behavior via eye-gaze data. We show significant error reductions to recognize these attributes and provide a detailed study of the most discriminative features.

Eye-tracking devices find applications in human-machine interaction, hypothesis testing in psycholinguistic and usability studies, relevant feature extraction when designing models related to human behavior and to build user-centered information systems. We aim at providing a general and robust framework to do quantitative analysis and inference using data collected by eye-trackers when users read text. To achieve this objective, first the accuracy of eye-trackers has to be increased beyond sensor capabilities by using information from the content or the structure of the text. Then, natural language processing techniques will be used to process text appearing on the screen and the recognized reading word sequence. Within this framework, it will be possible to better understand user's intentions, record knowledge acquisition and predict information needs. The intention is to build a user model and user model of the World from texts that users have read. This opens the door to more personalized systems with on-line adaptation capabilities.

Applications using eye-tracking devices need a higher accuracy in recognition when the task reaches a certain complexity. Thus, more sophisticated methods to correct eye-tracking measurement errors are necessary to lower the penetration barrier of eye-trackers in unconstrained tasks. We propose to take advantage of the content or the structure of textual information displayed on the screen to build informed error-correction algorithms that generalize well. The idea is to use feature-based image registration techniques to perform a linear transformation of gaze coordinates to find a good alignment with text printed on the screen. In order to estimate the parameters of the linear transformation, three optimization strategies are proposed to avoid the problem of local minima, namely Monte Carlo, multi-resolution and multi-blur optimization. Experimental results show that a more precise alignment of gaze data with words on the screen can be achieved by using these methods, allowing a more reliable use of eye-trackers in complex and unconstrained tasks.

Interactive machine translation (IMT) is an increasingly popular paradigm for semi-automated machine translation, where a human expert is integrated into the core of an automatic machine translation system. The human expert interacts with the IMT system by partially correcting the errors of the system's output. Then, the system proposes a new solution. This process is repeated until the output meets the desired quality. In this scenario, the interaction is typically performed using the keyboard and the mouse. However, speech is also a very interesting input modality since the user does not need to abandon the keyboard to interact with it.