Abstract There are many types of approaches for Paraphrase Identification (PI), an NLP task of determining whether a sentence pair has equivalent semantics. Traditional approaches mainly consist of unsupervised learning and feature engineering, which are computationally inexpensive. However, their task performance is moderate nowadays. To seek a method that can preserve the low computational costs of traditional approaches but yield better task performance, we take an investigation into neural network-based transfer learning approaches. We discover that by improving the usage of parameters efficiently for feature-based transfer, our research goal can be accomplished. Regarding the improvement, we propose a pre-trained task-specific architecture. The fixed parameters of the pre-trained architecture can be shared by multiple classifiers with small additional parameters. As a result, the computational cost left involving parameter update is only generated from classifier-tuning: the features output from the architecture combined with lexical overlap features are fed into a single classifier for tuning. Furthermore, the pre-trained task-specific architecture can be applied to natural language inference and semantic textual similarity tasks as well. Such technical novelty leads to slight consumption of computational and memory resources for each task and is also conducive to power-efficient continual learning. The experimental results show that our proposed method is competitive with adapter-BERT (a parameter-efficient fine-tuning approach) over some tasks while consuming only 16% trainable parameters and saving 69-96% time for parameter update.

In this paper we describe an extension of question answering dataset to be utilized for testing security export control expert system. Differently to the long questions and answers with rich context mostly describing exceptional interpretation of regulatory texts, our addition contains short questions and answers which were manually created by an expert who often answers similar questions from academic researchers. This dataset extension should allow broader and more realistic experiments for evaluating dialog-based expert systems with more realistic input. The dataset itself could be used broadly in other studies as a testbed for testing both text classification methods and QA systems requiring external data processing. We introduce the task, problems of previously used data, and describe how they are tackled with the additional set. Examples are presented together with term frequency analysis. Unlike the government-made QAs, the newly created dataset is freely available upon request.

In this paper, we report our initial findings from creating a question answering module for a dialog-based expert system which aims at advising users on export control regulations. We describe problems of data scarcity and knowledge transfer showing results of preliminary trials with utilizing contextual embeddings to extend keyword-based matching in our conversational expert system. We analyze pros and cons of the neural NLP techniques in systems which require precise information, for which no labelled data is available and which deal with less-resourced languages (Japanese in our case). Data scraped from governmental guidelines is presented and problems specific to legal documents in this area are discussed. Using examples from the preliminary experiments with deep learning-based methods we show when they fail and when they might be useful for further development of NLP systems operating on legal data. We also discuss evaluation-related problems and risks of imprecise output.

This paper presents a method of acquiring IsA assertions (hyponymy relations), AtLocation assertions (informing of location of objects) and LocatedNear assertions (informing of neigh boring locations) automatically from Japanese Wikipedia XML dump files. To extract IsA assertions, we use the Hyponymy extraction tool v1.0, which analyses definition, category and hierarchy structures of Wikipedia articles. The tool also produces information-rich taxonomy from which, using our original method, we can extract additional information, in this case AtLocation and LocatedNear type of assertions. Experiments showed that both methods produce positive results: we were able to acquire 5,866,680 IsA assertions with 99.0% reliability, 131,760 AtLocation assertion pairs with 93.0% reliability and 6,217 LocatedNear assertion pairs with 99.0% reliability. Our method exceeded the baseline system considering both precision and the number of acquired assertions.

This paper presents our research in detection of emotive (emotionally loaded) sentences. The task is defined as a text classification problem with an assumption that emotive sentences stand out both lexically and grammatically. The assumption is verified exper- imentally. The experiment is based on n-grams as well as more sophisticated patterns with disjointed elements. To deal with the sophisticated patterns a novel language modelling algorithm based on the idea of language combinatorics is applied. The results of experiments are explained with the standard means of Precision, Recall and balanced F-score. The algorithm also provides a refined list of most frequent sophisticated patterns typical for both emotive and non-emotive context.

The existence of social media has made computer-mediated communication more widespread among users around the world. This paper describes the development of an emoticon recommendation system that allows users to express their feelings with their input. In order to develop this system, an innovative emoticon database consisting of a table of emoticons with points expressed from each of 10 distinctive emotions was constructed. An evaluation experiment showed that 71.3% of user-selected emoticons were among the top 10 emoticons recommended by the proposed system.

In this paper we propose a method of filtering excessive amount of textual data acquired from the Internet. In our research on pun generation in Japanese we experienced problems with extensively long data processing time, caused by the amount of phonetic candidates generated (i.e. phrases that can be used to generate actual puns) by our system. Simple, naive approach in which we take into considerations only phrases with the highest occurrence in the Internet, can effect in deletion of those candidates that are actually usable. Thus, we propose a data filtering method in which we compare two Internet-based rankings: a co-occurrence ranking and a hit rate ranking, and select only candidates which occupy the same or similar positions in these rankings. In this work we analyze the effects of such data reduction, considering 1 cases: when the candidates are on exactly the same positions in both rankings, and when their positions differ by 1, 2, 3 and 4. The analysis is conducted on data acquired by comparing pun candidates generated by the system (and filtered with our method) with phrases that were actually used in puns created by humans. The results show that the proposed method can be used to filter excessive amounts of textual data acquired from the Internet.

This paper presents CAO, a system for affect analysis of emoticons. Emoticons are strings of symbols widely used in text-based online communication to convey emotions. It extracts emoticons from input and determines specific emotions they express. Firstly, by matching the extracted emoticons to a raw emoticon database, containing over ten thousand emoticon samples extracted from the Web and annotated automatically. The emoticons for which emotion types could not be determined using only this database, are automatically divided into semantic areas representing "mouths" or "eyes," based on the theory of kinesics. The areas are automatically annotated according to their co-occurrence in the database. The annotation is firstly based on the eye-mouth-eye triplet, and if no such triplet is found, all semantic areas are estimated separately. This provides the system coverage exceeding 3 million possibilities. The evaluation, performed on both training and test sets, confirmed the system's capability to sufficiently detect and extract any emoticon, analyze its semantic structure and estimate the potential emotion types expressed. The system achieved nearly ideal scores, outperforming existing emoticon analysis systems.

This paper presents CAO, a system for affect analysis of emoticons in Japanese online communication. Emoticons are strings of symbols widely used in text-based online communication to convey user emotions. The presented system extracts emoticons from input and determines the specific emotion types they express with a three-step procedure. First, it matches the extracted emoticons to a predetermined raw emoticon database. The database contains over 10,000 emoticon samples extracted from the Web and annotated automatically. The emoticons for which emotion types could not be determined using only this database, are automatically divided into semantic areas representing “mouths” or “eyes,” based on the idea of kinemes from the theory of kinesics. The areas are automatically annotated according to their co-occurrence in the database. The annotation is first based on the eye-mouth-eye triplet, and if no such triplet is found, all semantic areas are estimated separately. This provides hints about potential groups of expressed emotions, giving the system coverage exceeding 3 million possibilities. The evaluation, performed on both training and test sets, confirmed the system's capability to sufficiently detect and extract any emoticon, analyze its semantic structure, and estimate the potential emotion types expressed. The system achieved nearly ideal scores, outperforming existing emoticon analysis systems.

By our demonstration we want to introduce our achievements in combining different purpose algorithms to build a chatbot which is able to keep a conversation on any topic. It uses snippets of Internet search results to stay within a context, Nakamura's Emotion Dictionary to detect an emotional load existence and categorization of a textual utterance and a causal consequences retrieval algorithm when emotive features are not found. It is also able to detect a possibility to make a pun by analyzing the input sentence and create one if timing is adequate.

This paper introduces an early stage of a smart toy development project which combines several techniques to achieve a level of conversational skills and knowledge higher than currently available robots for children. We describe our ideas and achievements for three modules which we treat as the most important - topic unlimited talking engine, emotions recognizer and the moral behavior analyzer. We will also mention our novel evaluation method for freely speaking agents and possibilities of adding another module - an automatic joke generator.