Misinformation spreads across media, community, and knowledge graphs in the Web by not only human agents but also information extraction algorithms that extract factual statements from unstructured textual data to populate the existing knowledge graphs.Traditional fact checking by experts or crowds is increasingly difficult to keep pace with the volume of newly created misinformation in the Web.Therefore, it is important and necessary to enhance the computational ability to determine whether a given factual statement is truthful or not.We view this problem as a truth scoring task in a knowledge graph.We present a novel rule-based approach that finds positive and negative evidential paths in a knowledge graph for a given factual statement, and calculates a truth score for the given statement by unsupervised ensemble of the found positive and negative evidential paths.For example, we can determine the factual statement "United States is the birth place of Barack Obama" as truthful if there is the positive evidential path (Barack Obama, birthPlace, Hawaii) ∧ (Hawaii, country, United States) in a knowledge graph.For another example, we can determine the factual statement "Canada is the nationality of Barack Obama" as untruthful if there is the negative evidential path (Barack Obama, nationality, United States) ∧ (United States, ≠ , Canada) in a knowledge graph.For evaluating on a real-world situation, we constructed an evaluation dataset by labeling truth or untruth label on factual statements that were extracted from Wikipedia texts by using the state-of-the-art BERT-based information extraction system.Our evaluation results show that our approach outperforms the state-of-the-art unsupervised approaches significantly by up to 0.12 AUC-ROC and even outperforms the supervised approach by up to 0.05 AUC-ROC not only in our dataset but also in the two different standard datasets.

Using current methods, the construction of multilingual resources in FrameNet is an expensive and complex task. While crowdsourcing is a viable alternative, it is difficult to include non-native English speakers in such efforts as they often have difficulty with English-based FrameNet tools. In this work, we investigated cross-lingual issues in crowdsourcing approaches for multilingual FrameNets, specifically in the context of the newly constructed Korean FrameNet. To accomplish this, we evaluated the effectiveness of various crowdsourcing settings whereby certain types of information are provided to workers, such as English definitions in FrameNet or translated definitions. We then evaluated whether the crowdsourced results accurately captured the meaning of frames both cross-culturally and cross-linguistically, and found that by allowing the crowd workers to make intuitive choices, they achieved a quality comparable to that of trained FrameNet experts (F1 > 0.75). The outcomes of this work are now publicly available as a new release of Korean FrameNet 1.1.

Information extraction from unstructured texts plays a vital role in the field of natural language processing. Although there has been extensive research into each information extraction task (i.e., entity linking, coreference resolution, and relation extraction), data are not available for a continuous and coherent evaluation of all information extraction tasks in a comprehensive framework. Given that each task is performed and evaluated with a different dataset, analyzing the effect of the previous task on the next task with a single dataset throughout the information extraction process is impossible. This paper aims to propose a Korean information extraction initiative point and promote research in this field by presenting crowdsourcing data collected for four information extraction tasks from the same corpus and the training and evaluation results for each task of a state-of-the-art model. These machine learning data for Korean information extraction are the first of their kind, and there are plans to continuously increase the data volume. The test results will serve as an initiative result for each Korean information extraction task and are expected to serve as a comparison target for various studies on Korean information extraction using the data collected in this study.

Distant supervision can automatically generate labeled data between a large-scale corpus and a knowledge base without utilizing human efforts.Therefore, many studies have used the distant supervision approach in relation extraction tasks.However, existing studies have a disadvantage in that they do not reflect the homograph in the word embedding used as an input of the relation extraction model.Thus, it can be seen that the relation extraction model learns without grasping the meaning of the word accurately.In this paper, we propose a relation extraction model with multi-sense word embedding.We learn multi-sense word embedding using a word sense disambiguation module.In addition, we use convolutional neural network and piecewise max pooling convolutional neural network relation extraction models that efficiently grasp key features in sentences.To evaluate the performance of the proposed model, two additional methods of word embedding were learned and compared.Accordingly, our method showed the highest performance among them.

Over the past years, several challenges and calls for research projects have pointed out the dire need for pushing natural language interfaces. In this context, the importance of Semantic Web data as a premier knowledge source is rapidly increasing. But we are still far from having accurate natural language interfaces that allow handling complex information needs in a user-centric and highly performant manner. The development of such interfaces requires collaboration of a range of different fields, including natural language processing, information extraction, knowledge base construction and population, reasoning, and question answering. With the goal to join forces in the collaborative development of natural language QA systems, the second OKBQA workshop is organized within the 40th SIGIR conference.

Although entity linking is a widely researched topic, the same cannot be said for entity linking geared for languages other than English.Several limitations including syntactic features and the relative lack of resources prevent typical approaches to entity linking to be used as effectively for other languages in general.We describe an entity linking system that leverage semantic relations between entities within an existing knowledge base to learn and perform entity linking using a minimal environment consisting of a part-of-speech tagger.We measure the performance of our system against Korean Wikipedia abstract snippets, using the Korean DBpedia knowledge base for training.Based on these results, we argue both the feasibility of our system and the possibility of extending to other domains and languages in general.

Emails are very popular method for information exchange between people. In this paper, an approach to annotate the starting time (stime) and ending time (etime) of duration in schedule notices is proposed. Most related works have reported on only seminar announcements, most of which contain only one schedule per announcement and are written in very restricted format. Different from those seminar announcements, an email frequently contains information about multiple schedules with highly complex format. To process the emails, the proposed system first detects and normalizes all time expressions of the email using regular expression patterns, and then determines which time expression actually represents stime and etime information of schedules. Evaluation is carried out on newly constructed Korean email corpus, and it shows 87.35 % of F1-score for stime and 85.13 % for etime.

In this paper, a new summarization system is proposed, which summarizes a document by interactively scoring the sentences using already-extracted summary so that the sen-tence which contains the most amount of relevant informa-tion to the summary to be extracted. To calculate the amount of relevant information contained by a sentence, the system makes heavy use of already existing taxonomies. The experiment shows a result pretty close to the current state-of-the-art systems.

Machine transliteration is a method for automatically converting words in one language into phonetically equivalent ones in another language. Machine transliteration plays an important role in natural language applications such as information retrieval and machine translation, especially for handling proper nouns and technical terms. Four machine transliteration models -- grapheme-based transliteration model, phoneme-based transliteration model, hybrid transliteration model, and correspondence-based transliteration model -- have been proposed by several researchers. To date, however, there has been little research on a framework in which multiple transliteration models can operate simultaneously. Furthermore, there has been no comparison of the four models within the same framework and using the same data. We addressed these problems by 1) modeling the four models within the same framework, 2) comparing them under the same conditions, and 3) developing a way to improve machine transliteration through this comparison. Our comparison showed that the hybrid and correspondence-based models were the most effective and that the four models can be used in a complementary manner to improve machine transliteration performance.

Machine transliteration is an automatic method for converting words in one language into phonetically equivalent ones in another language. There has been growing interest in the use of machine transliteration to assist machine translation and information retrieval. Three types of machine transliteration models---grapheme-based, phoneme-based, and hybrid---have been proposed. Surprisingly, there have been few reports of efforts to utilize the correspondence between source graphemes and source phonemes, although this correspondence plays an important role in machine transliteration. Furthermore, little work has been reported on ways to dynamically handle source graphemes and phonemes. In this paper, we propose a transliteration model that dynamically uses both graphemes and phonemes, particularly the correspondence between them. With this model, we have achieved better performance---improvements of about 15 to 41% in English-to-Korean transliteration and about 16 to 44% in English-to-Japanese transliteration---than has been reported for other models.

This paper describes KORTERM’s test suite and their practicability. The test-sets have been being constructed on the basis of fine-grained classification of linguistic phenomena to evaluate the technical status of English-to-Korean MT systems systematically. They consist of about 5000 test-sets and are growing. Each test-set contains an English sentence, a model Korean translation, a linguistic phenomenon category, and a yes/no question about the linguistic phenomenon. Two commercial systems were evaluated with a yes/no test of prepared questions. Total accuracy rates of the two systems were different (50% vs. 66%). In addition, a comprehension test was carried out. We found that one system was more comprehensible than the other system. These results seem to show that our test suite is practicable.

The syntactic structure of a nominal compound must be analyzed first for its semantic interpretation. In addition, the syntactic analysis of nominal compounds is very useful for NLP application such as information extraction, since a nominal compound often has a similar linguistic structure with a simple sentence, as well as representing concrete and compound meaning of an object with several nouns combined. In this paper, we present a novel model for structural analysis of nominal compounds using linguistic and statistical knowledge which is coupled based on lexical information. That is, the syntactic relations defined between nouns (complement-predicate and modifier-head relation) are obtained from large corpora and again used to analyze the structures of nominal compounds and identify the underlying relations between nouns. Experiments show that the model gives good results, and can be effectively used for application systems which do not require deep semantic information.

Word alignment problem between parallel corpora is based on the similar characteristics of two aligned words in two languages. We investigate linguistic-knowledge-based word similarity measures while other previous works heavily rely on statistical information, and their limits will be discussed. Linguistic knowledge is acquired from linguistic comparison of all layers between two languages, for Chinese and Korean in this paper.

We investigate a novel approach to solve the problem of sparse data through dimension reduction. Linear algebraic technique called LSA/SVD is used to find co-relationships of sparse words. Three variant estimation methods are suggested and they are evaluated for estimating unseen noun-verb co-occurrence probability. The model shows possibility to be alternative probability smoothing method.

Pattern-Based Machine Translation is one of the machine translation methods which performs syntactic analysis and structure transfer at the same time using bilingual patterns. PBMT is used to expand the length of patterns up to sentence-length in order to reduce ambiguities in translation, but it brought out the problem of rapidly increased patterns. We propose a model which shortens the length of patterns to phrase-length and reduces ambiguities in translation by using two level translation pattern selection method. In the first level, the proper translation patterns are selected by using a hybrid method of exact example matching and semantic constraint by thesaurus. In the second level, the most natural translation pattern for the verb phrase is selected among the selected translation pattern categories by using statistic information of the target language. By using this proposed model, we could shorten the length of patterns without raising the ambiguities in translation.

In recent years, statistical approaches on ATR (Automatic Term Recognition) have achieved good results. However, there are scopes to improve the performance in extracting terms still further. For example, domain dictionaries can improve the performance in ATR. This paper focuses on a method for extracting terms using a dictionary hierarchy. Our method produces relatively good results for this task.

This paper presents MATES/CK, a Chinese-to-Korean machine translation system. We introduce the design philosophy, component modules, implementation and some other aspects of MATES/CK system in this paper.

A probabilistic parameter reestimation algorithm plays a key role in the automatic acquisition of stochastic grammars. In the case of context-free phrase structure grammars, the inside-outside algorithm is widely used. However, it is not directly applicable to Probabilistic Dependency Grammar (PDG), because PDG is not based on constituents but on a head-dependent relation between pairs of words. This paper presents a reestimation algorithm which is a variation of the inside-outside algorithm adapted to probabilistic dependency grammar. The algorithm can be used either to reestimate the probabilistic parameters of an existing dependency grammar, or to extract a PDG from scratch. Using the algorithm, we have learned a PDG from a part-of-speech-tagged corpus of Korean, which showed about 62·82% dependency accuracy (the percentage of correct dependencies) for unseen test sentences.

In current multi-agent systems, the user is typically interacting with a single agent at a time through relatively inflexible and modestly intelligent interfaces. As a consequence, these systems force the users to submit simplistic requests only and suffer from problems such as the low-level nature of the system services offered to users, the weak reusability of agents, and the weak extensibility of the systems. In this paper, a framework for multi-agent systems called the open agent architecture (OAA) which reduces such problems, is discussed. The OAA is designed to handle complex requests that involve multiple agents. In some cases of complex requests from users, the components of the requests do not directly correspond to the capabilities of various application agents, and therefore, the system is required to translate the user's model of the task into the system's model before apportioning subtasks to the agents. To maximize users' efficiency in generating this type of complex requests, the OAA offers an intelligent multi-modal user interface agent which supports a natural language interface with a mix of spoken language, handwriting, and gesture. The effectiveness of the OAA environment including the intelligent distributed multi-modal interface has been observed in our development of several practical multi-agent systems.

Automaticthesaurus construction is accomplished by extracting term relations mechanically.A popular method uses statistical anaJysis to discover the term relations.For low frequency terms, however, the statistical information of the terms cannot be reliably used for deciding the relationship of terms.This problem is generally referred as data sparseness problem.Unfortunately, many studies have been said that the low frequency terms are of most use in information retrieval.In this paper, we formalize the statistical behaviors of terms by using inference network.A formal approach for the data sparseness problem which is crucial in constructing a thesaurus is developed.The validity of this approach is shown by experiments.

An efficient way of automatic link generation for a high quality hypertext is described in this article. The syntactic information is not enough to select the correct destination node for a button in a hypertext. The information besides syntactic information such as semantic information and even background knowledge are indispensable. In order to utilize the information, a partner concept table is used along with the ideas of canonical graph and conformity relation in conceptual graph theory. A conceptual graph representation form called U-Form is presented here for efficient processing of conceptual graphs in a machine.

Abstract Inside parsing is a best parse parsing method based on the Inside algorithm that is often used in estimating probabilistic parameters of stochastic context free grammars. It gives a best parse in O(N3G3) time where N is the input size and G is the grammar size. Earley algorithm can be made to return best parses with the same complexity in N. By way of experiments, we show that Inside parsing can be more efficient than Earley parsing with sufficiently large grammar and sufficiently short input sentences. For instance, Inside parsing is better with sentences of 16 or less words for a grammar containing 429 states. In practice, parsing can be made efficient by employing the two methods selectively. The redundancy of Inside algorithm can be reduced by the topdown filtering using the chart produced by Earley algorithm, which is useful in training the probabilistic parameters of a grammar. Extensive experiments on Penn Tree corpus show that the efficiency of Inside computation can be improved by up to 55%.

In this paper, we address there important issues with query expansion using a thesaurus; how to give weights to the terms in expanded queries, how to select additional search terms in the thesaurus, and how to enrich the terms in the manual thesaurus (namely, thesaurus reconstruction). To weight the terms in expanded queries, we construct the weighted thesaurus that has a similarity value between the terms in the thesaurus, using statistical co-occurrence in a corpus. To enrich the terms in the manual thesaurus, domain dependent terms which occur in a corpus are inserted into the weighted thesaurus using the co-occurrence information. In this paper, the reconstructed thesaurus with weights is defined as a domain-adapted, weighted thesaurus. Then we explain query expansion using the domain-adapted, weighted thesaurus in an extended Boolean retrieval model. To select additional search terms during query expansion, our model uses semi-automatic query expansion and a restriction method. In the experiments, our system had almost twice the recall of the boolean retrieval system not using the thesaurus or the query expansion retrieval system using the original thesaurus. And also, the precision of our system was almost the same precision as the other systems.

This paper aims at developing a knowledge-based information retrieval system that allows a naive user to retrieve relevant documents effectively, without any help

A technique is proposed for specifying universal quantification and existential quantification (combined with negation) in a two-dimensional (graphical) database query language. Unlike other approaches that provide set operators to simulate universal quantification, this technique allows a direct representation of universal quantification. Syntactic constructs for specifying universal and existential quantifications, two-dimensional translation of universal quantification to existential quantification (with negation), and translation of existentially quantified two-dimensional queries to relational queries are presented. The resulting relational queries can be processed directly by many existing database systems. The authors claim that this technique renders universal quantifications easy to understand. To substantiate this claim, they provide a simple, easy-to-follow guideline for constructing universally quantified queries.>