This paper empirically studies whether BERT can really learn to conduct natural language inference (NLI) without utilizing hidden dataset bias; and how efficiently it can learn if it could.This is done via creating a simple entailment judgment case which involves only binary predicates in plain English.The results show that the learning process of BERT is very slow.However, the efficiency of learning can be greatly improved (data reduction by a factor of 1,500) if task-related features are added.This suggests that domain knowledge greatly helps when conducting NLI with neural networks.

We present ASDiv (Academia Sinica Diverse MWP Dataset), a diverse (in terms of both language patterns and problem types) English math word problem (MWP) corpus for evaluating the capability of various MWP solvers.Existing MWP corpora for studying AI progress remain limited either in language usage patterns or in problem types.We thus present a new English MWP corpus with 2,305 MWPs that cover more text patterns and most problem types taught in elementary school.Each MWP is annotated with its problem type and grade level (for indicating the level of difficulty).Furthermore, we propose a metric to measure the lexicon usage diversity of a given MWP corpus, and demonstrate that ASDiv is more diverse than existing corpora.Experiments show that our proposed corpus reflects the true capability of MWP solvers more faithfully.

This article proposes to conduct natural language inference with novel Enhanced-Relation-Head-Dependent triplets (RHD triplets) , which are constructed via enhancing each word in the RHD triplet with its associated local context. Most previous approaches based on deep neural network (DNN) for this task either perform token alignment without considering syntactic dependency among words, or directly use tree- LSTM to generate passage representation with irrelevant information. To improve token alignment and inference judgment with word-pair-dependency, the RHD triplet structure is first proposed. To avoid incorporating irrelevant information, this proposed approach performs comparison directly on each triplet-pair of the given passage-pair (instead of comparing each triplet in a passage with the content merged from the whole opposite passage). Furthermore, to take local context into consideration while conducting token alignment and inference judgment, we also enhance the words of the triplets with their associated local context to improve the performance. Experimental results show that the proposed approach is better than most previous approaches that adopt tree structures, and its performance is comparable to other state-of-the-art approaches (however, our approach is more human comprehensible).

This paper proposes to perform natural language inference with Word-Pair-Dependency-Triplets. Most previous DNN-based approaches either ignore syntactic dependency among words, or directly use tree-LSTM to generate sentence representation with irrelevant information. To overcome the problems mentioned above, we adopt Word-Pair-Dependency-Triplets to improve alignment and inference judgment. To be specific, instead of comparing each triplet from one passage with the merged information of another passage, we first propose to perform comparison directly between the triplets of the given passage-pair to make the judgement more interpretable. Experimental results show that the performance of our approach is better than most of the approaches that use tree structures, and is comparable to other state-of-the-art approaches.

Chao-Chun Liang, Yu-Shiang Wong, Yi-Chung Lin, Keh-Yih Su. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

A goal-oriented meaning-based statistical framework is presented in this paper to solve the math word problem that requires multiple arithmetic operations with understanding, reasoning and explanation. It first analyzes and transforms sentences into their meaning-based logical forms, which represent the associated context of each quantity with role-tags (e.g., nsubj, verb, etc.). Logic forms with role-tags provide a flexible and simple way to specify the physical meaning of a quantity. Afterwards, the main-goal of the problem is decomposed recursively into its associated sub-goals. For each given sub-goal, the associated operator and operands are selected with statistical models. Lastly, it performs inference on logic expressions to get the answer and explains how the answer is obtained in a human comprehensible way. This process thus resembles the human cognitive understanding of the problem and produces a more meaningful problem solving interpretation.

This article proposes a unified, character-based, generative model to incorporate additional resources for solving the out-of-vocabulary (OOV) problem of Chinese word segmentation, within which different types of additional information can be utilized independently in corresponding submodels. This article mainly addresses the following three types of OOV: unseen dictionary words, named entities, and suffix-derived words, none of which are handled well by current approaches. The results show that our approach can effectively improve the performance of the first two types with positive interaction in F-score. Additionally, we also analyze reason that suffix information is not helpful. After integrating the proposed generative model with the corresponding discriminative approach, our evaluation on various corpora---including SIGHAN-2005, CIPS-SIGHAN-2010, and the Chinese Treebank (CTB)---shows that our integrated approach achieves the best performance reported in the literature on all testing sets when additional information and resources are allowed.

In this article, an integrated model is derived that jointly identifies and aligns bilingual named entities (NEs) between Chinese and English. The model is motivated by the following observations: (1) whether an NE is translated semantically or phonetically depends greatly on its entity type, (2) entities within an aligned pair should share the same type, and (3) the initially detected NEs can act as anchors and provide further information while selecting NE candidates. Based on these observations, this article proposes a translation mode ratio feature (defined as the proportion of NE internal tokens that are semantically translated), enforces an entity type consistency constraint, and utilizes additional new NE likelihoods (based on the initially detected NE anchors). Experiments show that this novel method significantly outperforms the baseline. The type-insensitive F-score of identified NE pairs increases from 78.4% to 88.0% (12.2% relative improvement) in our Chinese–English NE alignment task, and the type-sensitive F-score increases from 68.4% to 83.0% (21.3% relative improvement). Furthermore, the proposed model demonstrates its robustness when it is tested across different domains. Finally, when semi-supervised learning is conducted to train the adopted English NE recognition model, the proposed model also significantly boosts the English NE recognition type-sensitive F-score.

Among statistical approaches to Chinese word segmentation, the word-based n-gram ( generative ) model and the character-based tagging ( discriminative ) model are two dominant approaches in the literature. The former gives excellent performance for the in-vocabulary (IV) words; however, it handles out-of-vocabulary (OOV) words poorly. On the other hand, though the latter is more robust for OOV words, it fails to deliver satisfactory performance for IV words. These two approaches behave differently due to the unit they use (word vs. character) and the model form they adopt (generative vs. discriminative). In general, character-based approaches are more robust than word-based ones, as the vocabulary of characters is a closed set; and discriminative models are more robust than generative ones, since they can flexibly include all kinds of available information, such as future context. This article first proposes a character-based n -gram model to enhance the robustness of the generative approach. Then the proposed generative model is further integrated with the character-based discriminative model to take advantage of both approaches. Our experiments show that this integrated approach outperforms all the existing approaches reported in the literature. Afterwards, a complete and detailed error analysis is conducted. Since a significant portion of the critical errors is related to numerical/foreign strings, character-type information is then incorporated into the model to further improve its performance. Last, the proposed integrated approach is tested on cross-domain corpora, and a semi-supervised domain adaptation algorithm is proposed and shown to be effective in our experiments.

In this paper, the major problems of the current machine translation systems are first outlined. A new direction, highlighting the system capability to be customizable and self-learnable, is then proposed for attacking the described problems, which are mainly resulted from the very complicated characteristics of natural languages. The proposed solution adopts an unsupervised two-way training mechanism and a parameterized architecture to acquire the required statistical knowledge, such that the system can be easily adapted to different domains and various preferences of individual users.

A joint learning algorithm is proposed in this paper to enhance the integrated speech and language model operating in the character-synchronous mode. A character-synchronous approach is first proposed to integrate speech and language information, including morphology and parts-of-speech, for ranking the candidates right after each Chinese character is uttered. Since the search space is cut down very efficiently by applying high-level knowledge in early time, the character-synchronous score function enables our system to operate in real time. To further enhance system performance, a joint learning algorithm is then derived to adjust all parameters of the speech and language processing modules simultaneously, according to their contributions to the overall discrimination power, to minimize the error rate. The proposed approaches are compared with a baseline system, which directly couples an HMM-based speech recognizer with a bigram language model. The performance of 75.71% character accuracy rate is obtained for the baseline system when it is tested on the task of recognizing 1000 Chinese spoken sentences with a very large vocabulary (90495 words) in the speaker-independent isolated-character mode. The character accuracy rate is improved to 88.26% with the character-synchronous approach, and 94.16% after the joint learning algorithm is applied.

The two-dimensional pattern matching problem finds the occurrences of a given two-dimensional pattern array in a two-dimensional text array. The paper presents two efficient algorithms, which combine Fan-Su (1993) and Aho-Corasick (1979) string search algorithms, to solve this problem. The proposed algorithms need not inspect each character of the text array during the pattern matching in most cases. Additionally, unlike the algorithms proposed by Zhu and Takaoka (1989) which are based on the hashing method, these new algorithms require no preprocessing of the text array. A comparison of the performance of various algorithms is presented. The result shows that the proposed algorithms are about three to six times faster than the best algorithm proposed previously when the size of the pattern array is less than 1/100 of the size of the text array, which occurs frequently in many applications.>

An automatic compound retrieval method is proposed to extract compounds within a text message. It uses n-gram mutual information, relative frequency count and parts of speech as the features for compound extraction. The problem is modeled as a two-class classification problem based on the distributional characteristics of n-gram tokens in the compound and the non-compound clusters. The recall and precision using the proposed approach are 96.2% and 48.2% for bigram compounds and 96.6% and 39.6% for trigram compounds for a testing corpus of 49,314 words. A significant cutdown in processing time has been observed.

An automatic treebank conversion method is proposed in this paper to convert a treebank into another treebank. A new treebank associated with a different grammar can be generated automatically from the old one such that the information in the original treebank can be transformed to the new one and be shared among different research communities. The simple algorithm achieves conversion accuracy of 96.4% when tested on 8,867 sentences between two major grammar revisions of a large MT system.

A joint learning algorithm which enabled the parameters in an integrated speech and language model to be trained jointly, was proposed in this paper. The integrated model enhanced the spoken language system with high level knowledge, and operated in a character-synchronous mode. This integration model was tested on the task of recognizing isolated Chinese characters in the speaker independent mode with very large vocabulary of 90,495 words, and the performance of 88.26% character accuracy rate was obtained. In contrast, only 75.71% accuracy rate was achieved with the baseline system, which directly coupled the speech recognizer with a character bi-gram language module. Afterwards, the parameters of both speech and language modules were jointly adjusted according to their contribution in discrimination. The dynamic range variations among the parameters in different modules were also well tuned during the learning processes. After applying this procedure to the character-synchronous integration model, a very promising result of 94.16% character accuracy (75.96% error reduction rate) was obtained.>

A weighted hidden Markov model (HMM) algorithm and a subspace projection algorithm are proposed to address the discrimination and robustness issues for HMM-based speech recognition. A robust two-stage classifier is also proposed to incorporate these two approaches to further improve the performance. The weighted HMM enhances its discrimination power by first jointly considering the state likelihoods of different word models, then assigning a weight to the likelihood of each state, according to its contribution in discriminating words. The robustness of this model is then improved by increasing the likelihood difference between the top and the second candidates. The subspace projection approach discards unreliable observations on the basis of maximizing the divergence between different word pairs. To improve robustness, the mean of each cluster is then adjusted to obtain maximum separation different clusters. The performance was evaluated with a highly confusable vocabulary consisting of the nine English E-set words. The test was conducted in a multispeaker (100 talkers), isolated-word mode. The 61.7% word accuracy for the original HMM-based system was improved to 74.9% and 76.6%, respectively, by using the weighted HMM and the subspace projection methods. By incorporating the weighted HMM in the first stage and the subspace projection in the second stage, the two-stage classifier achieved a word accuracy of 79.4%.>

An efficient algorithm for performing multiple pattern match in a string is described. The match algorithm combines the concept of deterministic finite state automata (DFSA) and the Boyer-Moore algorithm to achieve better performance. Experimental results indicate that in the average case, the algorithm is able to perform pattern match operations sublinearly, i.e. it does not need to inspect every character of the string to perform pattern match operations. The analysis shows that the number of characters to be inspected decreases as the length of patterns increases, and increases slightly as the total number of patterns increases. To match an eight-character pattern in an English string using the algorithm, only about 17% of all characters of the strong and 33% of all characters of the string, when the number of patterns is seven, are inspected. In an actual testing, the algorithm running on SUN 3/160 takes only 3.7 s to search seven eight-character patterns in a 1.4-Mbyte English text file.>

In natural language processing, ambiguity resolution is a central issue, and can be regarded as a preference assignment problem. In this paper, a Generalized Probabilistic Semantic Model (GPSM) is proposed for preference computation. An effective semantic tagging procedure is proposed for tagging semantic features. A semantic score function is derived based on a score function, which integrates lexical, syntactic and semantic preference under a uniform formulation. The semantic score measure shows substantial improvement in structural disambiguation over a syntax-based approach.

To enhance the performance of spoken language processing, a unified framework is proposed to integrate speech and language information together. This framework uses probabilistic formulation to characterize different language analyses generated from a language processing module. As probabilistic formulations are used in both speech and language processing modules, information from both modules can be easily integrated. To further improve the performance, a discrimination and robustness oriented learning procedure is proposed to adjust the parameters of probabilistic formulations. Significant improvement has been observed in the task of reading aloud Chinese computer manuals, which operates in a speaker dependent, isolated word mode.>

Two algorithms, a weighted hidden Markov model (HMM) algorithm and a subspace projection algorithm, are proposed to address some of the discrimination and robustness issues for HMM-based speech recognition. A robust two-stage classifier is also proposed to enhance the discrimination capability of the classifiers in each of the two stages so that the overall discrimination power is improved. The proposed algorithms were evaluated using a highly confusable vocabulary consisting of the nine English E-set letters. The test was conducted in a multi-speaker, isolated-word mode. The average word accuracy for the original HMM-based system was 61.7%. When the weighted HMM and the subspace projection methods were incorporated, the word accuracy improved to 74.9% and 76.4%, respectively. By incorporating the weighted HMM in the first stage and the subspace projection in the second stage, the two-stage classifier achieved a word accuracy of 79.4%.>