Abstract Many fundamental tasks in natural language processing (NLP) such as part-of-speech tagging, text chunking, and named-entity recognition can be formulated as sequence labeling problems. Although neural sequence labeling models have shown excellent results on standard test sets, they are very brittle when presented with misspelled texts. In this paper, we introduce an adversarial training framework that enhances the robustness against typographical adversarial examples. We evaluate the robustness of sequence labeling models with an adversarial evaluation scheme that includes typographical adversarial examples. We generate two types of adversarial examples without access (black-box) or with full access (white-box) to the target model’s parameters. We conducted a series of extensive experiments on three languages (English, Thai, and German) across three sequence labeling tasks. Experiments show that the proposed adversarial training framework provides better resistance against adversarial examples on all tasks. We found that we can further improve the model’s robustness on the chunking task by including a triplet loss constraint.

Abstract To achieve lifelong language learning, pseudo-rehearsal methods leverage samples generated from a language model to refresh the knowledge of previously learned tasks. Without proper controls, however, these methods could fail to retain the knowledge of complex tasks with longer texts since most of the generated samples are low in quality. To overcome the problem, we propose three specific contributions. First, we utilize double language models, each of which specializes in a specific part of the input, to produce high-quality pseudo samples. Second, we reduce the number of parameters used by applying adapter modules to enhance training efficiency. Third, we further improve the overall quality of pseudo samples using temporal ensembling and sample regeneration. The results show that our framework achieves significant improvement over baselines on multiple task sequences. Also, our pseudo sample analysis reveals helpful insights for designing even better pseudo-rehearsal methods in the future.

Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjorn, Boonserm Kijsirikul, Peerapon Vateekul. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Sequential tagging tasks, such as Part-Of-Speech (POS) tagging and Named-Entity Recognition, are the building blocks of many natural language processing applications. Although prior works have reported promising results in standard settings, they often underperform on non-standard text, such as microblogs and social media. In this article, we introduce an adversarial evaluation scheme for the Thai language by creating adversarial examples based on known spelling errors. Furthermore, we propose novel methods including UNK masking, condition initialization with affixation embeddings, and untied-directional self-attention mechanism to enhance robustness and interpretability of the neural networks. We conducted experiments on two Thai corpora: BEST2010 and ORCHID. Our adversarial evaluation schemes reveal that bidirectional LSTM (BiLSTM) do not perform well on adversarial examples. Our best methods match the performance of the BiLSTM baseline model and outperform it on adversarial examples.

BACKGROUND: The Bacteria Biotope (BB) task is a biomedical relation extraction (RE) that aims to study the interaction between bacteria and their locations. This task is considered to pertain to fundamental knowledge in applied microbiology. Some previous investigations conducted the study by applying feature-based models; others have presented deep-learning-based models such as convolutional and recurrent neural networks used with the shortest dependency paths (SDPs). Although SDPs contain valuable and concise information, some parts of crucial information that is required to define bacterial location relationships are often neglected. Moreover, the traditional word-embedding used in previous studies may suffer from word ambiguation across linguistic contexts. RESULTS: Here, we present a deep learning model for biomedical RE. The model incorporates feature combinations of SDPs and full sentences with various attention mechanisms. We also used pre-trained contextual representations based on domain-specific vocabularies. To assess the model's robustness, we introduced a mean F1 score on many models using different random seeds. The experiments were conducted on the standard BB corpus in BioNLP-ST'16. Our experimental results revealed that the model performed better (in terms of both maximum and average F1 scores; 60.77% and 57.63%, respectively) compared with other existing models. CONCLUSIONS: We demonstrated that our proposed contributions to this task can be used to extract rich lexical, syntactic, and semantic features that effectively boost the model's performance. Moreover, we analyzed the trade-off between precision and recall to choose the proper cut-off to use in real-world applications.

Natural disasters cause significant damage to the country as well as its citizens as we have seen in the news. Drought, wild fire, earthquake and flooding are some examples of the primary natural disasters occurred in Thailand. In this research, we focus on "flooding" and use data from Twitter, where users' mobile devices are utilized as IoT input channels. The goal of this work is to analyze near real-time data (tweets) for early flood warning. Traditional methods in processing, analyzing and reporting a flooding event take quite some time. In social medias (through cellphones), on the other hand, by harvesting crowdsources, potential flooding can be predicted faster-though with the price of reliability of the retrieved tweets. In our research, several techniques are incorporated in order to maximize the accuracy of results, including, tokenization, geo-encoding and decoding, NLP via string matching (Levenshtein's algorithms), and Google APIs for visualization. Finally, the dynamic yet user-friendly map is produced with respect to the posted relevant tweets along their associated frequencies.

Nowadays, text classification is recognized as one of crucial tools for business users to gain more insights from customers. However, textual data from customers such as comments are usually short. Thus, there are two main issues in short text categorization: (1) insufficient contextual information and (2) noisy data due to misspellings. Recently, there is a prior attempt to propose a deep learning approach for short text categorization by expanding semantic using word embeddings clustering via an algorithm based on density peaks searching. Since the number of words in word embeddings is usually large, the clustering algorithm does not scale with the size of data set, thus demanding unacceptable computational cost. In this paper, we aim to propose a fast short-text categorization framework. Rather than using a CNN with k-max pooling layer, we propose to use a faster version of Convolutional Neural Network (CNN) called “multichannel CNN.” To speed up the semantic expansion process, we propose to employ mini batch K-Means++, which is considerably faster and scales well with the size of data set. Furthermore, we also introduce an additional preprocessing step to increase vocabulary coverage rate on word embeddings. We conducted experiments on four public data sets: Google Snippets, TREC, MR, and Subj. The results showed that the proposed framework does not only improve an accuracy on all data sets, but also reduces computational costs from several days to a few hours.

Road map extraction is considered an essential task in GIS as its results are the basis of location-based applications in various domains. Examples include GPS navigation on cell phone, delivery route optimization and planning, tourist attraction locator, and location-based marketing. Satellite imagery, one of the big spatial data sources, is used in this research - though other types of remotely-sensed images can also be applied, such as aerial photographs from aircrafts, UAVs or drones. Despite several methods and techniques proposed and accompanied with different performance criteria, the focus was mainly on the accuracy aspect rather than the completeness of the result sets. That is, the results were said to be satisfactory if it met a certain accuracy criterion associated with some benchmark data sets, regardless of whether all results were retrieved. In many cases; however, both accuracy and completeness are equally important. In this paper, we enhance the result accuracy by incorporating connected component analysis into the method as well as the completeness performance by utilizing an ecology concept, called Landscape Metrics, which describes spatial characteristics, patterns, and correlations of areas/patches through different indices. Two types of metrics are used: shape metrics and isolation metrics. The performance is evaluated based on four criteria: precision, recall, quality, and F1 scores. The results show that more than 90% of performance is achieved in all four criteria.

HF (high frequency) coastal radar system is used to capture the surface current behavior - in terms of velocity and direction - in the ocean near the coast. 18 HF coastal radar stations were implemented along the Gulf of Thailand in order to monitor for disasters (e.g., Tsunami) as well as relevant risks. The HF systems are also to serve other life-critical applications, such as water quality control and monitoring, chemical spill backtracking, and marine navigation. However, not all the applications can benefit from this near-real-time HF data; some applications in different domains require forecast values. The examples include search-and-rescue system and hazardous materials spill trajectory prediction. Therefore, in this paper, we propose a predictive model for future current data based on historical HF coastal radar data sets, utilizing association rule mining combined with an object dispersion concept. So, the full potential of HF radar systems can be exploited. The spatial and temporal dimensions are taken into account when designing our predictive system, which consists of two phases: ocean surface current track formulation and spatio-temporal association rule mining. The experiments are performed on a two-year HF radar dataset (2014-2015) using Google Cloud Platform. The resulting forecast current values: velocity and direction are then compared with testing datasets (using 10-fold cross validation) of the actual recorded values and evaluated based on percentage accuracy and RMSE, respectively.

Hierarchical multi-label classification is a relatively new research topic in the field of classifier induction. What distinguishes it from earlier tasks is that it allows each example to belong to two or more classes at the same time, and by assumin

An interesting issue in machine learning is induction in multi-label domains where each example can be labeled with two or more classes at the same time. In a work focusing on text categorization, we followed the most commonly used approach and induced a binary classifier for each class. Analyzing the results, we noticed that performance had been impaired by two factors. First, in text domains, each class is characterized by a different set of attributes; an appropriate attribute-selection technique thus has to be applied separately to each of them. Second, the individual classes often have to be induced from imbalanced training sets, a circumstance we addressed here by majority-class undersampling. The paper provides details of the induction system and reports the results of systematic experimentation.