In the last years, several variants of transformers have emerged. In this paper, we compare different transformer-based models for solving the reverse dictionary task and explore their use in the context of a serious game called The Dictionary Game.

This paper explores the development of a first Neural Machine Translation for a low-resource language pair, Kabyle- French. Contrary to the idea that more sentences would enhance the translator's quality, our study suggests that using Back-translation on low-quality monolingual corpora can harm the translation performance. We deepen an analysis of the translator's performance to identify the optimal operational range, allowing us to use the Back-translation effectively and thus reduce the error rate during the generation of the parallel corpus. Our results highlight the central role played by the quality of the monolingual corpus in determining the effectiveness of Back-translation, highlighting the benefits of prioritizing data quality over quantity. Furthermore, we introduce a first method to evaluate the effectiveness of Back-translation, taking into account sentence length and part-of-speech tags. This evaluation demonstrates an improved performance when applying Back-translation to sentences of medium length.

Gender bias is a prevalent issue in Natural Language Processing, and it can be particularly pronounced in under-represented indigenous languages. This degrades the performance of NLP applications such as machine translation and promotes unreliable prejudices. In recent years there has been growing interest in developing methods to mitigate gender bias in NLP tasks, using lexicon, dictionary, WordNet or ConceptNet. This study explores the effectiveness of gender debiasing and Named Entity Recognition in the context of indigenous languages, such as Inuktitut, an under-represented Inuit language in Canada. Experimental results showed an improvement in the performance of Inuktitut- English Neural Machine Translation.

Human dialogues often show underlying dependencies between turns, with each interlocutor influencing the queries/responses of the other. This article follows this by proposing a neural architecture for conversation modeling that looks at the dialogue history of both sides. It consists of a generative model where one encoder feeds three decoders to process three successive turns of dialogue for predicting the next utterance, with a multidimension attention mechanism aggregating the past and current contexts for a cascade effect on each decoder. As a result, a more comprehensive account of the dialogue evolution is obtained than by focusing on a single turn or the last encoder context, or on the user side alone. The response generation performance of the model is evaluated on three corpora of different sizes and topics, and a comparison is made with six recent generative neural architectures, using both automatic metrics and human judgments. Our results show that the proposed architecture equals or improves the state-of-the-art for adequacy and fluency, particularly when large open-domain corpora are used in the training. Moreover, it allows better tracking of the dialogue state evolution for response explainability.

We present a generative neural model for open and multi-turn dialog response generation that relies on a multi-dimension attention process to account for the semantic interdependence between the generated words and the conversational history, so as to identify all the words and utterances that influence each generated response. The performance of the model is evaluated on the wide scope DailyDialog corpus and a comparison is made with two other generative neural architectures, using several machine metrics. The results show that the proposed model improves the state of the art for generation accuracy, and its multi-dimension attention allows for a more detailed tracking of the influential words and utterances in the dialog history for response explainability by the dialog history.

Chatbots or conversational agents have enjoyed great popularity in recent years. They surprisingly perform sensitive tasks in modern societies. However, despite the fact that they offer help, support, and fellowship, there is a task that is not yet mastered: dealing with complex emotions and simulating human sensations. This research aims to design an architecture for an emotional conversation agent for long-text conversations (multi-turns). This agent is intended to work in areas where the analysis of users feelings plays a leading role. This work refers to natural language understanding and response generation.

Emotion recognition and mining tasks are often limited by the availability of manually annotated data. Several researchers have used emojis and specific hashtags as forms of training and supervision.This research paper proposes a new textual and social corpus, the corpus labeled using basic emotions following Plutchik's theory. Thus, This paper propose a first study for the representation and interpretation of complex emotional interactions, using deep neural networks.

In our research, we propose a new multimodal fusion architecture for the task of sentiment analysis. The 3 modalities used in this paper are text, audio and video. Most of the current methods deal with either a feature level or a decision level fusion. In contrast, we propose an attention-based deep neural network and a training approach to facilitate both feature and decision level fusion. Our network effectively leverages information across all three modalities using a 2 stage fusion process. We test our network on the individual utterance based contextual information extracted from the CMU-MOSI Dataset. A comparison is drawn between the state-of-the-art and our network.

Indigenous languages have been very challenging when dealing with NLP tasks and applications because of multiple reasons.These languages, in linguistic typology, are polysynthetic and highly inflected with rich morphophonemics and variable dialectal-dependent spellings; which affected studies on any NLP task in the recent years.Moreover, Indigenous languages have been considered as low-resource and/or endangered; which poses a great challenge for research related to Artificial Intelligence and its fields, such as NLP and machine learning.In this paper, we propose a study on the Inuktitut language through pre-processing and neural machine translation, in order to revitalize the language which belongs to the Inuit family, a type of polysynthetic languages spoken in Northern Canada.Our focus is concentrated on: (1) the preprocessing phase, and (2) applications on specific NLP tasks such as morphological analysis and neural machine translation, both for Indigenous languages of Canada.Our evaluations in the context of lowresource Inuktitut-English Neural Machine Translation, showed significant improvements of the proposed approach compared to the state-of-the-art.

As fas as we are aware, using Sequence to Sequence algorithms for query expansion has not been explored yet in Information Retrieval literature. We tried to fill this gap in the literature with a custom Query Expansion system trained and tested on open datasets. One specificity of our engine compared to classic ones is that it does not need the documents to expand the introduced query. We test our expansions on two different tasks : Information Retrieval and Answer preselection. Our method yielded a slight improvement in performance in both two tasks . Our main contributions are :• Starting from open datasets, we built a Query Expansion training set using sentence-embeddings-based Keyword Extraction.• We assess the ability of the Sequence to Sequence neural networks to capture expanding relations in the words embeddings’ space.We afterwards started a quantitative and qualitative analysis of the weights learned by our network. In the second part, I will discuss what is learned by a Recurrent Neural Network compared to what we know about human language learning.

Grapheme-to-phoneme models are key components in automatic speech recognition and text-to-speech systems. With low-resource language pairs that do not have available and well-developed pronunciation lexicons, grapheme-to-phoneme models are particularly useful. These models are based on initial alignments between grapheme source and phoneme target sequences. Inspired by sequence-to-sequence recurrent neural network--based translation methods, the current research presents an approach that applies an alignment representation for input sequences and pretrained source and target embeddings to overcome the transliteration problem for a low-resource languages pair. Evaluation and experiments involving French and Vietnamese showed that with only a small bilingual pronunciation dictionary available for training the transliteration models, promising results were obtained with a large increase in BLEU scores and a reduction in Translation Error Rate (TER) and Phoneme Error Rate (PER). Moreover, we compared our proposed neural network--based transliteration approach with a statistical one.

Information extraction today faces new challenges with noisy, short, unstructured data. This is especially the case for social media messages, such as tweets, in which language can be erroneous or cryptic, and contains references to a great number of new entities. Traditional NLP systems are challenged and need to develop new strategies to handle with these data. With the emergence of the neural network-based approach, the research about the word segmentation has benefited from large-scale raw texts by leveraging them for pretrained character and word embeddings. To this end, we experimented the use of both character and word embeddings to provide extra features to input layer of our neural network-based system architecture. This system has been tested on both Chinese and Japanese social media datasets. With the help of rich pretrained embeddings, our model achieved the promising results both on Chinese and Japanese social media word segmentation task by comparing with the state-of-the-art NLP tools.

Since about a decade ago, deep learning has emerged as a powerful machine learning technique and produced state-of-the-art results in many application domains, ranging from computer vision and speech recognition to NLP. Applying deep learning to sentiment analysis has also become very popular recently. In this paper, we propose a comparative study for multimodal sentiment analysis using deep neural networks involving visual recognition and natural language processing. Initially we make different models for the model using text and another for image and see the results on various models and compare them.

In traditional text classification, classes are mutually exclusive, i.e. it is not possible to have one text or text fragment classified into more than one class. On the other hand, in multi-label classification an individual text may belong to several classes simultaneously. This type of classification is required by a large number of current applications such as big data classification, images and video annotation. Supervised learning is the most used type of machine learning in the classification task. It requires large quantities of labeled data and the intervention of a human tagger in the creation of the training sets. When the data sets become very large or heavily noisy, this operation can be tedious, prone to error and time consuming. In this case, semi-supervised learning, which requires only few labels, is a better choice. In this paper, we study and evaluate several methods to address the problem of multi-label classification using semisupervised learning and data from social networks. First, we propose a linguistic pre-processing involving tokenisation, recognition of named entities and hashtag segmentation in order to decrease the noise in this type of massive and unstructured real data and then we perform a word sense disambiguation using WordNet. Second, several experiments related to multi-label classification and semi-supervised learning are carried out on these data sets and compared to each other. These evaluations compare the results of the approaches considered. This paper proposes a method for combining semisupervised methods with a graph method for the extraction of subjects in social networks using a multi-label classification approach. Experiments show that the performance of the proposed model increases in 4 p.p. the precision of the classification when compared to a baseline.

The phenomenon of big data is described using five Vs: Volume, Variety, Velocity, Variability and Veracity. In this paper, we are interested by analyzing and pre-processing tweets for NLP and machine learning applications such as machine translation and classification. Collected contents from Twitter (tweets) are considered as unstructured, highly noisy and short (140 characters) texts. Overcoming these complex challenges will help learn from such data and apply traditional NLP and machine learning techniques. In this paper, we propose a pre-processing pipeline for tweets consisting of filtering part-of-speech, named entities recognition, hashtag segmentation and disambiguation. Our proposed approach is also based on the graph theory and group words of tweets using semantic relations of WordNet and the idea of connected components. Evaluations on the task of classification showed promising results when using this proposed preprocessing pipeline, with an increase in the accuracy of the classification up to 87.6%.

Multilingual linguistic resources are usually constructed from parallel corpora, but since these corpora are available only for selected text domains and language pairs, the potential of other resources is being explored as well. This article seeks to explore and to exploit the idea of using multilingual web-based encyclopedias such as Wikipedia as comparable corpora for bilingual terminology extraction. We propose an approach to extract terms and their translations from different types of Wikipedia link information and data. The next step will be using a linguistic-based information to re-rank and filter the extracted term candidates in the target language. Preliminary evaluations using the combined statistics-based and linguistic-based approaches were applied on different pairs of languages including Japanese, French and English. These evaluations showed a real open improvement and a good quality of the extracted term candidates for building or enriching multilingual ontologies, dictionaries or feeding a cross-language information retrieval system with the related expansion terms of the source query.

Statistical machine translation is quite robust when it comes to the choice of input representation. It only requires consistency between training and testing. As a result, there is a wide range of possible preprocessing choices for data used in statistical machine translation. This is even more so for morphologically rich languages such as Arabic. In this paper, we study the effect of different word-level preprocessing schemes for Arabic on the quality of phrase-based statistical machine translation. We also present and evaluate different methods for combining preprocessing schemes resulting in improved translation quality.

This paper presents an approach to bilingual lexicon extraction from comparable corpora and evaluations on Cross-Language Information Retrieval. We explore a bi-directional extraction of bilingual terminology primarily from comparable corpora. A combined statistics-based and linguistics-based model to select best translation candidates to phrasal translation is proposed. Evaluations using a large test collection for Japanese-English revealed the proposed combination of bi-directional comparable corpora, bilingual dictionaries and transliteration, augmented with linguistics-based pruning to be highly effective in Cross-Language Information Retrieval.